Data Engineer, Data Insights

About PayPay

PayPay is a FinTech company that has grown to over 65M (as of August 2024) users since its launch in 2018. Our team is hugely diverse with members from over 50 different countries.

OUR VISION IS UNLIMITED_

We dare to believe that we do not need a clear vision to create a future beyond our imagination. PayPay will always stay true to our roots and realize a vision (future) that no one else can imagine by constantly taking risks and challenging ourselves. With this mindset, you will be presented with new and exciting opportunities on a daily basis and have the opportunity to grow and reach new dimensions that you could never have imagined. We are looking for people who can embrace this challenge, refresh the product at breakneck speed and promote PayPay with professionalism and passion.
※ Please note that you cannot apply or be selected in parallel with PayPay Corporation, PayPay Card Corporation and PayPay Securities Corporation.

Job Description

PayPay’s growth is driving a rapid expansion of PayPay product teams and the need for a robust Data Engineering Platform to support our growing business needs is more critical than ever. We are looking for a Data Engineer for the Data Insights department. The Data Insights department's mission is to drive product improvements by engineering systems founded on a scientific understanding of user and merchant behavior.

Main Responsibilities

• Design, develop, and maintain scalable data ingestion pipelines using AWS Glue, Step Functions, Lambda, and Terraform
• Design, build, and maintain infrastructure to continuously support the improvement and deployment of ML models
• Optimize and manage large scale data pipelines to ensure high performance, reliability, and efficiency
• Implement data processing workflows using Hudi, Delta Lake, Spark, and Scala
• Maintain and enhance Lakeformation and Glue Data Catalog for effective data management and discovery
• Collaborate with cross-functional teams to ensure seamless data flow and integration across the organization
• Implement best practices for observability, data governance, security, and compliance

Qualifications

• 5+ years experience as a Data Engineer or in a similar role
• Some familiarity with building machine learning systems, or data infrastructures supporting machine learning development and deployment is preferable
• Hands-on experience with Apache Hudi, Delta Lake, Spark, and Scala
• Experience designing, building, and operating a DataLake or Data Warehouse
• Knowledge of Data Orchestration tools such as Airflow, Dagster, Prefect
• Strong expertise in AWS services, including Glue, Step Functions, Lambda, and EMR
• Familiarity with change data capture tools like Canal, Debezium, and Maxwell
• Experience with data warehousing tools like AWS Athena, BigQuery, Databricks
• Proficiency in Python and SQL (any variant), preferably experience in Scala and/or Java
• Experience with data cataloging and metadata management using AWS Glue Data Catalog, Lakeformation, or Unity Catalog
• Proficiency in Terraform for infrastructure as code (IaC)
• Overall understanding of machine learning technologies and deep learning concepts
• Strong problem-solving skills and ability to troubleshoot complex data issues
• Excellent communication and collaboration skills
• Ability to work in a fast-paced, dynamic environment and manage multiple tasks simultaneously

PayPay 5 senses

• Please refer PayPay 5 senses to learn what we value at work.

Working Conditions 

Employment Status

• Full Time

Office Location

• WFA(Work From Anywhere at Anytime)  
• Remote Job 
• You can live anywhere in Japan 

Work Hours

• Super Flex Time (No Core Time)
• In principle, 9:00am-5:45pm + 1h break (actual working hours: 7h45m + 1h break)

Holidays

• Every Sat/Sun/National holidays (In Japan)/New Year's break/Company-designated Special days

Paid leave

• Annual leave (up to 14 days in the first year, granted proportionally according to the month of employment. Can be used from the date of hire)
• Personal leave (5 days each year, granted proportionally according to the month of employment）
  ＊PayPay's own special paid leave system, which can be used to attend to illnesses, injuries, hospital visits, etc., of the employee, family members, pets, etc.

Salary

• Annual salary paid in 12 installments (monthly)
• Based on skills, experience, and abilities
• Reviewed once a year
• Special Incentive once a year *Based on company performance and individual contribution and evaluation
• Late overtime allowance
  
  ※Payroll payment can be changed to digital salary payment “PayPay Paycheck” for an amount set by you

Benefits

• Social Insurance (health insurance, employee pension, employment insurance and compensation insurance)
• 401K
• Translation/Interpretation support
• VISA sponsor + Relocation support

Other Information:

• PayPay Inside-Out (Corporate Blog) /JP
• PayPay Inside-out (Corporate Blog) /ENG
• PayPay Product Blog /JP
• PayPay Product Blog /ENG