Principal Data Engineer - AI/Data Pipeline

TOP 3 REASONS TO JOIN US

• Competitive salary & benefits
• Hybrid working, flexible time
• Employee growth support

Who We're Looking For

• Someone who strongly resonates with Money Forward's vision.
• Someone who sees the potential in Money Forward's data.
• Someone committed to contributing to the team and society.

Job Description

• Since its establishment in December 2020, the Data Engineering Department has been responsible for building and maintaining the infrastructure for data analysis and MLOps, aiming to achieve an "Autonomous Back Office" and "Data Democratization."
• We have a department focused on data utilization, and we are also actively hiring and training data analysts and analytics engineers within our product divisions. Our goal is to provide a strong foundation that allows data users to focus on core themes such as analysis and ML.

Responsibilities

• Develop and operate cross-company data infrastructure.
• Manage data integration and data pipelines.
• Aggregate data from various internal products, services, and CRM tools into a data lake.
• Appropriately distribute the data collected in the data lake to analysis and ML platforms.
• Build and maintain data analysis infrastructure.
• Optimize DWH performance.
• Ensure data quality.

Your Skills and Experience

Must have

• At least 5 years of experience as a data engineer.
• Development experience with Python and SQL.
• Development and operational experience using AWS.

Nice to have

• Experience in data pipeline orchestration (e.g., Apache Airflow, Cloud Composer, or similar tools).
• Experience in developing data lake and DWH infrastructure.
• Experience in developing web applications.
• Experience in data processing with distributed systems.
• Experience in database performance tuning.
• Development and operational experience using Google Cloud.
• Experience in AI development and/or experience in using AI tools to improve development processes.
• Money Forward recently announced our AI Strategy roadmap which focuses on improving AI-driven operational efficiencies, as well as integrating AI agents into our products to deliver better value to our users.

Technology Stack

• Main languages: Python, SQL
• Cloud: AWS, Google Cloud, Azure
• ETL: AWS DMS, AWS Glue, Cloud Composer, Apache Airflow, Trocco, Databricks, Fivetran
• DWH/Analytics environment: BigQuery, Databricks

Tools Used

• Configuration management: Terraform
• CI/CD: GitHub Actions
• Monitoring and logging: Datadog, Cloud Monitoring, CloudWatch
• Project management: JIRA Cloud, Miro
• Documentation: Google Workspace

Skill Tag

#Data engineering, #DWH, #AWS, #Python, #SQL, #BigQuery, #Databricks, #Apache Airflow.