Mid-level Data Engineer - MIDAS Data Platform (Chennai)

Requirements

• 5-8 years of experience in data engineering or analytics engineering
  
• Strong proficiency in SQL and working knowledge of Python
  
• Hands-on experience building data pipelines using tools like Databricks, dbt, or similar
  
• Experience with AWS and its object storage
  
• Understanding of data modeling concepts including dimensional modeling and fact/dimension tables
  
• Experience with data quality validation and testing
  
• Ability to debug and troubleshoot data pipeline issues
  
• Experience with version control (Git) and basic understanding of CI/CD concepts
  
• Understanding of basic data governance: access control and audit logging
  
• Good problem-solving skills and ability to work with moderate independence
  
• Good communication skills and willingness to ask questions when blocked
  
• Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field, or equivalent practical experience
  

• Experience in financial services, fintech, or regulated industries
  
• Basic knowledge of banking domain concepts: core banking, payments, or regulatory reporting
  
• Exposure to data platforms in regulated environments (FISC Guidelines, GDPR, APPI)
  
• Hands-on experience with Databricks platform or AWS native data services
  
• Experience with performance tuning: partitioning strategies, file formats, query optimization
  
• Experience building REST APIs with Python (FastAPI, Flask, or similar)
  
• Knowledge of streaming data pipelines (Kafka, Kinesis, or similar)
  
• Basic experience with Terraform
  
• Experience with BI tools (QuickSight, Tableau, Looker, PowerBI)
  
• Experience with data visualization and dashboard design
  
• Interest in obtaining certifications (AWS Certified Data Analytics, Databricks certifications)
  
• Experience in AI development and/or experience in using AI tools to improve development processes.
  
• Money Forward recently announced our AI Strategy roadmap which focuses on improving AI-driven operational efficiencies, as well as integrating AI agents into our products to deliver better value to our users. 
  

• English: Business level
  
• TOEIC score of 700 or above(Note: If you have other qualifications or experiences demonstrating English proficiency, such as EIKEN Pre-1, EIKEN 2nd Grade (CSE score 1950+), TOEFL iBT 60+, IELTS 5.0+, or Cambridge FCE), feel free to discuss with us). For those without a TOEIC 700+ equivalent score, they will be asked to take a designated test during the interview process (generally after the first interview).
  

• Cloud Infrastructure :
  
• AWS (primary cloud platform in Tokyo region)
  
• S3 for data lake storage with VPC networking for secure connectivity
  
• AWS IAM for security and access management
  
• Data Lakehouse Architecture :
  
• Modern lakehouse architecture using Delta Lake for ACID transactions, time-travel, and schema evolution
  
• Columnar storage formats (Parquet) optimized for analytics
  
• Bronze/Silver/Gold medallion architecture for progressive data refinement
  
• Partition strategies and Z-ordering for query performance
  
• Unity Catalog for centralized governance and metadata management
  
•   Orchestration & Processing :
  
• Databricks Workflows for managed workflow orchestration
  
• Distributed data processing with Apache Spark on Databricks clusters
  
• Serverless compute and auto-scaling clusters for cost optimization
  
• Streaming and batch ingestion patterns with Databricks AutoLoader
  
• Data Transformation :
  
• dbt (data build tool) for SQL-based analytics engineering
  
• Delta Live Tables for declarative ETL pipelines with built-in data quality
  
• SQL and Python for data transformations
  
• Incremental materialization strategies for efficiency
  
• Query & Analytics :
  
• Databricks SQL for high-performance analytics queries
  
• Serverless and auto-scaling SQL warehouses for variable workloads
  
• Auto-scaling compute for variable workloads
  
• Query result caching and optimization
  
• REST APIs for data serving to downstream consumers
  
• Data Quality & Governance :
  
• Automated data quality with Delta Live Tables expectations and Great Expectations
  
• Cross-system reconciliation and validation logic
  
• Fine-grained access control with column/row-level security using Unity Catalog
  
• Automated data lineage tracking for regulatory compliance
  
• Audit logging and 10-year data retention policies
  
• Business Intelligence :
  
• Amazon QuickSight and/or Databricks SQL Dashboards
  
• Integration with enterprise BI tools (Tableau, PowerBI, Looker) 
  

• Version Control : GitHub
  
• CI/CD : GitHub Actions
  
• Infrastructure as Code : Terraform
  
• Monitoring : Databricks monitoring, AWS CloudWatch integration
  
• AI-Assisted Development : Claude Code, GitHub Copilot, ChatGPT