We are seeking an experienced Senior Data Engineer to design and build next-generation data platforms leveraging AI Copilot capabilities and Microsoft Fabric. This role focuses on enabling intelligent, scalable, and automated data workflows while supporting advanced analytics and AI-driven use cases. You will play a key role in modernizing data architecture and embedding AI-assisted development practices across the data lifecycle.
Key Responsibilities
- Design, build, and maintain scalable data pipelines and data platforms using Microsoft Fabric.
- Develop and optimize data ingestion, transformation, and orchestration workflows.
- Leverage AI Copilot tools to accelerate development, automate data engineering tasks, and improve productivity.
- Implement and manage Lakehouse architectures and unified data environments.
- Collaborate with data scientists and analysts to enable AI/ML and advanced analytics use cases.
- Ensure data quality, governance, security, and compliance across all pipelines.
- Optimize performance and cost efficiency of data workloads.
- Integrate data from multiple sources (structured, semi-structured, and streaming).
- Contribute to data modeling, metadata management, and lineage tracking.
- Support CI/CD pipelines, testing, and deployment of data solutions.
- Mentor junior engineers and promote best practices in data engineering and AI-assisted development.
Required Skills & Experience
- 6+ years of experience in data engineering or related roles.
- Strong hands-on experience with Microsoft Fabric (Data Factory, Lakehouse, Synapse, or equivalent components).
- Experience working with AI Copilot tools (e.g., GitHub Copilot, Azure AI Copilot) in development workflows.
- Proficiency in Python and/or Scala.
- Strong experience with SQL and large-scale data processing.
- Experience with Spark and distributed data systems.
- Solid understanding of data warehousing and lakehouse architectures.
- Familiarity with Azure ecosystem (Azure Data Factory, Azure Synapse, Azure Data Lake).
- Experience with data governance, security, and compliance frameworks.
- Knowledge of CI/CD pipelines, DevOps practices, and version control (Git).
- Experience enabling AI-driven data platforms or intelligent data workflows.
- Familiarity with real-time/streaming data processing.
- Knowledge of MLOps and integration with machine learning pipelines.
- Experience with Power BI or other data visualization tools.
- Exposure to enterprise data platform migrations or modernization programs.