Pod2 - DBRX Engineer

Data Platform Development Build and maintain large-scale data lakes leveraging Databricks and related technologies. Design and implement ETL/ELT pipelines for structured and unstructured data. Develop processes for data transformation, metadata management, and workload orchestration. Integration & Migration Migrate data from traditional Data Warehouse (DWH) systems to data lakes or Snowflake. Ensure smooth integration with Azure services (ADLS, ADF, ADLA, AAS) and other cloud platforms. Performance & Scalability Optimize query performance and ensure scalability for big data workloads. Implement stream processing and message queuing for real-time data ingestion. Collaboration Work with cross-functional teams (data scientists, analysts, architects) in a dynamic environment. Provide technical guidance and documentation for stakeholders. Agile Delivery Participate in Agile ceremonies and adhere to best practices for iterative development. 3+ years in Data Engineering or Software Engineering roles. Strong proficiency in SQL, Python, PySpark, and familiarity with Scala or Java. Experience with Databricks, Snowflake, and workflow management tools. Cloud & Big Data Hands-on experience with Azure cloud services and big data stores. Understanding of ELT vs ETL patterns and when to apply each. Data Modeling Knowledge of data warehousing, analytic models, and schema design. Undergraduate degree in Computer Science, Statistics, or related field (Graduate degree preferred). Familiarity with CI/CD pipelines and DevOps practices. "