Your mission
- Design, develop, and maintain scalable ETL pipelines on Databricks to integrate data from various sources, including app / web products, and marketing partner data.
- Build and manage the DataLakehouse on Databricks, including migrating legacy data products. Implement data transformations and processing logic using Databricks and PySpark to ensure efficient data handling and analysis.
- Monitor and maintain the data stack for nightly and near-real-time processing of in-house tracking solutions, ensuring high performance and reliability. Continuously optimize and troubleshoot data pipelines to ensure data quality and integrity.
- Collaborate with cross-functional teams to ensure seamless data flow and integration into the DataLakehouse. Provide Tableau support for data analysts as needed.
- Stay updated with industry best practices and emerging technologies in data engineering.