You will design, develop, and manage end-to-end data pipelines on the Azure Cloud Platform.
Responsibilities
- Design and build scalable data solutions using Medallion Architecture.
- Develop and manage ETL/ELT pipelines for large-scale data processing.
- Manage Azure Databricks workspaces, including cluster configuration and access control.
- Translate business requirements into technical data engineering solutions.
Required Skills
- 8+ years of experience in data engineering.
- Expert proficiency in SQL.
- Strong experience with PySpark and Python for data processing.
- Hands-on experience with Azure Data Factory (ADF) and Azure Databricks.
- Proven ability to design and implement ETL/ETL Process pipelines.
- Experience with Azure Data Lake Storage.
- Familiarity with SSIS.
- Knowledge of ingestion frameworks and orchestration.