Description
You will build and optimize data pipelines and solutions utilizing Azure data services.
Responsibilities
- Design, develop, and maintain data pipelines using Azure Data Factory for efficient processing and loading.
- Implement data transformation logic via Azure Databricks for enrichment, cleansing, and preparation.
- Manage and optimize data storage and access within Azure Data Lake Storage.
- Develop and maintain data models and cubes using Azure Analysis Services for querying and reporting.
- Utilize Azure Synapse for big data processing and implement data governance to maintain accuracy and security.
Required Skills
- 7-10 years of hands-on project development in Data Engineering.
- 5-7 years of development experience in Data Services.
- 3-4 years of experience with Azure Data Factory, Databricks, Data Lake Analytics, Analysis Services, and Synapse.
- Proficiency in Python, SQL, and Scala.
- Deep knowledge of ETL processes, data modeling, and data warehousing.
- Experience with Spark and big data processing.
- Proven ability in data pipeline processing troubleshooting.
- Expertise in data governance and data quality measures.
- Master’s degree in Computer Science, IT, or a related field, or equivalent work experience.
Preferred Skills
- Relevant Azure certifications.
- Familiarity with Azure DevOps and CI/CD pipelines for data engineering.