Description
You will build and maintain scalable data storage solutions and schema layers within the Azure ecosystem.
Responsibilities
- Develop and troubleshoot data pipelines using distributed frameworks like Apache Spark.
- Integrate data from core platforms into centralized data warehouses or data lakes.
- Design and implement ETL and streaming pipelines using Azure Data Factory and SQL.
- Maintain high code quality through automated testing and engineering best practices.
- Establish secure systems and access models for handling sensitive data.
Required Skills
- 5+ years of hands-on experience in Azure-based Data Engineering.
- Proficiency in Python or Java and their standard data processing libraries.
- Strong expertise in SQL and T-SQL.
- Experience with Azure Synapse and Azure Data Factory.
- Hands-on experience with PySpark and Apache Spark.
- Deep understanding of Data Warehousing and relational databases including Azure and RDS.
- Proven ability to architect shared datasets and extract technical requirements.
- Any Graduate degree.