Description

You will own data engineering tasks, focusing on feature design, development, validation, and pipeline maintenance.

Responsibilities

• Design and develop features for data platforms, including pipeline improvements across Cosmos (MapReducer) and Azure data services.
• Implement and maintain data governance requirements, such as data partitioning and physical access restrictions.
• Fix code security issues, manage identities/secrets, and ensure secure endpoints.
• Execute CI/CD processes, monitor pipelines, and resolve bugs and operational issues.

Required Skills

• Minimum 5 years experience with Big Data Platforms, specifically Cosmos (MapReducer) and the Azure data stack (HDI, ADLS Gen2).
• Minimum 5 years proficiency in SQL querying.
• Experience with high-level programming languages: C#, Scala, Java, or Python.
• Minimum 5 years experience in debugging, monitoring, and engineering lifecycle practices.
• Demonstrable experience with Spark and Scala within data processing environments.
• Familiarity with CI/CD processes and standard engineering deployment rules.
• Experience implementing distributed system design and architecture.

Key Skills

Education

Any Graduate