Description

Design and build data pipelines on Azure to support business requirements and technical designs. You will collaborate with stakeholders and data architects to translate needs into solutions, manage data ingestion and transformation, and maintain technical documentation.

This role is on-site.

Responsibilities

• Design and develop data pipelines using Microsoft Azure Data Platform services including Azure Data Factory, ADLS, Azure SQL Database, and Azure Databricks.
• Execute exploratory data analysis on structured and semi-structured datasets, generating reports and optimizing queries.
• Implement and maintain CI/CD pipelines using GitHub, Jenkins, or JGP for automated deployment.
• Write and maintain complex SQL and NoSQL jobs for analysis in MS-SQL, Oracle, and MongoDB environments.
• Manage updates, patches, and operational enhancements for on-premise or cloud-based data environments.

Required Skills

• 5+ years of experience as a Data Engineer with focus on data ingestion, transformation, and curation.
• Proficiency in Pyspark for programming and scripting tasks.
• Hands-on experience with Azure Data Factory, ADLS, Azure SQL Database, and Azure Databricks.
• Strong SQL skills with experience in MS-SQL and Oracle environments.
• Experience working with MongoDB and handling semi-structured data.
• Ability to implement CI/CD pipelines using GitHub, Jenkins, or JGP.
• Experience integrating business teams with technology solutions and legacy systems.

Preferred Skills

• Experience with Azure DevOps for pipeline management.
• Strong organizational skills with ability to manage competing priorities.

Key Skills

Education

Any Gradute