Description

Build and maintain scalable data architectures and integration pipelines within an Azure environment.

This role is on-site.

Responsibilities

• Design and maintain scalable data pipelines using Databricks and PySpark.
• Implement data integration solutions with Azure Data Factory and manage Azure Storage Accounts.
• Utilize Informatica Data Cloud Management for data governance and quality assurance.
• Manage and query data across SQL, NoSQL, DB2, and JDBC systems.
• Deploy code using GitHub, GitHub Actions, and Jenkins via CI/CD workflows.

Required Skills

• 5+ years of experience in data engineering or a related role.
• Extensive experience using Databricks as a primary data processing tool.
• Strong proficiency in PySpark for data transformation and analysis.
• Hands-on experience with Azure Data Factory and Azure Storage Accounts.
• Solid understanding of SQL, NoSQL, and DB2 databases.
• Experience with JDBC and Informatica Data Cloud Management.
• Proficiency with version control and CI/CD tools including GitHub, GitHub Actions, and Jenkins.
• Familiarity with Agile methodologies using JIRA and Confluence.

Preferred Skills

• Experience with automation in Microsoft Excel.
• Knowledge of messaging systems such as MQ.

Key Skills

Education

Any Graduate