Description

You will build and maintain scalable data pipelines and integration solutions within the Azure ecosystem.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks and PySpark.
• Implement data integration solutions using Azure Data Factory and manage Azure Storage Accounts.
• Utilize Informatica Data Cloud Management for data governance and quality assurance.
• Manage and query data across SQL, NoSQL, DB2, and JDBC environments.
• Automate data processes and deploy code using GitHub, GitHub Actions, and Jenkins within CI/CD workflows.

Required Skills

• 12+ years of experience in data engineering or a related role.
• Proven expertise using Databricks as a primary data processing tool.
• Strong proficiency in PySpark for data transformation and analysis.
• Hands-on experience with Azure Data Factory and Azure Storage Accounts.
• Solid understanding of SQL, NoSQL, and DB2 databases.
• Experience with Informatica Data Cloud Management.
• Proficiency with JDBC and version control via GitHub.
• Experience with CI/CD tools including GitHub Actions and Jenkins.

Preferred Skills

• Knowledge of messaging systems such as MQ.
• Experience with testing and quality assurance tools like QMetry.
• Familiarity with JIRA, Confluence, and OneDrive for project management.

Key Skills

Education

Bachelor's degree