Description

You will design and maintain data pipelines on Azure, integrating on-premises sources like mainframes, SQL Server, and Oracle into cloud environments.

This role is on-site.

Responsibilities

• Build and manage data ingestion workflows using Azure Data Factory and Azure Databricks.
• Develop and optimize Spark jobs in PySpark or Scala for large-scale data processing.
• Write and tune complex SQL queries (T-SQL/PL-SQL) for data transformation and analysis.
• Implement source-code versioning and promotion using Git and Jenkins.
• Manage orchestration and data movement using tools like Autosys, Oozie, and Sqoop.

Required Skills

• 10+ years of experience in data engineering and cloud platforms.
• Hands-on expertise with Azure Data Factory and Azure Databricks.
• Strong programming skills in Python, Scala, or Java.
• Proficiency in SQL, T-SQL, or PL-SQL.
• Experience ingesting data from on-premises sources (mainframe, SQL Server, Oracle).
• Familiarity with Sqoop, Hadoop, and Excel for metadata management.
• Knowledge of version control (Git) and CI/CD tools (Jenkins).

Preferred Skills

• Experience working directly with mainframe files.
• Familiarity with Agile methodologies, JIRA, and Confluence.

Key Skills

Education

Any Graduate