Description

You will migrate legacy ETL pipelines from Pentaho Data Integration to Azure Data Factory and maintain scalable Python-based data processing solutions.

This role is on-site.

Responsibilities

• Analyze and document existing Pentaho ETL jobs, transformations, and data flows.
• Translate Pentaho logic into Python scripts and Azure Data Factory pipeline components.
• Develop and maintain scalable Python-based data processing solutions.
• Validate data accuracy post-migration using automated testing and SQL queries.
• Collaborate with data engineers, architects, and QA teams to troubleshoot issues.

Required Skills

• Strong proficiency in Python for data manipulation and automation.
• Hands-on experience with Pentaho Data Integration (PDI).
• Solid understanding of ETL/ELT concepts, data warehousing, and data modeling.
• Experience with SQL (joins, aggregations, subqueries).
• Familiarity with Azure Data Factory, cloud storage (Blob, Data Lake), and DevOps tools.
• Version control using Git or Azure DevOps.
• 5+ years of experience in data engineering or ETL development.
• Detail-oriented approach to data validation and strong documentation abilities.

Preferred Skills

• Experience with PowerShell for automation tasks.

Key Skills

Education

Any Graduate