Description

You will design, build, and deploy data pipelines within the Azure ecosystem, managing the full lifecycle from functional design to production.

Responsibilities

• Design and develop data pipelines using Azure Data Factory V2, including pipelines, activities, datasets, and linked services.
• Establish data integration connections with heterogeneous sources such as Hadoop, Hive, Amazon S3, Salesforce, SAP, HANA, and various APIs.
• Create source-to-target documentation, including detailed transformation rules, and manage data design and metadata.
• Implement dynamic pipeline configurations and execute full lifecycle development tasks involving metadata management.
• Integrate data from diverse sources including Salesforce, SAP, HANA, and various databases into Azure storage objects.

Required Skills

• 5+ years of experience in data-focused development roles.
• Expertise in Azure Data Factory V2, including pipelines, activities, datasets, and linked services.
• Hands-on experience with Azure Data Lake Store, Azure SQL Database, Databricks, Synapse, and Azure Purview.
• Proficiency in Spark programming and writing complex SQL queries.
• Practical knowledge of CI/CD processes and version control using GIT or SVN.
• Experience working with file formats and storage objects such as SFTP, FTP, ADLS, BLOB, and Amazon S3.
• Degree in any field of study.

Preferred Skills

• Experience with Power BI report building.
• Knowledge of metadata management and data modelling tools like Erwin or ER Studio.

Key Skills

Education

Any Graduate