Description

You will design, implement, and maintain scalable data pipelines and complex data transformations for the Business Intelligence team.

Responsibilities

• Build and maintain optimal data pipeline architectures for extracting, transforming, and loading data from diverse sources.
• Design and implement Data Lake and Data Warehouse solutions using structured and unstructured datasets.
• Develop metadata-driven ELT/ETL frameworks for automated data ingestion and processing.
• Optimize data delivery by automating manual processes and re-designing infrastructure for scalability.
• Implement monitoring for data services platforms and maintain technical documentation.

Required Skills

• 5+ years of experience in data engineering and ETL development.
• Production experience with Azure ADLS Gen2, Databricks, Azure Data Factory, and Synapse Analytics.
• Hands-on expertise building and monitoring big data pipelines and architectures.
• Proficiency in Python, Scala, Java, or scripting via PowerShell and Bash.
• Strong command of SQL and NoSQL databases, including Azure SQL Database.
• Experience implementing CI/CD practices using Azure DevOps.
• Proven ability to work with various business systems such as ERP, HRMMS, Supply Chain, or financial solutions.
• Background in data extraction and transformation using MS SSIS and Azure Data Factory.
• Experience with data management and governance tools like Azure Purview.

Preferred Skills

• Experience using SAP Hana and/or SAP BODS.
• Working knowledge of stream processing with Azure Stream Analytics, Event Hubs, or Azure Data Explorer.

Key Skills

Education

Any Graduate