Description

You will design, build, and optimize scalable data pipelines and architectures to support analytics and machine learning initiatives.

Responsibilities

• Design and maintain efficient data pipelines using Azure Data Factory, Databricks, and Apache Spark.
• Develop ETL/ELT workflows to ingest and transform structured and unstructured data.
• Integrate cloud-native services including Azure Synapse, ADLS, Azure SQL, and Cosmos DB into enterprise platforms.
• Implement data governance, quality, and security best practices across all engineering processes.
• Monitor and troubleshoot pipeline infrastructure to ensure reliability and performance.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python, SQL, and Spark.
• Hands-on experience with Azure Data Factory, Databricks, Synapse, ADLS, and Azure DevOps.
• Strong knowledge of Scala, Azure Storage, Azure Web Jobs, and Azure Functions.
• Experience with Azure Cosmos DB, Azure SQL, and Azure Synapse Analytics.
• Deep understanding of data modeling, warehousing, and distributed computing.
• Practical experience with CI/CD pipelines and version control using Git and GitHub Actions.
• Familiarity with data lakehouse architecture and modern data stack principles.
• Competency in C#, Flexcube technical PL/SQL, and Generative AI/OpenAI/Vector DB.
• Ability to use Power BI for data visualization.

Preferred Skills

• Experience with MLOps, Kafka, and Event Hubs for real-time data streaming.
• Exposure to data cataloging tools and metadata management.

Key Skills

Education

Any Graduate