Description

What You'll Do:

• Design, develop, and optimize enterprise-scale data pipelines using Azure Databricks.
• Build scalable data ingestion, transformation, and processing frameworks for analytics and AI workloads.
• Develop and maintain ETL/ELT pipelines supporting structured and unstructured data sources.
• Translate business requirements into scalable Azure-based data solutions.
• Support Enterprise Data Platform (EDP) architecture initiatives and roadmap programs.
• Design and implement data models to support reporting, analytics, and machine learning applications.
• Collaborate with business users, data scientists, architects, and engineering teams to deliver data-driven solutions.
• Build and optimize data solutions for AI/ML and Generative AI use cases.
• Implement data quality, governance, security, and compliance standards across data platforms.
• Develop scalable processing frameworks using Python, PySpark, and SQL.
• Support implementation of Retrieval-Augmented Generation (RAG) and AI-ready data architectures.
• Monitor, troubleshoot, and optimize data platform performance and reliability.
• Participate in architecture reviews and provide technical recommendations for data modernization initiatives.
• Drive Agile delivery using DevOps, DataOps, and MLOps best practices.
• Create technical documentation, design specifications, and operational runbooks.
• Mentor junior engineers and contribute to continuous improvement initiatives.

Expertise You'll Bring:

• 8 to 12 years of experience in Data Engineering and enterprise-scale data platform development.
• Strong expertise in Azure cloud services and Azure-based data architectures.
• Hands-on experience with Azure Databricks and large-scale distributed data processing.
• Strong SQL programming, query optimization, and performance tuning expertise.
• Proficiency in Python and PySpark for data engineering and transformation workloads.
• Experience building scalable ETL/ELT pipelines and modern data platforms.
• Strong understanding of data modeling, data warehousing, and lakehouse architecture concepts.
• Experience working with large-scale structured, semi-structured, and unstructured datasets.
• Knowledge of AI/ML workflows and machine learning data preparation techniques.
• Hands-on experience with Generative AI concepts, RAG architectures, vector stores, and embeddings.
• Understanding of MLOps, LLMOps, model deployment, monitoring, and lifecycle management practices.
• Experience implementing data governance, data quality, security, and compliance frameworks.
• Familiarity with Azure Data Factory, Delta Lake, and cloud-native data services.
• Strong analytical, troubleshooting, and problem-solving capabilities.
• Excellent stakeholder management and communication skills.
• Experience working in Agile delivery environments and cross-functional teams

Key Skills

Education

Any Graduate