Description

What You'll Do:

• Design, develop, and maintain enterprise-scale data pipelines using Azure Databricks and PySpark.
• Build and optimize ETL/ELT frameworks for processing large-scale structured and unstructured datasets.
• Develop and manage Snowflake data warehouse solutions, including data modeling, transformation, and performance optimization.
• Design and implement scalable Lakehouse architectures using Azure and Snowflake platforms.
• Create reusable data ingestion, transformation, orchestration, and validation frameworks.
• Integrate data from various enterprise applications, databases, APIs, and cloud platforms into Azure Data Lake and Snowflake environments.
• Optimize data processing workloads for performance, scalability, reliability, and cost efficiency.
• Implement data governance, security, data quality, and monitoring frameworks across enterprise platforms.
• Collaborate with architects, business stakeholders, data analysts, and engineering teams to deliver business-driven solutions.
• Lead troubleshooting and resolution of production issues involving data pipelines and cloud-based data platforms.
• Participate in architecture reviews and contribute to data platform modernization initiatives.
• Implement CI/CD pipelines, deployment automation, and DevOps practices for data engineering workloads.
• Mentor junior engineers and promote engineering best practices across the team.

Expertise You'll Bring:

• 12-16 years of experience in Data Engineering, Cloud Data Platforms, and Enterprise Analytics solutions.
• Strong hands-on expertise in Azure Databricks, PySpark, Spark SQL, and Delta Lake.
• Proven experience with Snowflake including data loading, transformations, data sharing, and query performance optimization.
• Advanced SQL expertise including complex query development, optimization, troubleshooting, and performance tuning.
• Experience with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), Azure Key Vault, and Azure DevOps.
• Strong understanding of Data Warehousing, Dimensional Modeling, ETL/ELT architecture, and Lakehouse architecture.
• Experience developing and supporting enterprise-scale data pipelines and cloud-native analytics platforms.
• Knowledge of data governance, metadata management, security controls, and data quality frameworks.
• Experience working with Git, CI/CD pipelines, DevOps processes, and Agile delivery methodologies.
• Strong expertise in performance tuning, workload optimization, and scalable data platform design.
• Ability to lead technical discussions, architectural reviews, and engineering initiatives.
• Strong analytical, troubleshooting, debugging, and problem-solving skills.
• Excellent communication, leadership, and stakeholder management capabilities

Key Skills

Education

Any Graduate