Description

You will lead the design and development of end-to-end data pipelines for structured and unstructured data.

This role is hybrid.

Responsibilities

• Build and optimize ETL/ELT workflows using PySpark, SQL, and Azure Databricks.
• Design and implement data lakes, data warehouses, and real-time streaming solutions.
• Optimize query performance and data storage strategies for cloud environments.
• Implement CI/CD pipelines for data workflows and manage infrastructure on Azure.
• Mentor junior data engineers and manage end-to-end data engineering projects.

Required Skills

• 8+ years of experience in data engineering.
• Expertise in Python and PySpark for large-scale data processing.
• Strong proficiency in SQL for complex queries and transformations.
• Hands-on experience with Azure Databricks and the Hadoop ecosystem (HDFS, Hive, Spark).
• Experience with cloud-based data solutions, specifically Azure.
• Deep understanding of data modeling and ETL frameworks.
• Experience leading technical teams and managing project lifecycles.

Preferred Skills

• Experience with real-time processing frameworks like Kafka or Flink.
• Knowledge of MLOps, Snowflake, or Terraform/Infrastructure-as-Code.

Key Skills

Education

Any Gradute