Description

You will build and maintain enterprise-scale data pipelines using Azure Databricks, Apache Spark, and Delta Lake.

This role is hybrid.

Responsibilities

• Design and implement ETL/ELT pipelines for structured, semi-structured, and unstructured datasets.
• Develop and optimize solutions using Azure Data Lake Storage (ADLS Gen2) and Delta Lake architecture.
• Write efficient code in Python, PySpark, and Spark SQL to process big data workloads.
• Manage version control and CI/CD practices using Git.
• Apply distributed computing principles to solve complex data engineering challenges.

Required Skills

• 5+ years of professional experience in Data Engineering or Big Data development.
• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline.
• Hands-on experience with Azure Databricks and Apache Spark.
• Strong expertise in Azure Data Lake Storage (ADLS Gen2).
• Advanced proficiency in Python, PySpark, Spark SQL, and SQL.
• Experience designing enterprise-scale ETL/ELT pipelines.
• Strong understanding of Delta Lake architecture and Lakehouse concepts.
• Experience processing structured, semi-structured, and unstructured datasets.
• Knowledge of distributed computing and big data processing frameworks.

Key Skills

Education

Bachelor's degree