Description

You will build and optimize data processing pipelines using large-scale data technologies.

Responsibilities

• Engineer and optimize data pipelines using Spark and PySpark.
• Implement data asset modeling, lifecycle management, and optimization strategies.
• Tune cluster performance, debug complex issues, and implement observability solutions.
• Manage data governance and implement lakehouse patterns using Delta Lake.
• Maintain and evolve CI/CD processes using Git.

Required Skills

• 8+ years of experience in Data Engineering.
• 3-5 years of experience with Databricks.
• Expertise in Spark, PySpark, and SQL for distributed processing.
• Strong experience with Data Build Tool (DBT).
• Deep knowledge of Delta Lake, governance, and lakehouse concepts.
• Experience implementing observability (logging, metrics, alerts).
• Cloud experience, preferably with Azure.
• Proficiency in Git and mature CI/CD practices.

Key Skills

Education

Any Graduate