Description

Build and maintain scalable ETL pipelines using Databricks and Apache Spark.

Responsibilities

• Build, test, and maintain scalable ETL pipelines using Databricks and Apache Spark.
• Develop data models and optimize queries for performance and efficiency.
• Implement and maintain medallion architecture across bronze, silver, and gold layers.
• Design and implement data governance practices to ensure quality and consistency.
• Troubleshoot and optimize Databricks workflows to enhance performance and reduce costs.

Required Skills

• 5+ years of experience in data engineering roles.
• Hands-on experience with Databricks for building ETL pipelines.
• Proficiency in Python and SQL for data manipulation and analysis.
• Experience with Apache Spark.
• Experience building data models and optimizing queries.
• Experience implementing medallion architecture and data governance.
• Bachelor's degree in Computer Science.

Preferred Skills

• Databricks certification.

Key Skills

Education

Bachelor's degree in Computer Science