Description

You will design and build data solutions on Databricks to support organizational analytics.

Responsibilities

• Design data solutions including Delta Lake, data warehouses, and data marts.
• Develop and maintain ETL processes using Databricks notebooks for streaming and batch pipelines.
• Implement and optimize data pipelines for transformation and integration using cloud-based services.
• Collaborate with stakeholders to conduct discovery exercises and develop proofs of concept.
• Perform code reviews and manage version control using Bitbucket merge processes.

Required Skills

• 9+ years of experience in data engineering.
• Expertise in Databricks for developing data lakes and data marts.
• Proficiency in Python, SQL, and Spark.
• Experience with AWS Data Lake services including AWS Glue and IAM.
• Hands-on experience with Airflow and Jenkins.
• Working knowledge of Bitbucket.
• Ability to write complex SQL queries for data retrieval and manipulation.
• Experience in database performance tuning and optimization.
• Strong understanding of system architectures, design patterns, and data modeling.

Preferred Skills

• Understanding of ETL tools such as Azure Data Factory or Informatica.

Key Skills

Education

Any Graduate