Description

You will migrate Python-based ETL jobs and Looker views to Spark ETL jobs on Databricks.

Responsibilities

• Migrate underlying Looker views to Spark ETL jobs running on Databricks.
• Update corresponding tables in the Data Lake through regular ETL job execution.
• Collaborate with the team to perform QA on new tables.
• Ensure data quality across all migrated datasets.

Required Skills

• 3+ years of experience with PySpark.
• Hands-on experience with Databricks.
• Proficiency in SQL and Data Warehouse (DWH) concepts.
• Experience working within AWS environments.
• Bachelor's degree in Computer Science.
• Strong communication skills.

Key Skills

Education

Bachelor's degree in Computer Science