Description

You will build the data plumbing and ingestion pipelines required for a full-scale Databricks implementation.

Responsibilities

• Build data ingestion processes and ETL pipelines to move data from various sources into the Databricks bronze layer.
• Execute data modeling, transformations, and curation using medallion architecture.
• Migrate existing workflows from AWS to Databricks.
• Develop data models and perform curation within Databricks environments.

Required Skills

• 5+ years of experience in data engineering.
• Expertise in Databricks, Python, and SQL.
• Hands-on experience with PySpark and Spark.
• Strong understanding of ETL, data warehousing, and modeling fundamentals.
• Proven experience with cloud environments, specifically AWS.
• Ability to manage data through bronze, silver, and gold layers.
• Degree in any field.

Preferred Skills

• Experience with Delta Live Tables (DLT).
• Proficiency with Power BI for report mocking.

Key Skills

Education

Any Gradute