Description

You will drive data engineering solutions using Databricks and related cloud services.

Responsibilities

• Implement Medallion Architecture and Delta Live Tables (DLT) pipelines in production environments.
• Optimize Spark internals, focusing on query performance tuning for large datasets.
• Design and build data ingestion processes using streaming technologies like Kafka or Event Hubs.
• Troubleshoot and debug complex data workflows across Azure and Snowflake ecosystems.

Required Skills

• 10+ years of professional experience in data engineering.
• Expert proficiency in Python and SQL.
• Deep knowledge of Databricks and PySpark.
• Experience with Snowflake and SQL.
• Familiarity with Windows Azure services, including ADLS Gen2 and Synapse.
• Understanding of data governance principles, including Unity Catalog and RBAC.
• Experience with Agile development methodologies.
• Knowledge of streaming data ingestion patterns (Kafka).

Key Skills

Education

Any Gradute