Description

Drive data engineering solutions using Databricks, PySpark, and cloud services to build and optimize production-grade data pipelines.

This role is remote.

Responsibilities

• Implement Medallion Architecture and Delta Live Tables (DLT) pipelines in production environments.
• Optimize Spark internals and tune query performance for large datasets.
• Design and build data ingestion processes using streaming technologies like Kafka or Event Hubs.
• Troubleshoot and debug complex data workflows across Azure and Snowflake ecosystems.

Required Skills

• 10+ years of professional experience in data engineering.
• Expert proficiency in Python and SQL.
• Deep knowledge of Databricks and PySpark.
• Experience with Snowflake and SQL.
• Familiarity with Windows Azure services, including ADLS Gen2 and Synapse.
• Understanding of data governance principles, including Unity Catalog and RBAC.
• Experience with Agile development methodologies.
• Knowledge of streaming data ingestion patterns (Kafka).

Key Skills

Education

Any Gradute