Description

You will engineer repeatable ETL workflows for batch and stream processing using Databricks and Delta Live Tables.

Responsibilities

• Research and build resilient ETL workflows using Databricks notebooks and Delta Live Tables.
• Collaborate with business users to develop data products meeting domain expectations.
• Work with DBAs to ingest data from cloud and on-prem transactional databases.
• Contribute to Data Architecture by securing sensitive data and streamlining development for analysts and scientists.
• Maintain documentation for data engineering processes and ensure data quality through testing and validation.

Required Skills

• 2+ years of experience writing clean Databricks notebook code.
• 2+ years of experience with Spark, Databricks SQL, and PySpark.
• 2+ years of experience with Azure Databricks and cloud providers such as AWS, Google Cloud, or Azure.
• 2+ years of experience with Delta Live Tables and Delta Lake architecture.
• 2+ years of experience with OLTP/OLAP systems and streaming products like Azure Service Bus.
• Deep knowledge of data engineering best practices, data warehouses, and data lakes.
• 5+ years of experience in professional written and interpersonal communication.

Preferred Skills

• 7+ years of experience with object-oriented languages including C#, Java, or Python.

Key Skills

Education

Any Graduate