Description

You will own the design, development, and deployment of data engineering solutions.

Responsibilities

• Design, develop, and deploy ETL/ELT pipelines pulling data from databases, APIs, and flat files.
• Build and support data pipelines utilizing the Databricks Cloud Data Platform.
• Implement data transformations, integrations, and data quality checks.
• Optimize Databricks cluster performance for cost and scalability while monitoring via Azure Cloud.
• Automate data workflows using Azure DevOps CI/CD pipelines and orchestrate jobs using Databricks Jobs and/or Azure Data Factory.

Required Skills

• 7+ years of experience in Data Engineering or a related field.
• Strong programming skills in Python, PySpark, SQL, and/or Scala.
• Hands-on experience with Databricks.
• Strong experience in Azure cloud environments.
• Experience with Databricks Jobs and Azure Data Factory.
• Solid understanding of cloud data architecture.
• Knowledge of data storage technologies: Data Lakes, RDBMS, NoSQL, Graph DBs.

Key Skills

Education

Bachelor's degree