Description

You will own the design, development, and optimization of scalable data solutions.

This role is remote.

Responsibilities

• Build scalable data pipelines and optimize big data workflows integrating Databricks with cloud services.
• Execute complex ad-hoc queries using Databricks SQL to explore large Lakehouse datasets.
• Develop and maintain scalable data pipelines using Apache Spark on Databricks.
• Build ETL/ELT workflows utilizing AWS and Azure Services.
• Implement CI/CD pipelines for Databricks jobs using GitHub Actions, Azure DevOps, or Jenkins.

Required Skills

• 5+ years of experience in software or data engineering.
• 2+ years working specifically with Databricks and Apache Spark.
• Strong proficiency in Python, SQL, and PySpark.
• Deep understanding of AWS and Azure Cloud services.
• Experience with Databricks Data Lakehouse, Databricks Workflows, Databricks SQL, and DBT.
• Solid grasp of data Lakehouse and warehousing architecture, including Medallion Architecture.
• Prior experience supporting AI/ML workflows, including training data pipelines.
• Familiarity with infrastructure-as-code tools like Terraform or CloudFormation.

Preferred Skills

• Bachelor’s degree in Computer Science or related field.
• Experience with CI/CD pipeline implementation.

Key Skills

Education

Bachelor’s degree