Description

You will own the design, development, and optimization of scalable data solutions.

Responsibilities

• Build scalable data pipelines and optimize big data workflows integrating Databricks with cloud services.
• Execute complex ad-hoc queries using Databricks SQL to explore large Lakehouse datasets.
• Develop and maintain scalable data pipelines using Apache Spark on Databricks.
• Build ETL/ELT workflows utilizing AWS and Azure Services.
• Implement CI/CD pipelines for Databricks jobs using GitHub Actions, Azure DevOps, or Jenkins.

Required Skills

• 5+ years of experience in software/data engineering.
• 2+ years working specifically with Databricks and Apache Spark.
• Strong proficiency in Python, SQL, and PySpark.
• Deep understanding of AWS and Azure Cloud services.
• Experience with Databricks Data LakeHouse, Databricks Workflows, and Databricks SQL, DBT.
• Solid grasp of data Lakehouse and warehousing architecture (Medallion Architecture).
• Prior experience supporting AI/ML workflows, including training data pipelines.
• Familiarity with infrastructure-as-code tools like Terraform or CloudFormation.

Key Skills

Education

Bachelor’s degree