Description

You will design and implement scalable, cloud-native data pipelines to support AI and analytics initiatives.

This role is remote.

Responsibilities

• Design and deliver scalable, cloud-based data solutions in partnership with internal data engineering teams.
• Develop and maintain high-performance data pipelines using Apache Spark on Databricks.
• Execute complex ad-hoc queries using Databricks SQL to analyze large lakehouse datasets.
• Build ETL/ELT workflows leveraging AWS and Azure services.
• Implement CI/CD pipelines for Databricks jobs using GitHub Actions, Azure DevOps, or Jenkins.

Required Skills

• 5+ years of experience in software or data engineering.
• At least 2 years working with Databricks and Apache Spark.
• Strong proficiency in Python, SQL, and PySpark.
• Deep understanding of AWS and Azure Cloud services.
• Experience with Databricks Data Lakehouse, Workflows, and SQL.
• Solid grasp of data Lakehouse and warehousing architecture.
• Familiarity with infrastructure-as-code tools like Terraform or CloudFormation.
• Experience supporting AI/ML workflows, including data preparation and feature engineering.

Key Skills

Education

Bachelor's degree