Description

You will design, build, and operate end-to-end data pipelines on AWS and Databricks, delivering production-grade data products that support analytics and self-service.

This role is remote.

Responsibilities

• Develop and optimize ETL/ELT pipelines using Python, PySpark, and SQL on Databricks and AWS services (S3, Glue, Lambda, EMR).
• Implement robust data modeling practices, including dimensional modeling, SCD patterns, and conformed dimensions for BI and analytics.
• Build and maintain infrastructure using Terraform and Docker, applying CI/CD practices for safe, automated deployments.
• Ensure data reliability through automated testing, monitoring, alerting, and data quality controls with clear SLAs.
• Collaborate with product and analytics teams to define requirements, source-to-target mappings, and data governance standards.

Required Skills

• 8+ years of experience in Data Engineering with production-grade platforms.
• Strong proficiency in Python, PySpark, and complex SQL optimization.
• Hands-on experience with AWS data services (S3, IAM, Glue, Lambda, Step Functions, CloudWatch).
• Experience with Databricks for data transformation and performance tuning.
• Proficiency in Infrastructure as Code using Terraform.
• Knowledge of CI/CD, Docker, and data modeling (star/snowflake schemas).
• Familiarity with data orchestration tools (e.g., Airflow) and version control.

Preferred Skills

• Experience with data mesh/fabric architectures, streaming data (Kafka/Kinesis), and data governance tools (Collibra, Immuta).
• AWS certifications (Developer, Solutions Architect, Data Analytics) or Databricks certifications.

Key Skills

Education

Any Graduate