Description

You will architect and maintain scalable data architectures and ETL processes to support enterprise analytics.

This role is remote.

Responsibilities

• Architect and maintain scalable data pipelines using AWS and Databricks.
• Develop and optimize ETL processes for accurate data movement across systems.
• Implement orchestration and automation for data pipelines using Databricks.
• Establish data quality checks, governance policies, and monitoring for pipeline health.
• Design and publish performant data models to enable self-service consumption.

Required Skills

• 10+ years of experience in data engineering roles.
• Proficiency in Python, PySpark, and SQL.
• In-depth knowledge of Medallion architecture for data pipelines.
• Hands-on experience with Delta Lake, Unity Catalog, Delta Sharing, and Delta Live Tables (DLT).
• Expertise in AWS services and Databricks environments.
• Experience with data modeling, schema design, and database optimization.
• Experience implementing CI/CD on Databricks using GitHub Actions and Databricks CLI.
• Strong understanding of data management principles including quality, governance, security, and privacy.

Preferred Skills

• Proven ability to troubleshoot complex data issues and optimize query performance for large-scale datasets.

Key Skills

Education

Any Graduate