Description

You will design and build end-to-end data platforms using Databricks Lakehouse architecture to support ML/AI workloads.

This role is on-site.

Responsibilities

• Design end-to-end data platforms leveraging Databricks Lakehouse, Delta Lake, and Unity Catalog.
• Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, Delta Live Tables, and Databricks Workflows.
• Define data ingestion, transformation, governance, and consumption frameworks aligned with enterprise standards.
• Implement performance optimization strategies for notebooks, clusters, job orchestration, and data storage.
• Guide teams in building curated, governed data models using the Delta Lake Medallion Architecture.

Required Skills

• 5+ years of experience in big data architecture and engineering.
• Strong proficiency in Databricks, including Delta Lake and Unity Catalog.
• Expertise in PySpark and Spark SQL for large-scale data processing.
• Proven experience designing and implementing complex ETL/ELT pipelines.
• Proficiency in SQL for data manipulation and querying.
• Ability to develop reference architectures and solution blueprints.
• Bachelor's degree in Computer Science, Engineering, or a related field.

Key Skills

Education

Bachelor's degree