Description

Lead the design and implementation of scalable, enterprise-grade data platforms using Databricks, combining technical expertise in data engineering with architectural leadership.

This role is hybrid.

Responsibilities

• Design end-to-end data architectures using the Databricks Lakehouse Platform, defining data models, storage strategies, and integration patterns.
• Architect scalable ETL/ELT pipelines and real-time streaming solutions using Spark, Delta Lake, and streaming technologies like Kafka or Kinesis.
• Lead technical discussions, design workshops, and code reviews to provide best practices and mentorship to data engineering teams.
• Define and implement governance standards, including data lineage, Unity Catalog access control, and security compliance frameworks.
• Collaborate with product owners and business leaders to translate requirements into scalable technical solutions and present architectural diagrams.

Required Skills

• 10+ years of experience in data engineering and solution architecture.
• Strong hands-on experience with Databricks, including clusters, notebooks, Delta Lake, MLflow, and Unity Catalog.
• Proficiency in Spark, Python, SQL, and distributed data processing.
• Experience with at least one major cloud platform (AWS, Azure, or GCP).
• Knowledge of streaming technologies such as Structured Streaming, Kafka, Kinesis, or EventHub.
• Experience with DevOps and CI/CD practices for data pipelines using Azure DevOps, GitHub Actions, or Jenkins.
• Ability to design large-scale data solutions covering ingestion, transformation, storage, and analytics.

Preferred Skills

• Databricks Certified Data Engineer Professional or Architect certification.
• Cloud architect certifications (AWS, Azure, or GCP).
• Experience with BI tools (Tableau, Power BI, Looker) or machine learning operations.

Key Skills

Education

Any Graduate