Description

You will lead the design, implementation, and optimization of enterprise data architecture using cloud-native platforms.

Responsibilities

• Design and implement scalable, secure data architectures using Databricks, Spark, and Delta Lake.
• Develop and maintain data lakehouses to ensure performance and cost-efficiency.
• Define best practices for data ingestion, transformation, and storage via Databricks notebooks, jobs, and workflows.
• Architect solutions for both real-time and batch data processing.
• Lead migration efforts from legacy systems to modern cloud-based platforms.
• Collaborate with engineers and stakeholders to define data models, pipelines, and governance strategies.

Required Skills

• 12+ years of experience in data architecture.
• 5+ years of hands-on experience with Databricks.
• Proficiency in Apache Spark, Delta Lake, and PySpark.
• Experience with AWS and AWS Databricks.
• Strong experience with Snowflake.
• Expertise in data modelling, ETL/ELT pipelines, and data warehousing.
• Familiarity with CI/CD, DevOps, and Infrastructure as Code (Terraform, ARM templates).
• Knowledge of data governance, security, and compliance frameworks.
• Bachelor's degree.

Preferred Skills

• Databricks Certified Data Engineer or Architect.
• Experience with MLflow, Unity Catalog, and Lakehouse architecture.
• Experience with Apache Airflow, dbt, or Power BI/Tableau.

Key Skills

Education

Bachelor's degree