Description

Design and implement enterprise-grade data platforms using Databricks, ensuring scalability and performance. Own the end-to-end data architecture, from ingestion to consumption, leveraging distributed processing frameworks.

Responsibilities

• Architect and build scalable data solutions on Databricks using Delta Lake and Apache Spark.
• Develop robust ETL/ELT pipelines and orchestrate workflows using tools like Azure Data Factory (ADF).
• Optimize cluster configurations and performance tuning to reduce costs and improve query speed.
• Implement data security, governance, and access control policies within cloud environments.
• Collaborate with stakeholders to translate complex technical requirements into actionable data strategies.

Required Skills

• 5+ years of experience designing and implementing data platforms.
• Deep expertise in Apache Spark, Delta Lake, and distributed data processing concepts.
• Proficiency in Python, Scala, or SQL for data engineering and analytics.
• Hands-on experience with cloud platforms: Azure (ADF, ADLS), AWS (S3, Glue), or GCP (GCS, BigQuery).
• Strong understanding of data integration, ETL/ELT frameworks, and orchestration tools.
• Experience with performance tuning, cluster configuration, and cost optimization.
• Familiarity with DevOps and CI/CD practices for data platforms.
• Ability to communicate complex technical concepts to non-technical stakeholders.

Key Skills

Education

Any Gradute