Description

Design and own large-scale enterprise data platforms on hyperscaler clouds, ensuring robust architecture for AI and analytics.

This role is on-site.

Responsibilities

• Architect and implement enterprise data platforms using Databricks, PySpark, Spark SQL, and Delta Lake.
• Design cloud-native data solutions on AWS, Azure, or GCP, leveraging services like S3, Glue, ADF, Synapse, Dataflow, and BigQuery.
• Develop and optimize Python and SQL code, enforcing engineering best practices and conducting code reviews.
• Apply dimensional modeling and modern data design patterns to support AI-powered products and business intelligence.
• Integrate Large Language Models (LLMs) and Generative AI capabilities into the data platform infrastructure.

Required Skills

• 15+ years of experience in Data Engineering and Data Architecture.
• Hands-on expertise with Databricks Platform, including Workflows.
• Deep proficiency in PySpark, Spark SQL, and Delta Lake.
• Strong cloud experience on at least one hyperscaler: AWS (S3, Glue, Redshift, Kinesis, EMR), Azure (ADF, ADLS, Synapse, Event Hubs), or GCP (Dataflow, BigQuery, Pub/Sub).
• Advanced coding skills in Python and SQL with a focus on optimization.
• Proven experience with Data Modeling and Lakehouse Architecture.
• Understanding of AI ecosystems, including LLMs, Generative AI, and AI coding assistants (e.g., GitHub Copilot, ChatGPT).

Preferred Skills

• Experience with enterprise AI platforms and modern data design patterns.
• Background in enabling advanced analytics and business intelligence solutions.

Key Skills

Education

Bachelor's degree