Description

You will design and implement scalable data and analytics architectures on Databricks and AWS, delivering secure, high-performance solutions for business intelligence and machine learning.

This role is on-site.

Responsibilities

• Design end-to-end data architectures using Databricks SQL, Delta Lake, and PySpark to support scalable reporting and advanced analytics.
• Develop secure data ingestion patterns from Amazon S3 and enterprise sources, ensuring reliable availability and optimized storage.
• Implement Databricks Workflows to orchestrate complex batch and near real-time data processing pipelines.
• Define best practices for Delta Lake table organization, including partitioning and optimization strategies for high-performance queries.
• Collaborate with BI and AI teams to design semantic layers and reusable data models using AI BI Genie capabilities.

Required Skills

• Minimum 12 years of experience in data engineering, analytics, or architecture roles.
• Extensive hands-on experience with Databricks SQL, Databricks Delta Lake, and Databricks Workflows.
• Strong proficiency in PySpark and Python for building standardized logging, error handling, and testing frameworks.
• Deep understanding of AWS S3 and cloud computing principles for large-scale data platforms.
• Experience with GenAI fundamentals and applying foundation models to augment analytics and reporting.
• Proficiency in SQL and data architecture design for enterprise environments.
• Knowledge of machine learning pipelines and data governance practices.

Preferred Skills

• Experience with cluster optimization, cost management, and mentoring data engineers.
• Familiarity with AI BI Genie capabilities and semantic layer design.

Key Skills

Education

Any Graduate