Description

The role focuses on data architecture, cloud integration, data engineering best practices, analytics enablement, and platform optimization across Azure, AWS, or GCP environments. Responsibilities • Design and implement scalable data lakehouse architectures using Databricks. • Define enterprise data models, governance standards, and security controls. • Lead migration of legacy data warehouses and ETL workloads to Databricks. • Architect batch and real-time data processing solutions using Spark. • Optimize performance, scalability, reliability, and cost efficiency. • Collaborate with business stakeholders, data engineers, data scientists, and cloud architects. • Establish CI/CD, monitoring, and operational best practices for data platforms. • Provide technical leadership and mentor engineering teams. Required Skills • 8+ years of data engineering and data platform experience. • 3+ years of hands-on Databricks architecture experience. • Strong expertise in Apache Spark, PySpark, SQL, and Delta Lake. • Experience with Azure Databricks, AWS Databricks, or GCP Databricks. • Strong expertise in streaming platforms: GCP Pub/Sub, Azure Event Hubs, AWS Kinesis, Apache Kafka • Hands-on experience with producer/consumer architecture decisions and trade-offs. • Knowledge of data lakehouse architecture and data governance. • Experience with data integration tools and orchestration frameworks. • Strong understanding of security, access management, and compliance requirements. Preferred Skills • Databricks Certified Data Engineer Professional or equivalent. • Experience with Unity Catalog, Delta Live Tables, and Databricks Workflows. • Experience with Power BI, Tableau, or other BI platforms. • Knowledge of DataOps, DevOps, and MLOps practices. Education • Bachelor's degree in Computer Science, Engineering, Information Systems, or related field. • Master's degree preferred

Education

Bachelor’s degree