Description

Key Responsibilities:

• Architecture & Platform Design Design and implement end-to-end data platform architecture (ingestion processing storage serving)
• Define batch and near real-time data pipelines ensuring low-latency and high reliability Make technology decisions across Databricks, Delta Lake, Snowflake, and Azure ecosystem
• Data Engineering & Pipelines Build scalable pipelines using: Databricks / Spark for large-scale processing Delta Lake for ACID-compliant data storage Snowflake for data warehousing and analytics Implement ETL/ELT pipelines with strong data modeling practices Streaming & Real-Time Processing
• Design and implement real-time pipelines using Kafka / Azure Event Hubs Ensure data freshness within ~15-minute SLA
• Enable incremental processing and efficient data updates Data Quality & Governance Establish data quality frameworks (validation, completeness, consistency checks) Implement monitoring, alerting, and data observability
• Define and enforce data governance, lineage, and metadata standards Data Serving & Analytics Enable optimized data layers for Power BI / Microsoft Fabric dashboards Design semantic models and curated data layers for business consumption
• Ensure consistent, accurate, and high-performance reporting Performance & Scalability Optimize pipelines and storage for large-scale datasets (TB/PB)
• Ensure low-latency query performance and efficient compute usage Implement partitioning, indexing, caching, and optimization strategies Leadership & Collaboration Lead and mentor a team of data engineers Collaborate with Technical Product Managers, BI teams, and business stakeholders Drive best practices in coding, architecture, and delivery Manage technical risks, dependencies, and roadmap execution

Required Skills:

• Strong experience in data engineering and platform architecture (Lead level)
• Expertise in: Databricks, Spark, Delta Lake Snowflake or similar cloud data warehouses Hands-on with streaming technologies (Kafka / Event Hubs)
• Strong knowledge of data modeling, ETL/ELT, and pipeline design Experience with data quality frameworks and monitoring tools
• Familiarity with Power BI / Microsoft Fabric Strong programming skills (Python, SQL) Experience with Azure ecosystem (ADF, ADLS, AKS - preferred)
• Nice to Have Experience with real-time analytics platforms Exposure to data governance / MDM frameworks
• Familiarity with CI/CD and DevOps practices for data platforms

Key Expectations:

• Own and deliver a robust, scalable data platform
• Ensure high data quality and near real-time availability (15 min SLA) Drive standardization, reusability, and performance optimization
• Enable business-ready, trusted data for analytics and decision-making
• Business Impact Build and scale a modern data platform that delivers trusted, near real-time insights, enabling faster decisions and powering analytics across the organization

Key Skills

Education

Bachelor's degree