Description

Lead the design and modernization of enterprise data platforms for banking, healthcare, and retail domains.

Responsibilities

• Architect scalable ETL/ELT pipelines using Python, PySpark, Databricks, AWS Glue, and Azure Data Factory.
• Define logical and physical data models, governance standards, and cloud-native platform blueprints across AWS and Azure.
• Implement Medallion (Bronze/Silver/Gold) Lakehouse architectures using Delta Lake, S3, ADLS Gen2, Snowflake, Redshift, and Synapse Analytics.
• Engineer large-scale distributed processing workloads using Apache Spark, EMR, Hive, and HDFS to process billions of records.
• Orchestrate complex data workflows using Apache Airflow, Databricks Workflows, and Azure triggers to ensure SLA-driven execution.

Required Skills

• 12+ years of experience in data architecture and senior data engineering.
• Deep expertise in Apache Spark, PySpark, Databricks, and EMR.
• Strong proficiency in Python for data engineering tasks.
• Hands-on experience with AWS services and Azure cloud environments.
• Advanced SQL skills, including complex joins, CTEs, window functions, and optimization.
• Experience with Snowflake, Oracle, and PostgreSQL.
• Knowledge of Data Architecture principles and enterprise-scale implementation.

Preferred Skills

• Experience building real-time streaming architectures with Apache Kafka, Kinesis, or Azure Event Hub.
• Familiarity with Hive and HDFS ecosystems.

Key Skills

Education

Bachelor's degree