Description

You will architect and lead large-scale data platforms, ensuring robust data quality, governance, and automation across distributed systems.

This role is on-site.

Responsibilities

• Design and implement scalable data lakehouse architectures using Delta Lake, Apache Iceberg, and cloud storage solutions.
• Architect high-throughput streaming pipelines and batch processing frameworks using Apache Spark and Kafka.
• Define and enforce data quality standards, including profiling, validation, and anomaly detection methodologies.
• Integrate quality gates into CI/CD pipelines and drive DevOps best practices across the software development lifecycle.
• Mentor technical teams and guide complex technical decisions for senior leadership and stakeholders.

Required Skills

• 15+ years of progressive software engineering experience, with at least 5 years as a Technical or Lead Data Architect.
• Expert-level proficiency in Apache Spark (PySpark, Scala, or Java) for SQL, DataFrames, streaming, and performance tuning.
• Deep understanding of distributed storage and processing: Hadoop, HDFS, S3, Delta Lake, and Apache Iceberg.
• Hands-on experience with streaming technologies like Apache Kafka or AWS Kinesis.
• Extensive experience designing solutions on AWS (EMR, Glue, Redshift), Azure (Databricks, Synapse), or GCP (Dataproc, BigQuery).
• Expert-level SQL for complex data analysis and optimization.
• Strong Python skills for data engineering, automation, and building data quality solutions.

Preferred Skills

• Experience defining and implementing large-scale data automation frameworks.
• Proven track record of leading and mentoring technical teams in a high-excellence environment.

Key Skills

Education

Any Graduate