Description

Lead the architecture and implementation of enterprise-scale data platforms using PySpark, Apache Iceberg, and Trino.

This role is hybrid.

Responsibilities

• Architect and build large-scale distributed data pipelines using PySpark and Spark SQL.
• Design Data Lakehouse solutions with Apache Iceberg for schema evolution, time travel, and governance.
• Optimize federated query solutions using Trino across multiple data sources.
• Migrate traditional warehouses to Lakehouse architectures and drive platform modernization.
• Mentor data engineers and provide technical leadership on delivery streams.

Required Skills

• 10+ years of experience in Data Engineering and Big Data ecosystems.
• Expert knowledge of PySpark and Spark SQL.
• Strong hands-on experience with Apache Iceberg.
• Proven experience with Trino (Presto) query engine.
• Strong SQL skills and query optimization expertise.
• Experience with orchestration tools such as Airflow.
• Knowledge of file formats like Parquet, ORC, and Avro.
• Exposure to CI/CD, Git, DevOps, and Infrastructure as Code.
• Experience with cloud platforms (AWS, Azure, or GCP).

Key Skills

Education

Any Graduate