Description

Build and optimize high-scale data pipelines using Python, Apache Hudi, and Trino on Azure or AWS.

This role is remote.

Responsibilities

• Engineer and optimize batch and streaming data pipelines using Python, Apache Hudi, and Apache Trino.
• Implement resilient, low-latency architectures with event-driven design and distributed systems principles.
• Drive technical direction and mentor engineers while delivering complex, high-scale solutions as a hands-on contributor.
• Ensure data quality and operational excellence through CI/CD integration, automated testing, and observability tools.
• Collaborate with product, analytics, and AI teams to make data a strategic asset.

Required Skills

• 10+ years of professional experience in software or data engineering.
• Minimum 4 years focused specifically on data pipelines (batch and streaming).
• Strong hands-on proficiency in Python.
• Experience with Lakehouse architectures: Apache Hudi, Azure ADLS Gen2, HDFS, Trino, Databricks, Spark.
• Cloud platform experience (AWS, Azure, or GCP) and containerized deployments.
• Fluency in data quality practices: dbt, Great Expectations, schema management.
• Operational excellence in observability: metrics, logging, tracing, and alerting.
• CI/CD integration for data workloads.

Preferred Skills

• Proven experience driving technical direction and mentoring engineers.
• Solid foundation in data governance and performance optimization.

Key Skills

Education

Any Graduate