Description

Lead data engineering efforts by designing, building, and optimizing large-scale ETL pipelines and data processing workflows.

This role is on-site.

Responsibilities

• Design and maintain robust ETL pipelines using Python and modern tooling.
• Build and optimize data processing workflows with PySpark or Apache Spark for large-scale datasets.
• Write complex SQL for extraction, transformation, and loading; perform query optimization and performance tuning.
• Model and maintain relational data structures in PostgreSQL, supporting schema evolution.
• Implement and manage data lake or storage solutions, supporting distributed querying across data sources.

Required Skills

• 6+ years of experience in data engineering, ETL, and pipeline development.
• Strong proficiency in Python for data processing, automation, and integration.
• Hands-on experience with PySpark or Apache Spark for distributed data transformations.
• Expert-level SQL skills, including complex queries and optimization.
• Experience managing relational databases, specifically PostgreSQL.
• Experience with at least one major cloud platform: AWS, Azure, or GCP.
• Proven ownership of ETL pipeline development and operational data workflows.

Preferred Skills

• Experience with Apache Iceberg, Delta Lake, or Apache Hudi for data lake table management.
• Familiarity with distributed SQL engines like Trino or Presto for cross-source querying.

Key Skills

Education

Any Graduate