Description

Design, build, and maintain scalable data pipelines for ingesting, transforming, and processing large-scale datasets.

This role is on-site.

Responsibilities

• Build and optimize batch and streaming data solutions using Apache Spark (PySpark/Scala).
• Develop ETL/ELT workflows and integrate data from multiple internal and external sources.
• Implement CDC pipelines and event-driven integrations using Kafka.
• Design data models and solutions using Amazon Redshift and/or Snowflake.
• Optimize SQL queries and maintain data quality, validation, and monitoring frameworks.

Required Skills

• 8+ years of experience in Data Engineering or Big Data development.
• Strong hands-on experience with Apache Spark (PySpark and/or Scala).
• Advanced proficiency in SQL development and performance tuning.
• Experience with Amazon Redshift and/or Snowflake.
• Expertise in building ETL/ELT pipelines and orchestration workflows.
• Experience with Data Lake technologies (AWS S3, Parquet, Iceberg).
• Experience implementing Kafka-based streaming solutions and CDC pipelines.

Preferred Skills

• Experience with CI/CD and data pipeline automation.
• Exposure to modern orchestration tools (Airflow or equivalent).

Key Skills

Education

Bachelor's degree