Description

Design and build scalable, production-grade data pipelines and cloud-native data solutions across AWS and Snowflake environments.

This role is on-site.

Responsibilities

• Build scalable data pipelines using PySpark for batch and streaming processing, handling transformations, deduplication, and performance optimization.
• Develop cloud-native solutions on AWS, leveraging S3, Glue, EMR, Lambda, Step Functions, and Redshift.
• Engineer Snowflake data platforms, including warehouses, schemas, data models, Iceberg Tables, Dynamic Tables, and Materialized Views.
• Modernize legacy ETL workloads to cloud-based ELT/ETL patterns and implement orchestration using Airflow or AWS Step Functions.
• Optimize Snowflake and PySpark workloads through tuning and monitoring to ensure efficiency at scale.

Required Skills

• 8-10 years of experience as a Data Engineer.
• Advanced proficiency in Python and SQL with a focus on production-grade code.
• Strong expertise in PySpark, including partitioning, caching, and shuffle optimization.
• Demonstrated experience with Snowflake, specifically Iceberg Tables, Dynamic Tables, and Materialized Views.
• Deep knowledge of AWS services: S3, Glue, EMR, Lambda, Step Functions, and Redshift.
• Experience with orchestration tools such as Airflow or AWS Step Functions.
• Solid understanding of data pipeline design, ETL/ELT patterns, data modeling, and warehousing best practices.

Preferred Skills

• Experience with data quality, governance, and documentation including technical designs and runbooks.
• Ability to collaborate with data architects, analysts, and stakeholders to align solutions with business requirements.

Key Skills

Education

Bachelor's degree