Description

You will design and maintain scalable data pipelines and AWS utilities to support analytics and engineering teams. This role focuses on building resilient ETL processes, managing cloud infrastructure, and optimizing data access across S3, Redshift, and Dremio.

This role is on-site.

Responsibilities

• Build scalable ETL and data pipelines using AWS Glue, PySpark, and Spark SQL.
• Design AWS utilities to query data across S3 and Redshift for downstream consumers.
• Implement and support Dremio views, reflections, and system tables to improve data discoverability.
• Manage infrastructure deployment and configuration using CloudFormation and Infrastructure as Code principles.
• Troubleshoot data pipeline failures, performance bottlenecks, and distributed computing issues.

Required Skills

• 6–8+ years of Data Engineering experience with a focus on cloud-native solutions.
• 4+ years of AWS cloud development experience, including S3, Lambda, IAM, and Step Functions.
• Strong proficiency in Python, SQL, PySpark, and Spark SQL.
• Hands-on experience implementing Dremio for data virtualization and acceleration.
• Experience with Athena and Presto for ad-hoc querying and analytics.
• Proficiency with CloudFormation and Infrastructure as Code for deployment automation.
• Strong understanding of distributed computing, scalable data models, and resilient data processing.
• Experience with Git and CI/CD workflows for code management and delivery.

Preferred Skills

• Experience with Tableau for data visualization and reporting.

Key Skills

Education

Bachelor's degree