Description

You will build and maintain production-grade ELT/ETL pipelines to ingest, transform, and curate data into trusted datasets.

This role is on-site.

Responsibilities

• Develop data processing jobs using Spark and Python, optimizing for performance and reliability.
• Implement data quality checks, reconcile defects, and partner with analysts to resolve issues.
• Build workflows and scheduling using MWAA (Airflow) and AWS-native services.
• Support day-to-day pipeline operations including monitoring, incident triage, root-cause analysis, and SLA management.
• Translate business requirements into technical tasks and deliver well-documented solutions, runbooks, and operational playbooks.

Required Skills

• 7+ years of hands-on data engineering experience building and supporting production data pipelines.
• Strong SQL skills for design, tuning, and troubleshooting with relational and analytical data stores.
• 3+ years of coding/scripting experience with Python (Java/Scala is a plus).
• 3+ years of experience on AWS services including S3, Glue, EMR, Lambda, IAM, CloudWatch, and MWAA/Airflow.
• 2+ years of experience with infrastructure-as-code and CI/CD practices (Terraform preferred).
• 2+ years of experience with Spark; streaming experience with Kafka is a plus.
• Bachelor’s degree in computer science, Engineering, Information Systems, or equivalent practical experience.

Preferred Skills

• Experience leveraging AI tools (e.g., GenAI assistants) to improve productivity while ensuring secure and responsible usage.
• Familiarity with domain areas such as Portfolio Management, Cyber Security, and ServiceNow reporting.
• Strong operational mindset with focus on observability, alerting, and continuous improvement to reduce pipeline failures.

Key Skills

Education

Bachelor's degree