Description

You will design and maintain large-scale data pipelines within a distributed AWS environment.

This role is on-site.

Responsibilities

• Build and optimize batch and streaming data applications using AWS technologies.
• Develop Terraform modules to automate infrastructure deployments.
• Lead the design and development of data infrastructure and frameworks.
• Perform root cause analysis on internal and external data to identify process improvements.
• Coordinate cross-functional projects involving internal teams, vendors, and external partners.

Required Skills

• 5+ years of experience in data engineering or ETL development.
• Proficiency in Python, PySpark, and Bash scripting.
• Hands-on experience with AWS services including Airflow, Glue, Lambda, Redshift, S3, and Athena.
• Expertise in creating Terraform modules for automation.
• Experience with Databricks and Data Lake technologies.
• Strong SQL skills and experience with Big Query, SQL Server, SSMSS, and Oracle SQL Developer.
• Ability to work with structured, semi-structured, and unstructured datasets.
• Experience performing data validation and testing.
• Bachelor's degree in a quantitative field such as statistics, software engineering, or business analytics.

Preferred Skills

• Master's degree.
• Experience with R, Quantum Metrics, or Akamai.
• CI/CD experience using Agile methodologies.

Key Skills

Education

Bachelor's degree