Description

You will design and build data applications to ingest, process, and analyze large, disparate datasets using big data technologies. You own the end-to-end delivery of cloud-based data pipelines and APIs.

Responsibilities

• Build data pipelines on AWS using Glue, Aurora Postgres, EKS, Redshift, PySpark, Lambda, and Snowflake.
• Develop REST-based data APIs using Python and Lambda.
• Construct infrastructure for optimal ETL processes from various data sources using SQL and AWS big data technologies.
• Implement architectures designed to handle large-scale data organization and processing.
• Collaborate with development teams and stakeholders to provide technical solutions that enable business capabilities.

Required Skills

• Hands-on experience with AWS Glue and PySpark.
• Proficiency with Aurora Postgres and Redshift.
• Experience managing workloads on EKS.
• Practical knowledge of AWS Lambda for serverless computing.
• Strong SQL skills for data extraction and transformation.
• Ability to build REST-based APIs using Python.
• Experience with Snowflake.
• Background in designing big data architectures.
• Any graduate degree.

Preferred Skills

• 1+ years of experience in data engineering roles.

Key Skills

Education

Any graduate