Description

Design, build, and deploy production-level data pipelines for ingestion and transformation.

You will own the release pipeline.

Responsibilities

• Develop data ingestion and transformation workflows using Scala or Python.
• Implement data processing solutions using AWS Glue, Lambda, and Kinesis.
• Manage data storage and querying using Aurora and Redshift.
• Deploy and maintain data pipelines using AWS Glue.

Required Skills

• 6+ years of experience designing and developing data pipelines.
• 4+ years of experience with Python and Spark.
• 3+ years of experience working with AWS technologies.
• Hands-on experience with Spark programming using PySpark or Scala.
• Production experience with AWS EMR, S3, Redshift, Lambda, API Gateway, and Kinesis.
• Proficiency in Python and Scala for data engineering tasks.

Key Skills

Education

Any Graduate