Description

You will design and maintain scalable data pipelines on AWS using PySpark and Java.

This role is on-site.

Responsibilities

• Design and maintain scalable data pipelines using PySpark and Java.
• Migrate data from legacy platforms to an AWS-based architecture.
• Write and optimize complex Spark transformations and SQL queries.
• Collaborate with data scientists and stakeholders to ensure data quality, integrity, and security.
• Build and work within data lakes.

Required Skills

• 5+ years of experience in data engineering.
• Strong proficiency in PySpark and Python.
• Solid experience in Java development.
• Advanced SQL skills for complex data querying and manipulation.
• Extensive experience with AWS cloud services, specifically S3, EMR, and Redshift.
• Deep understanding of Big Data concepts and technologies.
• Ability to articulate experience with Databricks.

Preferred Skills

• Bachelor's degree or equivalent experience.

Key Skills

Education

Any Graduate