Description

You will lead the design, development, and maintenance of scalable data pipelines using PySpark and Java.

Responsibilities

• Design and maintain scalable data pipelines using PySpark and Java.
• Migrate data from legacy platforms to a new AWS-based architecture.
• Write and optimize complex Spark transformations and SQL queries.
• Ensure data quality, integrity, and security across all pipelines.

Required Skills

• Strong proficiency in PySpark (70% focus) and Python.
• Solid experience in Java development.
• Advanced SQL skills for complex data querying and manipulation.
• Extensive experience with AWS cloud services, including S3, EMR, and Redshift.
• Hands-on experience building and working within Data Lakes.
• 2+ years of professional experience in data engineering roles.

Preferred Skills

• Proven experience working with Databricks.

Key Skills

Education

Any Graduate