Description

You will build and manage scalable data pipelines using Apache Spark and Java.

This role is on-site.

Responsibilities

• Design and maintain scalable data pipelines using Java and Apache Spark.
• Implement ETL processes to ingest, transform, and store large datasets.
• Optimize data processing jobs for performance and cost-efficiency.
• Monitor pipelines to ensure high availability and troubleshoot issues.
• Develop data models and schemas while ensuring data quality and security.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Java and Apache Spark.
• Hands-on experience with AWS services: S3, EMR, Lambda, and Glue.
• Experience with Snowflake.
• Strong knowledge of SQL and NoSQL databases.
• Experience with CI/CD tools, including Jenkins and Spinnaker.
• Ability to collaborate with data scientists and analysts.

Preferred Skills

• Experience with Jules.

Key Skills

Education

Any Graduate