Description

You will build and maintain large-scale big data pipelines using AWS and Spark.

This role is on-site.

Responsibilities

• Develop and deploy big data pipelines using Spark, Scala, and Python.
• Build and manage data workflows within AWS Databricks and Data Lake environments.
• Perform performance tuning, troubleshooting, and debugging for Spark processes.
• Optimize and tune SQL queries specifically for Redshift environments.
• Work within an Agile Scrum framework to deliver data solutions.

Required Skills

• 5+ years of professional software engineering experience.
• Hands-on experience with AWS Databricks.
• Proficiency in Spark, PySpark, and Spark SQL.
• Strong development skills in Python, Scala, and SQL.
• Experience with AWS services including S3, Redshift, EC2, and Lambda.
• Expertise in SQL development and optimization techniques.
• Experience using notebooks such as Jupyter, Databricks, or Zeppelin.

Preferred Skills

• Background in developing and deploying big data pipelines.

Key Skills

Education

Any Graduate