Description

You will build and maintain data platforms focused on PySpark migrations and AWS integration.

Responsibilities

• Develop and maintain data platforms using Python and Spark.
• Handle migrations to PySpark on AWS environments.
• Design and implement data pipelines to ensure data quality and integrity.
• Collaborate with cross-functional teams to design solutions based on business requirements.
• Implement and manage agents for monitoring, logging, and automation within AWS.

Required Skills

• 3+ years of experience with Python and Spark.
• Strong proficiency in PySpark for big data processing.
• Hands-on experience with AWS big data architecture.
• Proven track record in data pipeline design.
• Experience with data migration strategies.
• Ability to implement monitoring and logging solutions.
• Any graduate degree in a relevant field.

Key Skills

Education

Any graduate