Description

You will execute end-to-end datastore migrations from on-premises DataLake to an AWS-hosted LakeHouse within the migration Factory team.

This role is on-site.

Responsibilities

• Migrate and optimize data stores using Python and PySpark.
• Build and maintain data processing pipelines with Apache Spark and Hadoop.
• Deploy and manage infrastructure using AWS services including EMR, Redshift, Athena, and Glue.
• Implement CI/CD practices and Kubernetes deployments.

Required Skills

• 20+ years of experience in software engineering or data engineering.
• Strong proficiency in Python and PySpark.
• Hands-on experience with Apache Spark and Hadoop (HDFS/Hive).
• Experience with Apache Iceberg and Sybase IQ.
• Proficiency in Shell scripting.
• Familiarity with AWS Lambda, Step Functions, and CI/CD best practices.
• Experience with Kubernetes deployment.

Key Skills

Education

Any Gradute