Description

You will own the modernization of legacy mainframe batch processes into a cloud-based architecture.

This role is hybrid.

Responsibilities

• Design and maintain high-volume data transformation logic using AWS Glue jobs written in Python.
• Develop custom code and AWS Lambda functions to handle complex batch process logic.
• Query, filter, and manipulate data using PySpark and SQL against various data stores.
• Ingest data from multiple sources, including vendors, flat files, CSVs, and APIs.
• Collaborate with mainframe experts to translate legacy system logic into modernized requirements.

Required Skills

• 7+ years of experience as a Data Engineer focusing on data pipelines and ETL/ELT processes.
• Expertise in Python, comprising 80–90% of development work.
• Proficiency in PySpark and SQL for data handling and querying.
• Hands-on experience with AWS services, specifically AWS Glue and Lambda.
• Familiarity with DevOps practices and the Git/GitHub development workflow.
• Experience with AWS databases, such as Aurora or Redshift.
• Familiarity with Java batch processes.

Key Skills

Education

Any Graduate