Description

You will own the modernization of legacy mainframe batch processes into a cloud-based architecture.

Responsibilities

• Design, develop, and maintain high-volume data transformation logic using AWS Glue jobs written in Python.
• Develop custom code and AWS Lambda functions to handle complex batch process logic.
• Query, filter, and manipulate data using PySpark and SQL against various data stores.
• Ingest data from multiple sources, including vendors, flat files, CSVs, and APIs.
• Collaborate with mainframe experts to translate legacy system logic into modernized requirements.

Required Skills

• 7+ years of experience as a Data Engineer focusing on data pipelines and ETL/ELT processes.
• Expertise in Python (expected 80–90% of development).
• Proficiency in PySpark and SQL for data handling and querying.
• Hands-on experience with AWS services, specifically AWS Glue and Lambda.
• Familiarity with DevOps practices and the Git/GitHub development workflow.
• Experience with AWS databases, such as Aurora or Redshift.
• Experience dealing with varied data formats and sources (vendors, files, APIs).
• Familiarity with Java batch processes.

Key Skills

Education

Any Graduate