Description

You will own the end-to-end data integration lifecycle, acquiring, cleansing, transforming, and storing external vendor data within AWS environments.

Responsibilities

• Lead the design and implementation of data cleansing and transformation processes to align vendor inputs with internal standards.
• Develop data acquisition strategies that ensure data quality, integrity, and security throughout the pipeline.
• Utilize internal tools and SDKs to integrate data, focusing on scalability and performance optimization.
• Implement and manage agents for monitoring, logging, and automation within AWS environments.
• Integrate with the internal Archival Service Platform to handle data purging and lifecycle management.

Required Skills

• 5+ years of experience in data integration or related engineering roles.
• Hands-on experience with AWS services: S3, Athena, Glue, SNS, and Lambda.
• Proficiency in Java for data processing and integration logic.
• Experience deploying and managing Apache Kafka for streaming data.
• Expertise in data cleansing, transformation, and enrichment techniques.
• Ability to design scalable data acquisition and storage solutions.

Preferred Skills

• Degree in Computer Science, Information Technology, or a related field.

Key Skills

Education

Any Graduate