Description

You will design and implement event-driven data pipelines to migrate near real-time fraud detection systems to AWS.

This role is remote.

Responsibilities

• Build scalable ETL pipelines using AWS Glue, PySpark, and Python for data ingestion and transformation.
• Architect event-driven workflows using Amazon EventBridge, Kinesis Data Streams, and Kinesis Firehose to process inbound fraud data.
• Orchestrate complex processing logic and automate workflows using AWS Step Functions.
• Provision and manage cloud infrastructure using Terraform, ensuring scalability and cost-efficiency.
• Optimize data storage using Parquet and enable analytics via Amazon Athena, while managing Glue Crawlers for schema discovery.

Required Skills

• 5+ years of experience with Python and PySpark.
• Strong expertise in AWS services: Lambda, S3, EventBridge, Kinesis, Glue, Step Functions, Athena, and Macie.
• Proficient in building ETL pipelines and managing data lakes with Parquet.
• Hands-on experience with Terraform for Infrastructure as Code.
• Experience with CI/CD tools and automation pipelines.
• Understanding of event-driven architectures and Pub/Sub models.

Preferred Skills

• AWS Certified Developer or Solutions Architect certification.
• Experience with Kafka and Amazon Connect messaging platforms.

Key Skills

Education

Any Gradute