Description

You will build and maintain scalable data processing pipelines in the cloud.

Responsibilities

• Develop data processing pipelines using AWS Glue and EMR.
• Implement Iceberg-based data lake solutions for storage and analytics.
• Develop and maintain efficient ETL workflows using PySpark.
• Orchestrate complex workflows using AWS Step Functions.
• Write Python scripts to automate processes and improve efficiency.

Required Skills

• 8+ years of experience in data engineering.
• Strong experience in PySpark for large-scale data processing.
• Hands-on expertise with AWS Glue, AWS EMR, and core AWS services.
• Knowledge of ETL workflows and data pipeline automation.
• Experience implementing Iceberg-based data lake solutions.
• Familiarity with AWS Step Functions and DynamoDB operations (GSI scan/query).
• Proficiency in writing Python scripts for automation.
• Proficient in logging, monitoring, and alerting using AWS CloudWatch.
• Bachelor's degree in Computer Science, Information Technology, or related field.

Key Skills

Education

Any Graduate