Description

You will design, build, and configure data applications and pipeline architectures to meet business requirements.

None of the work mode flags are set.

Responsibilities

• Design and implement data ingestion solutions and pipeline architectures.
• Develop and optimize ETL processes using Python and PySpark.
• Build Redshift Spectrum transformations and data models using S3 data.
• Integrate datasets using AWS Glue, Lambda functions, and Airflow.
• Monitor ETL processes using CloudWatch events.

Required Skills

• 3+ years of big data ETL experience within an AWS environment.
• Proficiency in PySpark for designing, coding, and tuning big data processes.
• Hands-on experience with Python, S3, Lambda, DynamoDB, Athena, and Glue.
• Expert-level skills in writing and optimizing SQL.
• Experience building data platforms using Redshift and RDS.
• Practical knowledge of AWS services including API, CLI, and SDK.
• Experience developing data lakes, cloud data warehouses, and batch or streaming pipelines.

Preferred Skills

• Expertise in Kinesis and EC2 clusters.
• Experience orchestrating workflows using Airflow.

Key Skills

Education

Any Graduate