Description

You own the design and implementation of production data pipelines.

Responsibilities

• Develop and maintain ETL processes using Python and PySpark.
• Orchestrate data workflows using AWS Glue components (jobs, crawlers, triggers).
• Model, manage, and optimize data stores within DynamoDB.
• Implement data infrastructure using IaC tools like CloudFormation or Terraform.
• Monitor and manage AWS ecosystem components including S3, Lambda, Step Functions, and CloudWatch.

Required Skills

• 12+ years of experience in data engineering.
• Expert proficiency in Python for data processing and ETL.
• Deep experience with PySpark.
• Strong expertise in DynamoDB data modeling and performance tuning.
• Hands-on experience with AWS services (Glue, S3, Lambda, Step Functions, IAM, CloudWatch).
• Proficiency in SQL and data warehousing principles.
• Experience with Infrastructure as Code (CloudFormation or Terraform).
• Familiarity with Git and CI/CD tooling.

Key Skills

Education

Any Gradute