Description

You design and maintain ETL processes, database systems, and analytic tools for real-time and offline data processing. You integrate data from diverse sources to enable business decision-making and ensure data consistency and integrity.

This role is hybrid.

Responsibilities

• Build and optimize AWS cloud architecture using ECS, Glue, S3, Redshift, Lambda, and Step Functions.
• Develop Spark data processing applications using Python and PySpark.
• Write complex SQL queries and generate reports using Tableau.
• Translate business requirements into technical design specifications and code.
• Implement test automation, TDD practices, and CI/CD pipelines via GitHub.

Required Skills

• 10+ years of experience in data engineering roles.
• Expert proficiency in Python and PySpark for data processing.
• Strong SQL development and Tableau reporting skills.
• Hands-on experience with AWS services: Glue, Redshift, S3, Lambda, CloudFormation, IAM, and CloudWatch.
• Experience designing and building ETL pipelines.
• Proficiency with CI/CD tools, specifically GitHub.
• Ability to troubleshoot software for data integrity and consistency.

Preferred Skills

• Experience with SageMaker and foundational AWS services like Secrets Manager.
• Background in test-driven development and automated testing.

Key Skills

Education

Any Graduate