Description

You will design and maintain data infrastructure for batch and real-time processing of billions of records daily.

Responsibilities

• Design, build, and maintain scalable data infrastructure for high-volume data processing.
• Develop sustainable and adaptable data pipelines using Python, SQL, and Databricks.
• Build and manage Data Lakes using AWS technologies including S3, EKS, ECS, Glue, and EMR.
• Operationalize data pipelines to support advanced analytics and decision-making.
• Create optimized end-to-end solutions for fast data delivery.

Required Skills

• 10+ years of industry experience in Data Engineering with large-scale data-driven systems.
• 3+ years of experience using the AWS platform.
• 3+ years of experience with Python scripting.
• 3+ years of experience in Data Pipeline Development.
• Proficiency in SQL and Databricks.
• Experience with AWS services: EC2, S3, Redshift/Spectrum, Glue, Athena, RDS, Lambda, and API Gateway.
• Experience with AWS EKS, ECS, and EMR.

Preferred Skills

• Bachelor's degree or equivalent experience.

Key Skills

Education

Any Graduate