Description

You will design and maintain scalable data infrastructure on AWS, focusing on performance optimization and cost efficiency.

Responsibilities

• Build and manage ETL pipelines using AWS Glue, Step Functions, and batch processing workflows.
• Implement and optimize data lake architectures using S3, Lake Formation, and Apache Iceberg.
• Develop Python-based data processing scripts and tools for automated ingestion and transformation.
• Configure and maintain CI/CD pipelines in GitHub for seamless deployment of data platform components.
• Analyze and refine cloud infrastructure to reduce costs while maintaining high performance.

Required Skills

• 5-8 years of software development experience.
• Strong proficiency in Python for data engineering tasks.
• Hands-on experience with AWS services: S3, Glue, Step Functions, Lake Formation, and Athena.
• Experience with Apache Iceberg and data lake governance.
• Proficiency in building CI/CD pipelines using GitHub.
• Proven track record of optimizing cloud infrastructure for performance and cost.

Key Skills

Education

Any Gradute