Description

Design and implement scalable data solutions using AWS and distributed computing frameworks.

Responsibilities

• Build and maintain scalable data pipelines using ETL tools and AWS services.
• Write clean, maintainable, and reusable code libraries for data processing.
• Implement automated testing using PyTest to ensure code quality.
• Manage deployment workflows through AWS CI/CD pipelines.
• Apply fundamental design principles to ensure system scalability and reliability.

Required Skills

• 8 to 11 years of experience in Data Engineering.
• 6+ years of experience with PySpark.
• 5+ years of experience with AWS Glue and AWS Redshift.
• 5+ years of experience with AWS CI/CD (CodeBuild, CodeCommit, CodeDeploy, and CodePipeline).
• Strong proficiency in Python, Hadoop, Spark, and PySpark.
• Hands-on experience with AWS S3, EC2, EMR, SNS, Lambda, Step Functions, and EventBridge.
• Proficiency with version control tools such as Git, SVN, or TFS.
• Bachelor's degree or higher.

Key Skills

Education

Bachelor's degree