Description

You will own the data infrastructure and automation pipelines within an AWS environment, ensuring reliable data flow from on-premises systems to the cloud.

Responsibilities

• Automate ETL processes using Python and AWS services.
• Build and maintain data infrastructure using S3, Athena, EMR, Glue, and Redshift.
• Migrate data from on-premises systems to the cloud.
• Develop SQL and Unix/Linux scripts for data processing and validation.
• Design, test, and build automation frameworks for data quality and testing.

Required Skills

• 10+ years of professional experience in data engineering or software development.
• Expertise in Python and SQL.
• Extensive experience with AWS services including S3, Athena, EMR, Glue, and Redshift.
• Proficiency in Unix/Linux shell scripting.
• Experience with ETL tools such as Ab Initio, AWS Glue, or Informatica.
• Hands-on experience with Git and Jenkins.
• Strong understanding of data modeling concepts and data warehousing.
• Bachelor's Degree in Computer Science, Data Science, or a related field.

Preferred Skills

• Knowledge of data science platforms like SageMaker.
• Familiarity with DevOps/DataOps practices, Agile methodologies, and RESTful APIs.
• Experience with test case management tools.

Key Skills

Education

Bachelor's Degree