Description

You will build and maintain scalable data pipelines and ETL/ELT processes within an AWS ecosystem.

Responsibilities

• Design, develop, and optimize data pipelines using Python and AWS services including Glue, Lambda, S3, EMR, Redshift, Athena, and Kinesis.
• Implement ETL/ELT workflows to move data from diverse sources into data lakes and warehouses.
• Monitor and troubleshoot data workflows to improve performance and cost-efficiency.
• Automate repetitive engineering tasks using Python scripts and frameworks.
• Deploy data workflows using CI/CD pipelines and enforce data quality, governance, and security practices.

Required Skills

• 5+ years of experience in data engineering or a related field.
• Strong proficiency in Python, specifically using pandas, pyspark, or boto3.
• Hands-on experience with AWS Glue, S3, Redshift, Athena, Lambda, Kinesis, and IAM.
• Proficiency in SQL and experience with relational databases like PostgreSQL and MySQL.
• Experience with NoSQL databases, specifically DynamoDB.
• Proficiency with Git-based workflows and version control.
• Experience using CI/CD tools such as Jenkins, Git, or CodePipeline.
• Strong analytical and debugging skills.

Preferred Skills

• Knowledge of big data frameworks such as Hadoop and Spark.

Key Skills

Education

Any Gradute