Description

You will design, develop, and manage data integration processes to ensure seamless data flow across the organization.

Responsibilities

• Design and implement data integration workflows using AWS Glue, EMR, Lambda, and Redshift.
• Develop and maintain data pipelines using PySpark, Apache Spark, and Python to process large datasets.
• Validate and cleanse data to ensure high quality, integrity, and efficient loading into target systems.
• Optimize data workflows for performance, scalability, and cost-efficiency on AWS infrastructure.
• Translate business requirements into technical specifications and coded data pipelines.

Required Skills

• 7-10 years of experience in data engineering, database design, and ETL processes.
• 5+ years of experience with AWS tools including S3, EMR, Glue, Athena, Redshift, Postgres, RDS, and Lambda.
• 5+ years of programming experience specifically with PySpark and Python.
• 3+ years of experience working with databases, data marts, and data warehouses.
• Proven experience in ETL development, system integration, and CI/CD implementation.
• Experience managing complex database objects to move changed data across multiple environments.
• Strong understanding of data security, privacy, and compliance.
• Bachelor's degree in Computer Science or a related field.

Preferred Skills

• Master's degree in a related field.
• Experience providing technical guidance and mentorship to junior developers.

Key Skills

Education

Bachelor's degree in Computer Science