Description

You will build, validate, and maintain large-scale data pipelines and distributed systems in cloud environments.

Responsibilities

• Develop data flow pipelines using Python and PySpark.
• Design and maintain distributed systems within AWS or similar cloud providers.
• Perform quality engineering for data warehouses, including Snowflake and Redshift.
• Execute ETL testing to ensure data integrity across all pipelines.

Required Skills

• 8+ years of experience in data engineering.
• Advanced Python programming proficiency.
• Strong PySpark skills, including functional programming experience.
• Hands-on experience building data flow pipelines in AWS or other cloud providers.
• Experience working with distributed systems.
• Proficiency with Snowflake and Redshift.
• Direct experience with ETL testing and Quality Engineering (QE) for data pipelines.
• Bachelor's degree or equivalent graduate qualification.

Preferred Skills

• Knowledge of test automation frameworks.

Key Skills

Education

Any Graduate