Description

You will design and execute automated tests for data pipelines, warehouses, and data lakes.

Responsibilities

• Design, develop, and execute automated tests for data pipelines, data warehouses, and data lakes.
• Validate data quality, accuracy, and consistency across the entire data lifecycle.
• Develop and maintain test scripts for data integration, transformation, and loading processes.
• Utilize data profiling and validation tools to resolve data quality issues.
• Collaborate with data engineers and data scientists to ensure data integrity.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in PySpark and Databricks.
• Strong SQL and data querying skills.
• Experience with Python.
• Knowledge of Shell scripting.
• Experience working within AWS or Azure environments.
• Any Graduate degree.

Key Skills

Education

Any Graduate