Description

Key Skills: ETL Testing, Test Automation, Azure Databricks, Python, SQL, Spark SQL, PySpark, Delta Lake, Data Validation, Data Quality Testing

Roles & Responsibilities:

• Design and develop automated testing frameworks for ETL/ELT pipelines.
• Perform end-to-end validation of data pipelines running on Azure Databricks.
• Execute source-to-target data reconciliation and validation activities.
• Develop SQL and Python-based automation scripts for data quality checks.
• Validate data transformations built using PySpark and Delta Lake.
• Perform completeness, consistency, duplicate, and null-value validations.
• Support data migration testing and reconciliation activities.
• Collaborate with development and data engineering teams in Agile environments.
• Analyze test results, identify defects, and ensure data accuracy.
• Contribute to test planning, estimation, and continuous quality improvement.

Experience Required:

• 5-7 years of experience in ETL/ELT Testing and Data Validation.
• Strong hands-on experience with Azure Databricks, notebooks, and jobs.
• Expertise in SQL/Spark SQL for data validation and reconciliation.
• Proficiency in Python for test automation and data quality testing.
• Experience testing PySpark and Delta Lake-based data pipelines.
• Strong understanding of source-to-target validation and data migration testing.
• Experience with Agile methodologies and test management practices.
• Knowledge of data quality frameworks and validation techniques is preferred.

Education: B.E., B.Tech

Key Skills

Education

Any Graduate