Description

Own end-to-end data validation, integrity checks, and automation for ETL pipelines and migrations.

This role is hybrid.

Responsibilities

• Validate end-to-end data pipelines and ensure integrity during migration workflows.
• Automate ETL testing workflows and develop frameworks for regression testing.
• Write complex SQL queries for debugging, data retrieval, and reconciliation.
• Implement CI/CD practices for continuous testing and deployment of data pipelines.
• Optimize AWS automation workflows for scalability and performance.

Required Skills

• 7+ years of experience in ETL/data testing.
• Hands-on exposure to Databricks (Spark, Delta Lake, Notebooks).
• Strong SQL skills for validation queries, joins, aggregations, and data comparisons.
• Experience testing pipelines on AWS (S3, Glue, Redshift) or Azure (ADF, ADLS Gen2, Synapse).
• Understanding of data warehousing concepts (dimensional modeling, SCD types, medallion architecture).
• Basic scripting in Python or PySpark for test automation.
• Familiarity with test management tools (JIRA, TestRail, Azure DevOps, qTest).

Preferred Skills

• Understanding of batch and streaming data pipeline concepts.
• Experience with data quality and validation frameworks.

Key Skills

Education

Any Graduate