Description

You will ensure the quality, consistency, and accuracy of data across the enterprise analytics ecosystem.

Responsibilities

• Validate data ingestion, transformation, and load (ETL/ELT) processes built on Databricks.
• Design and execute test cases to verify data accuracy, completeness, and lineage across medallion layers.
• Monitor pipeline executions and identify data drift, schema changes, or transformation issues.
• Test semantic data models supporting BI tools for accuracy and alignment with business definitions.
• Implement automated checks for data quality dimensions like completeness, validity, and timeliness.

Required Skills

• 6+ years of experience working on Data platforms.
• Strong proficiency in SQL and experience with relational databases.
• Hands-on experience with Databricks, Spark, and Delta Lake environments.
• Experience testing data pipelines and transformations in Azure or cloud-based data platforms.
• Proficiency in Python or PySpark for developing data validation scripts.
• Experience with CI/CD tools including Azure DevOps or GitHub Actions.
• Familiarity with cloud platforms, specifically Azure.
• Knowledge of data modeling, data warehousing, and Online Analytical Processing.

Key Skills

Education

Bachelor's degree