Description

You will design and deliver secure, scalable, and traceable data pipelines in Azure, integrating and transforming sensitive and public-facing datasets.

Responsibilities

• Build ingestion workflows and transformation logic to support reporting, analytics, and transparency initiatives.
• Develop modular, reusable pipeline components with robust error handling.
• Manage data quality through automated validation, logging, and reconciliation.
• Apply data masking, PII handling, and encryption techniques for sensitive data.
• Support data publishing for regulatory or open data requirements.

Required Skills

• 3+ years building and maintaining ETL/ELT pipelines using Azure-native tools.
• Hands-on expertise with Azure Data Factory, Dataflows, or Synapse Pipelines.
• Proficiency in SQL, Python, or PySpark for data transformation.
• Experience with Delta Lake, Azure Data Lake Storage Gen2, JSON, and Parquet formats.
• Strong grasp of DevOps/DataOps practices, including CI/CD for data pipelines.
• Knowledge of data masking, PII handling, and encryption techniques.
• Experience with data quality frameworks.
• Familiarity with public data sources and government transparency datasets.
• Any Graduate degree.

Key Skills

Education

Any Graduate