Description
You will design and implement data pipelines for global and regional data synchronization using Azure SQL and Data Lake.
Responsibilities
- Build and optimize ETL processes for data anonymization, transformation, and distribution.
- Develop solutions for secure handling of PII and non-PII data to ensure GDPR compliance.
- Collaborate with software architects and DevOps to integrate data flows with application logic and deployment pipelines.
- Set up monitoring, alerting, and documentation for data processes within existing frameworks.
- Advise on best practices for data partitioning, replication, and schema evolution.
Required Skills
- 4–7 years of experience as a Data Engineer in cloud environments, preferably Microsoft Azure.
- Proficiency in Python and SQL.
- Hands-on experience with Azure SQL, Data Lake, and Data Factory.
- Experience with ETL tools and distributed data architectures.
- Knowledge of data privacy, security, and compliance regulations such as GDPR.
- Experience with data synchronization across multiple regions.
- Bachelor's degree.
Preferred Skills
- Experience with MS-SQL, Cosmos DB, Databricks, and event-driven architectures.
- Knowledge of CI/CD and infrastructure-as-code using Azure DevOps, ARM/Bicep, or Terraform.
- Prior work on global or multi-region data solutions.