Description

Experience: 5–8 years in Data Engineering, with at least 3 years on Azure.
- Azure Data Factory: Pipelines, Data Flows (Mapping/Wrangling), Triggers, Integration Runtimes, Parameterization, Error Handling, and Retries.
- Azure Databricks: Databricks Notebooks, Jobs/Workflows, PySpark, Scala, Delta Lake, Cluster Configuration, and Optimization.
- Programming: Advanced Python or Scala (at least one), strong coding standards, modularization, and unit testing (e.g., pytest).
- SQL: Expert-level SQL (CTEs, Window Functions, Query Tuning, and working with large datasets).
- Data Storage Formats: ADLS Gen2, Delta Lake, Parquet, Partitioning, Z-Ordering, OPTIMIZE, and VACUUM.
- Ops: CI/CD, Git, Branching Strategies, and CI/CD for ADF/Databricks Environment Promotion.
- Infrastructure as Code (IaC) exposure is a plus

Key Skills

Education

Any Gradute