Description

You will own the design and implementation of scalable data platforms and pipelines on Azure and Databricks.

Responsibilities

• Architect and implement data ingestion, transformation, and processing workflows for batch and real-time data streams.
• Lead development of complex ETL/ELT pipelines, ensuring quality and performance.
• Design data models for analytics and operational workloads, utilizing relational and lakehouse systems.
• Define and enforce data governance, data quality, and security best practices.
• Troubleshoot production issues and mentor junior engineers on engineering standards.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python and Spark for large-scale data processing.
• Experience with Delta Lake and ADLS.
• Knowledge of PostgreSQL for relational workloads.
• Understanding of Data Governance and Data Quality principles.
• Experience building data pipelines.

Key Skills

Education

Any Gradute