Description

You will lead the design and evolution of AI and Data Products, owning the full lifecycle from ingestion to ML readiness.

Responsibilities

• Lead technical design meetings and serve as the primary technical contact for Product Owners.
• Design and implement batch and real-time data ingestion pipelines using Databricks and Azure Data Factory.
• Transform and enrich raw data using DBT and PySpark to ensure analytics readiness.
• Optimize data storage through partitioning, indexing, and Parquet or Delta Lake formats.
• Mentor junior engineers, manage technical debt, and collaborate with data scientists for ML deployment.

Required Skills

• 5+ years of experience in Data Engineering within large corporate organizations.
• Strong proficiency in Python, including Pandas and dependency management (poetry or venv).
• Expertise in building data pipelines within the Azure cloud ecosystem.
• Hands-on experience with Databricks and PySpark for big data processing.
• Experience using DBT for data transformation.
• Proficiency in SQL for query optimization.
• Practical knowledge of Azure Data Factory, Azure Data Lake Storage, and Blob Storage.
• Ability to implement role-based access controls and data encryption standards.

Preferred Skills

• Experience working within a dedicated Data/AI environment.

Key Skills

Education

Engineering Master’s degree or PhD