Description

You will design, implement, and optimize scalable data engineering solutions and high-volume distributed systems on Azure.

This role is on-site.

Responsibilities

• Build and maintain optimal data pipeline architectures to move workloads from development to production.
• Assemble large, complex datasets that meet functional and non-functional business requirements.
• Drive optimization, testing, and tooling to improve the quality and velocity of data solutions.
• Manage end-to-end delivery by investigating problem areas and working cross-functionally with product managers and stakeholders.
• Account for operational effectiveness regarding performance, uptime, and release management of the enterprise data platform.

Required Skills

• 7-12 years of relevant experience in data engineering.
• Strong hands-on experience with Databricks and Unity Catalog.
• Expertise in Python programming and PySpark.
• Deep knowledge of Apache Spark and Azure Cloud platform.
• Proficiency in SQL and database engineering and design principles.
• Experience handling large, complex datasets from various sources and databases.
• Ability to orchestrate workloads on the cloud and use Azure Data Factory.
• Bachelor's or Master's degree in Computer Science or equivalent experience.

Preferred Skills

• At least two years of experience building and leading highly complex technical engineering teams.
• Experience with APIs, containerization, orchestration, and CI/CD methods.
• Teradata experience.

Key Skills

Education

Bachelor's or Master's degrees