Description

Lead the design, implementation, and operational management of scalable data engineering solutions on the Azure platform.

This role is on-site.

Responsibilities

• Build and maintain optimal data pipeline architectures to move workloads from development to production.
• Assemble large, complex datasets that satisfy both functional and non-functional business requirements.
• Drive optimization, testing, and tooling to increase the quality and velocity of data solutions.
• Manage end-to-end delivery by investigating problem areas and working cross-functionally with product managers and stakeholders.
• Oversee operational effectiveness regarding performance, uptime, and release management for the enterprise data platform.

Required Skills

• 7-15 years of relevant experience in data engineering.
• Expertise in Databricks and the Azure Cloud platform.
• Strong hands-on experience with PySpark, Python, and Apache Spark.
• Advanced SQL knowledge and solid grasp of database engineering and design principles.
• Experience handling large, complex datasets from various sources and databases.
• Experience orchestrating workloads on the cloud.
• Experience with Unity Catalog.
• Ability to set and lead technical vision while balancing business drivers.

Preferred Skills

• At least two years of experience building and leading highly complex technical engineering teams.
• Proficiency with APIs, containerization, and orchestration.
• Familiarity with CI/CD methods and Teradata.

Key Skills

Education

Bachelor's and/or master’s degree