Description

Design and oversee the end-to-end delivery of scalable data engineering solutions and enterprise data platforms.

Responsibilities

• Create and maintain optimal data pipeline architecture to assemble large, complex datasets.
• Manage end-to-end delivery by investigating problem areas and working cross-functionally with product managers and stakeholders.
• Drive optimization, testing, and tooling to improve solution quality and operational effectiveness.
• Set and lead the technical vision while balancing business drivers and capacity projections.
• Train counterparts in data pipelining and preparation techniques to improve data consumption.

Required Skills

• 10-14 years of relevant experience in data engineering or architecture.
• Strong hands-on experience with Databricks and Unity Catalog.
• Expertise in the Azure Cloud platform.
• Proficiency in Python and PySpark.
• Experience with Apache Spark and orchestrating workloads on the cloud.
• Solid grasp of database engineering, design principles, and SQL.
• Experience handling large, complex datasets from various sources and databases.
• Ability to manage high-volume distributed systems in a SaaS environment.
• Bachelor's degree in Computer Science or equivalent experience.

Preferred Skills

• At least two years of experience building and leading technical engineering teams.
• Experience with Teradata, APIs, containerization, or CI/CD methods.

Key Skills

Education

Bachelor's degree