Description

You will design and manage end-to-end data pipeline architectures and high-volume distributed systems within a SaaS environment.

Responsibilities

• Build, maintain, and optimize scalable data pipelines as workloads move from development to production.
• Assemble large, complex datasets that meet functional and non-functional business requirements.
• Drive optimization, testing, and tooling to improve the quality and velocity of data solutions.
• Manage operational effectiveness regarding performance, uptime, and release management of the enterprise data platform.
• Collaborate cross-functionally with product managers and stakeholders to investigate problem areas and execute via Agile methodology.

Required Skills

• 7-15 years of relevant data engineering experience.
• Strong hands-on experience with Databricks, Azure, Apache Spark, and Python.
• Expertise in PySpark and Python programming.
• Deep knowledge of SQL and database engineering and design principles.
• Proven experience handling large, complex datasets from various sources and databases.
• Experience with Unity Catalog and orchestrating workloads on cloud platforms.
• Bachelor's degree in Computer Science or equivalent experience.

Preferred Skills

• At least two years of experience building and leading highly complex technical engineering teams.
• Experience with APIs, containerization, orchestration, and CI/CD methods.
• Teradata experience.

Key Skills

Education

Bachelor's Degree