Description

Lead the design and management of scalable data engineering solutions and high-volume distributed systems in Pittsburgh, PA.

Responsibilities

• Create and maintain optimal data pipeline architecture to assemble large, complex datasets.
• Manage end-to-end delivery by investigating problem areas and working cross-functionally with product managers.
• Drive optimization, testing, and tooling to improve solution quality and velocity.
• Ensure operational effectiveness regarding performance, uptime, and release management of the enterprise data platform.
• Train counterparts in data pipelining and preparation techniques.

Required Skills

• 10-14 years of relevant data engineering experience.
• Strong hands-on experience with Databricks and Unity Catalog.
• Expertise in Azure Cloud platform and orchestrating workloads on cloud.
• Advanced proficiency in Python and PySpark.
• Deep knowledge of Apache Spark and SQL.
• Solid grasp of database engineering and design principles.
• Experience handling large, complex datasets from various sources.
• Bachelor's degree in Computer Science or equivalent experience.

Preferred Skills

• At least two years of experience building and leading highly complex technical engineering teams.
• Experience with Snowflake, Teradata, APIs, containerization, and CI/CD methods.

Key Skills

Education

Bachelor's degree