Description

You will lead the design, implementation, and optimization of scalable data engineering solutions on the Azure platform.

Responsibilities

• Create and maintain optimal data pipeline architecture to handle large, complex datasets from various sources.
• Manage end-to-end delivery by investigating problem areas and working cross-functionally with product managers and stakeholders.
• Drive optimization, testing, and tooling to improve the quality and velocity of data solutions.
• Oversee operational effectiveness regarding performance, uptime, and release management for the enterprise data platform.
• Train counterparts in data pipelining and preparation techniques to facilitate easier data consumption.

Required Skills

• 7-12 years of relevant experience in data engineering.
• Strong hands-on experience with Databricks and Unity Catalog.
• Proficiency in Python and PySpark programming.
• Deep expertise in the Azure Cloud platform and Apache Spark.
• Solid knowledge of SQL and database engineering and design principles.
• Experience orchestrating workloads on the cloud.
• Ability to handle large, complex datasets from multiple databases and sources.
• Experience working within Agile development methodologies.
• Bachelor's or Master's degree in Computer Science or equivalent experience.

Preferred Skills

• Experience building and leading highly complex technical engineering teams.
• Proficiency with APIs, containerization, and orchestration.
• Familiarity with CI/CD methods and Teradata.

Key Skills

Education

Any Graduate