Description

You own the development and maintenance of all data pipelines for the Data Engineering team.

Responsibilities

• Build and maintain data pipelines for optimal extraction, transformation, and loading from diverse data sources using Spark, Python, and Airflow.
• Collaborate with internal and external stakeholders to resolve data quality and technical issues.
• Conduct proofs of concept and technical demonstrations for customers and technical teams.
• Participate in agile ceremonies and triage production defects.

Required Skills

• 7+ years of professional experience.
• Proficiency in Python and PySpark.
• Strong command of Airflow.
• Solid understanding of Object-Oriented and Functional Programming paradigms.
• Experience working with Spark architecture.
• Advanced SQL knowledge (Oracle preferred).
• Experience processing large structured and unstructured datasets from multiple sources.
• Familiarity with software engineering best practices.
• Experience with GitHub and Jenkins.

Key Skills

Education

Any Graduate