Description

You will design and build end-to-end data pipelines for a forecasting platform, ensuring rigorous engineering standards and data integrity.

This role is remote.

Responsibilities

• Develop, test, and document scalable data pipelines.
• Build and evolve data integrations and APIs to handle increasing data volume.
• Establish data governance processes to ensure data integrity and security.
• Design, implement, and automate deployment of distributed systems for streaming event processing.
• Collaborate with analytics teams to improve data models for business intelligence tools.

Required Skills

• 7+ years of relevant work experience in data engineering.
• Strong proficiency in Python and object-oriented programming.
• Expertise in PySpark and advanced SQL querying on complex datasets.
• Strong experience with Databricks.
• Proven experience developing integrations across multiple systems and APIs.
• Experience with Azure technologies, specifically Azure Data Lake, ADF, and Azure DevOps.
• Familiarity with Agile software development methodologies.
• Experience designing and operating SQL/NoSQL database systems.

Preferred Skills

• Experience with Snowflake.
• Knowledge of Kafka.

Key Skills

Education

Any Graduate