Description

You will build and maintain data pipelines to integrate, consolidate, and cleanse information from various source systems for analytical and operational use.

Responsibilities

• Build data pipelines to aggregate information from disparate source systems.
• Integrate, consolidate, and cleanse data to prepare it for analytical applications.
• Design effective data models by analyzing source data and user requirements.
• Translate user requirements into functional technical solutions.
• Apply dimensional and conceptual data modeling techniques to structure data.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency with Databricks and Spark.
• Hands-on experience with PySpark and Spark Structured Streaming.
• Experience with cloud architectures, specifically Microsoft Azure or AWS.
• Strong knowledge of conceptual and dimensional data modeling.
• Working experience with Agile, Scrum, and DevOps methodologies.
• Degree in any field.

Key Skills

Education

Any Gradute