Description

You will build and maintain data pipelines within a Databricks environment.

Responsibilities

• Build data pipelines using Databricks to process diverse datasets.
• Implement Spark structured streaming for real-time data processing.
• Manage Databricks workflows and scheduling tasks.
• Maintain code integrity using Git branching strategies and CI/CD processes.

Required Skills

• 6+ years of professional experience.
• Hands-on experience with PySpark and Spark SQL.
• Proficiency in Python and SQL.
• Working knowledge of Apache Spark and Delta Lake core concepts.
• Experience with Databricks workflows and scheduling.
• Practical knowledge of Git repositories and branching strategies.
• Experience implementing CI/CD pipelines.
• Working knowledge of public cloud platforms, specifically Azure.
• Any Graduate degree.

Key Skills

Education

Any Gradute