Description

You will architect and manage data processing workflows within the Databricks ecosystem.

This role is on-site.

Responsibilities

• Architect and manage data processing workflows using Databricks.
• Secure and manage data assets through Databricks Unity Catalog.
• Develop Python and PySpark scripts for data processing and analysis.
• Configure CI/CD pipelines and automate workflows via GitHub Actions.
• Implement monitoring and observability using Datadog.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python and PySpark.
• Hands-on experience with Databricks architecture and design principles.
• Experience managing data assets through Unity Catalog.
• Version control expertise using GitHub.
• Experience with DevOps and CI/CD pipeline configurations.
• Practical use of GitHub Actions for workflow automation.
• Experience using Datadog for monitoring and observability.

Preferred Skills

• Familiarity with AI coding assistants such as GitHub Copilot and Databricks Assistant.

Key Skills

Education

Any Graduate