Description

You will architect and manage data processing workflows within the Databricks ecosystem.

Responsibilities

• Manage and secure data assets using Databricks Unity Catalog.
• Develop Python and PySpark scripts for data processing and analysis.
• Implement monitoring and observability using Datadog.
• Configure CI/CD pipelines and automate workflows via GitHub Actions.
• Apply transactional and dimensional data models to complex datasets.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python and PySpark.
• Hands-on experience with Databricks architecture and design principles.
• Experience managing data assets through Unity Catalog.
• Version control expertise using GitHub.
• Experience with DevOps and CI/CD pipeline configurations.
• Practical use of GitHub Actions for workflow automation.
• Experience using Datadog for monitoring and observability.
• Familiarity with AI coding assistants such as GitHub Copilot and Databricks Assistant.

Key Skills

Education

Any Graduate