Description

Design and maintain scalable data pipelines on Azure using Databricks and Scala. Optimize PostgreSQL and Elasticsearch data models while managing CI/CD workflows.

This role is hybrid.

Responsibilities

• Develop and optimize data pipelines using Azure Databricks and Scala.
• Build and maintain CI/CD pipelines for data applications with Azure DevOps.
• Design and tune PostgreSQL and Elasticsearch schemas for search and analytics.
• Monitor, troubleshoot, and optimize cloud infrastructure, application performance, and costs.
• Collaborate with data scientists and DevOps teams to deliver secure, high-quality data solutions.

Required Skills

• 10+ years of experience with Microsoft Azure services, including Data Lake and Azure Functions.
• Proficiency in Scala programming and Apache Spark.
• Strong experience with PostgreSQL database design and optimization.
• Hands-on experience with Elasticsearch for search and analytics.
• Experience with CI/CD tools such as Azure DevOps, GitLab, or GitHub.
• Hands-on experience with Docker containerization and Kubernetes orchestration.
• Familiarity with Agile development methodologies.

Preferred Skills

• Knowledge of data governance and security best practices.
• Azure certifications or experience with machine learning pipelines.

Key Skills

Education

Any Graduate