Description

You design, develop, and maintain data pipelines using Databricks and PySpark, delivering high-quality solutions on cloud platforms like Azure, AWS, or GCP.

This role is on-site.

Responsibilities

• Build and manage data solutions on Azure, AWS, or GCP, optimizing SQL queries and handling Azure SQL Database, Synapse Analytics, and Azure Data Factory.
• Develop and maintain CI/CD pipelines using Azure DevOps to streamline deployment processes.
• Collaborate with cross-functional teams to define data requirements and ensure data integrity through rigorous testing and validation.
• Apply Data Science concepts to drive data engineering best practices and ensure accuracy.

Required Skills

• 8+ years of experience in data engineering or related fields.
• Hands-on expertise with Databricks and PySpark for big data processing.
• Strong proficiency in SQL query optimization and management of Azure SQL Database and Synapse Analytics.
• Experience deploying and managing cloud infrastructure on Azure, AWS, or GCP.
• Proven track record of developing CI/CD pipelines using Azure DevOps.
• Ability to lead initiatives and collaborate effectively with diverse technical teams.
• Bachelor’s degree in a related field or equivalent experience.

Preferred Skills

• Working knowledge of PySpark for advanced data transformations.
• Awareness of current data engineering trends and industry best practices.

Key Skills

Education

Any Graduate