Description

You will design, develop, and maintain data pipelines using Databricks and PySpark, delivering high-quality solutions on cloud platforms such as Azure, AWS, or GCP.

Responsibilities

• Design and build scalable data solutions on Azure, AWS, or GCP.
• Optimize SQL queries and manage Azure SQL Database, Synapse Analytics, and Azure Data Factory.
• Develop CI/CD pipelines using Azure DevOps to streamline deployment processes.
• Ensure data integrity and accuracy through rigorous testing and validation.
• Collaborate with cross-functional teams to define and meet data requirements.

Required Skills

• 3-12 years of experience with Databricks and PySpark in enterprise environments.
• Strong experience managing cloud data platforms including Azure, AWS, or GCP.
• Experience working with large-scale ETL processes, data lakes, and data warehouses.
• Familiarity with data governance, metadata management, and data security practices.
• Exposure to Agile methodologies and DevOps practices for continuous delivery.
• Proficiency in Azure DevOps for CI/CD pipeline development.

Preferred Skills

• Experience with Synapse Analytics and Azure Data Factory.
• Background in data engineering consulting or cross-team collaboration.

Key Skills

Education

Any Graduate