Description

You will develop and maintain data pipelines using the Azure Databricks ecosystem.

Responsibilities

• Build and optimize data workflows using PySpark and Spark SQL.
• Implement data processing logic with Databricks Delta Live Tables.
• Integrate Azure Service Bus into existing data architectures.
• Maintain data integrity and performance within the Azure environment.

Required Skills

• 5+ years of professional experience in data engineering.
• Expertise in PySpark and Spark SQL.
• Hands-on experience with Databricks and Databricks Delta.
• Proficiency in SQL for data manipulation and analysis.
• Experience working within the Azure cloud ecosystem.
• Practical knowledge of Azure Service Bus.
• Degree in any graduate field.
• Must be a local candidate in Raleigh, NC.

Key Skills

Education

Any Graduate