Description

You will design, develop, and optimize scalable data pipelines using Azure Databricks.

Responsibilities

• Lead the design, development, and enhancement of complex data pipelines using Azure Databricks.
• Write clean, maintainable, and efficient PySpark and Python code for data ingestion, transformation, and analytics.
• Troubleshoot and resolve data pipeline and workflow issues to ensure high availability.
• Collaborate with architects, analysts, and stakeholders to gather requirements and deliver scalable data solutions.
• Document data processes, architectures, and workflows for knowledge sharing.

Required Skills

• 7+ years of professional experience.
• Deep expertise with Azure Databricks.
• Strong proficiency in Python programming.
• Extensive experience with PySpark for large-scale data processing.
• Proven track record in ETL development.

Preferred Skills

• Experience with data architecture patterns.
• Strong communication skills for stakeholder collaboration.

Key Skills

Education

Any Gradute