Description

You will build and optimize data pipelines and workflows within an Azure environment.

Responsibilities

• Develop and maintain scalable data pipelines using Databricks, PySpark, and Python.
• Process structured and unstructured data in cloud-based environments.
• Implement data transformation, cleansing, and integration solutions.
• Collaborate with data engineers, analysts, and stakeholders to meet business requirements.

Required Skills

• 5+ years of experience in data engineering roles.
• Expertise in Python and PySpark.
• Hands-on experience with Apache Spark and big data processing.
• Proficiency in SQL.
• Experience with ETL development and performance tuning.
• Working knowledge of the Azure cloud platform.
• Understanding of data lake, data warehouse, and data modeling principles.
• Experience managing big data architectures.

Key Skills

Education

Any Graduate