Description

You will design, build, and optimize large-scale data pipelines and models within Azure cloud environments.

This role is on-site.

Responsibilities

• Design and implement efficient data models for large-scale datasets.
• Build data processing workflows using Python and PySpark.
• Optimize Spark performance and manage complex data workflows.
• Maintain CI/CD pipelines for data engineering processes.

Required Skills

• 5+ years of hands-on experience in Big Data environments.
• Proficiency in Azure Databricks, Azure Data Lake, and Azure Data Factory.
• Strong experience with Python and PySpark for data processing.
• Advanced SQL capabilities for large datasets and performance optimization.
• Experience with Spark performance tuning.
• Knowledge of CI/CD pipelines for data workflows.

Preferred Skills

• Exposure to PowerShell, Bash, or Perl scripting.
• Experience with Power BI.

Key Skills

Education

Any Graduate