Description

You will build and optimize scalable data pipelines and end-to-end ETL workflows within the Azure ecosystem.

This role is on-site.

Responsibilities

• Design, implement, and optimize end-to-end data pipelines on Azure for scalability and reliability.
• Develop and maintain ETL workflows to ensure efficient data processing.
• Execute advanced data transformations, cleansing, and enrichment using PySpark.
• Manage data storage solutions on Azure to ensure optimal performance.
• Identify and resolve performance bottlenecks within data processing workflows and PySpark jobs.

Required Skills

• 5+ years of professional experience as a Data Engineer.
• Proven expertise with Azure Cloud services, specifically Azure Data Factory, Azure SQL Database, and Azure Databricks.
• Proficiency in PySpark for data processing, analytics, and transformations.
• Strong background in building and optimizing data pipelines and workflows.
• Solid experience with ETL processes, data modeling, and data warehousing.
• Ability to perform performance tuning on data pipelines and queries.
• Experience troubleshooting and resolving bottlenecks in data workflows.
• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field.

Preferred Skills

• Relevant Azure certifications.
• Knowledge of data governance best practices.
• Familiarity with Hadoop, Spark, and Kafka.

Key Skills

Education

Any Graduate