Description

You will design and implement data pipelines and architectures within an Azure cloud environment.

This role is on-site.

Responsibilities

• Build and maintain data pipelines using Apache Spark, Spark SQL, and PySpark.
• Orchestrate complex workflows and data processes using Apache Airflow.
• Design data models, warehousing solutions, and integration patterns.
• Manage data integration across relational databases and Big Data frameworks.
• Integrate data workflows into CI/CD pipelines to support DevOps practices.

Required Skills

• 5+ years of experience in data engineering roles.
• Proficiency with Apache Spark, including Spark SQL and Spark Streaming.
• Strong programming skills in Python, Scala, or Java.
• Hands-on experience with Azure, AWS, or Google Cloud platforms.
• Expertise in SQL and relational database management.
• Experience with workflow orchestration using Apache Airflow.
• Practical knowledge of data modeling and data warehousing.
• Solid understanding of CI/CD pipelines and DevOps practices.
• Bachelor's degree in any field.

Preferred Skills

• Experience with containerization using Docker.
• Knowledge of Kubernetes orchestration.

Key Skills

Education

Any Gradute