Description

You will design and implement data pipelines and architectures within an Azure cloud environment.

Responsibilities

• Build and maintain data pipelines using Apache Spark, Spark SQL, and PySpark.
• Orchestrate complex workflows and data processes using Apache Airflow.
• Design and implement data models, data warehousing solutions, and integration patterns.
• Manage data integration tasks across relational databases and Big Data frameworks.
• Support DevOps practices by integrating data workflows into CI/CD pipelines.

Required Skills

• 5+ years of experience in data engineering roles.
• Proficiency with Apache Spark, including Spark SQL and Spark Streaming.
• Strong programming skills in Python, Scala, or Java.
• Hands-on experience with Azure, AWS, or Google Cloud platforms.
• Expertise in SQL and relational database management.
• Experience with workflow orchestration using Apache Airflow.
• Practical knowledge of data modeling and data warehousing.
• Solid understanding of CI/CD pipelines and DevOps practices.
• Bachelor's degree in any field.

Preferred Skills

• Experience with containerization using Docker.
• Knowledge of Kubernetes orchestration.

Key Skills

Education

Any Gradute