Description

You will build and maintain scalable data pipelines and workflows on Azure.

This role is hybrid.

Responsibilities

• Design and maintain scalable data pipelines using Apache Spark and Azure Databricks.
• Develop ETL/ELT processes with Apache Airflow and open-source frameworks.
• Implement CI/CD workflows and monitor pipeline performance and accuracy.
• Integrate and process streaming data from Kafka and RabbitMQ.

Required Skills

• 6+ years of experience in data engineering.
• Strong programming skills in Python, including Pandas and NumPy.
• Proficiency in SQL and experience with relational databases (Sybase, DB2, Snowflake, PostgreSQL, SQL Server).
• Hands-on experience with Azure Databricks and Apache Spark.
• Experience with pipeline monitoring and CI/CD workflows.
• Familiarity with Git for version control.
• Experience with Kafka and RabbitMQ.

Preferred Skills

• Visualization tools experience (Tableau, Power BI).
• Ability to work independently and collaborate across distributed teams.

Key Skills

Education

Not specified