Description

You will implement, optimize, and maintain ETL processes critical to backend operations.

Responsibilities

• Implement Azure Logic App flows to ingest data from client systems into the cloud data store server.
• Configure Azure Functions using NodeJS and SQL to extract, transform, and load data from APIs and objects.
• Develop data processing functions in NodeJS for the ETL pipeline, extracting data from various sources and inserting it into SQL databases.
• Design and implement ETL processes using Apache Spark in standalone mode for enhanced data processing.
• Deploy and manage ETL workflows within Docker containers.

Required Skills

• 5+ years of proven experience implementing ETL processes in cloud environments, specifically Azure.
• Hands-on experience with Azure Logic Apps, Azure Functions, and NodeJS.
• Strong proficiency in Python, including experience developing Python-based ETL pipelines.
• Experience with Apache Spark and PySpark.
• Proficiency with Docker for deployment and Git CICD pipelines.
• Experience with SQL databases.
• Familiarity with Linux operating systems.

Key Skills

Education

Any Graduate