Description

You will design and implement complex data transformation pipelines using big data technologies.

Responsibilities

• Develop ETL/ELT processes to transform complex datasets across distributed systems.
• Implement and manage data warehousing and data management frameworks.
• Execute data lifecycle processes to ensure data integrity and availability.
• Build scalable data pipelines using Spark and streaming technologies.

Required Skills

• 5+ years of experience in data engineering roles.
• Hands-on experience with Azure and AWS Big Data solutions.
• Proficiency with Hadoop, Hive, HBase, and Sqoop.
• Strong experience with Spark and Spark Streaming.
• Solid programming skills in Python, Scala, and Java.
• Experience with PLSQL and Shell scripting.
• Deep understanding of data warehousing and ETL/ELT frameworks.

Preferred Skills

• Experience using Azure Databricks for ETL/ELT development.
• Knowledge of R programming for data tasks.

Key Skills

Education

Any Graduate