Description

You will build and manage data pipelines within a GCP environment.

Responsibilities

• Develop and maintain ETL processes using Spark and Hive.
• Optimize complex queries to improve data processing efficiency.
• Manage data workflows using Airflow or Automic.
• Operate within Linux/Unix and Shell environments to support data infrastructure.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in PySpark, Python, or Scala.
• Hands-on experience with Hadoop, YARN, and Hive.
• Experience working with Spark and GCP DataProc technologies.
• Strong understanding of ETL patterns and query optimization.
• Working knowledge of Linux, Unix, or Shell scripting.
• Experience with workflow orchestration via Airflow or Automic.
• Degree in BTech, BE, ME, or MTech.

Preferred Skills

• Experience with Kafka.
• Knowledge of restAPI and reporting tools.

Key Skills

Education

Any Graduate