Description

You will design, build, and maintain large-scale data pipelines on Google Cloud Platform. This role owns the development and optimization of distributed data processing systems.

Responsibilities

• Build and optimize batch and streaming data pipelines using Spark, PySpark, and Hive.
• Develop and maintain ETL processes in Python, integrating with GCP services like BigQuery.
• Manage data ingestion from Kafka and storage in MySQL and NoSQL databases.
• Implement CI/CD practices to ensure reliable deployment of data engineering code.

Required Skills

• 5+ years of experience in data engineering or cloud data development.
• Strong proficiency in Spark, PySpark, and Hive for distributed data processing.
• Hands-on experience with Google Cloud Platform (GCP) and BigQuery.
• Proficient in Python for scripting and application development.
• Experience with event streaming using Kafka.
• Familiarity with CI/CD tools and practices.
• Working knowledge of relational databases (MySQL) and NoSQL solutions.

Preferred Skills

• Experience adhering to project productivity metrics and timelines.
• Strong technical writing and verbal communication skills.

Key Skills

Education

Any Graduate