Description

Big Data Engineer with 5+ years of experience to design and maintain data pipelines on Google Cloud Platform. You will own the development and optimization of large-scale data processing systems.

Responsibilities

• Build and optimize data pipelines using Spark, PySpark, and Hive for batch and streaming workloads.
• Develop and maintain ETL processes in Python, integrating with GCP services like Big Query.
• Manage data ingestion from Kafka and storage in MySQL and NoSQL databases.
• Implement CI/CD practices to ensure reliable deployment of data engineering code.

Required Skills

• 5+ years of experience in data engineering or cloud data development.
• Strong proficiency in Spark, PySpark, and Hive for distributed data processing.
• Hands-on experience with Google Cloud Platform (GCP) and Big Query.
• Proficient in Python for scripting and application development.
• Experience with event streaming using Kafka.
• Familiarity with CI/CD tools and practices.
• Working knowledge of relational databases (MySQL) and NoSQL solutions.

Preferred Skills

• Experience adhering to project productivity metrics and timelines.
• Strong technical writing and verbal communication skills.

Key Skills

Education

Any Graduate