Description

Design, implement, and manage scalable Big Data pipelines within the Google Cloud Platform ecosystem.

Responsibilities

• Build and deploy GCP Big Data pipelines for large-scale data processing, reporting, and advanced analytics.
• Participate in the full software development lifecycle, including requirements, development, testing, and quality assurance.
• Maintain application performance, uptime, and scale while adhering to high code quality standards.
• Troubleshoot incidents, perform root cause analysis, and implement preventive measures.
• Manage GCP environments in accordance with security guidelines and track tasks via JIRA, GIT, and Maven.

Required Skills

• 5+ years of experience in Big Data and GCP architecture.
• Expertise in building and optimizing data pipelines, architectures, and Data Warehouse concepts.
• Proficiency in data operations including profiling, joins, aggregation, projection, and explosion.
• Hands-on experience with Hadoop (Dataproc) ecosystem, including Pig, Hive, and Spark.
• Experience with workflow schedulers such as Cloud Composer or Airflow.
• Proficiency in SQL and HQL.
• Knowledge of Unix/Linux platforms and Shell scripting, specifically Python.
• Experience with CI/CD tools like Jenkins or Screwdriver.
• Competency with version control tools like GIT.

Preferred Skills

• Familiarity with data formats such as AVRO, Parquet, and ORC.
• Experience with GCP database technologies, specifically BigQuery.

Key Skills

Education

ANY GRADUATE