Description

You will design, build, and manage data pipelines and storage solutions on Google Cloud Platform.

Responsibilities

• Build data systems and pipelines using Dataproc, Dataflow, Data Fusion, BigQuery, and Pub/Sub.
• Implement workflows and orchestration using Cloud Composer and Apache Airflow.
• Develop ETL/ELT processes and data ingestion workflows using PySpark and Dataflow.
• Automate data processing tasks and infrastructure deployments using Python, Bash, and Terraform.
• Manage data security and compliance through IAM roles, service accounts, and access policies.
• Monitor and troubleshoot pipelines using Stackdriver and Cloud Monitoring.

Required Skills

• 5+ years of application development experience with a core cloud platform (AWS, Azure, or GCP).
• 1+ years of hands-on GCP experience, specifically with Big Data deployments (Batch/Real-Time).
• 2+ years of coding experience in Python, Java, or PySpark.
• Strong proficiency in SQL.
• Experience with BigQuery, BigTable, Google Cloud Storage, and Pub/Sub.
• Experience with Dataflow, Dataproc, Data Fusion, and Cloud Composer.
• Practical knowledge of MapReduce or MPP systems.
• Experience working within CI/CD models and Agile/Lean methodologies.
• Ability to design architectures for data extraction, loading, transformation, and validation.

Key Skills

Education

Any Graduate