Description

You will build and manage data pipelines and infrastructure within the GCP ecosystem.

Responsibilities

• Build and enhance ETL data pipelines and IaC pipelines for new and existing services, including secrets and IAM service account creation.
• Administer GCP data services including DataProc, Airflow, Cloud Composer, and BigQuery, covering module fixing, slot allocation, and deployment.
• Develop FinOps optimization solutions and reduce toil through automation and AIOps for auto-healing.
• Perform cluster performance tuning, application optimization, and BigQuery optimizations.
• Implement observability and explainability practices for enhanced model monitoring and interpretability.

Required Skills

• Hands-on experience with GCP services: BigQuery, Cloud Composer, DataProc, and Cloud Functions.
• Strong programming proficiency in Python, Spark, and PySpark.
• Experience with L2 Administration of GCP data services.
• Hands-on experience with Cloudera and Atlas data platforms.
• Experience building ETL pipelines using Python.
• Knowledge of Cloud Networking and Storage.
• Proficiency with Jenkins and GitLab for version control and code management.
• Ability to manage work plans and track schedules to meet quality standards.
• Any graduate degree.

Preferred Skills

• Experience with L2 Administration of GCP data services.
• Hands-on experience with Cloudera and Atlas data platforms.

Key Skills

Education

ANY GRADUATE