Description

You will manage and automate data infrastructure and pipelines within a Google Cloud Platform environment.

Responsibilities

• Automate and manage cloud resources using Terraform for infrastructure as code.
• Administer Databricks platforms and Kubernetes clusters to ensure scalable containerized operations.
• Collaborate with data engineers to troubleshoot access issues and optimize data pipelines.
• Develop automation scripts using Python, Ruby, Bash, or Docker.
• Manage data orchestration workflows using Airflow and DAGs.

Required Skills

• Experience with Terraform and Google Cloud Platform management.
• Proficiency in Linux and Bash scripting.
• Hands-on experience with Kubernetes administration.
• Proficiency in Git and GitLab CI/CD pipelines.
• Working knowledge of Python or Ruby for automation tasks.
• Understanding of Data Lake architecture and medallion patterns.
• Familiarity with Kafka and the distinction between streaming and batch operations.
• Proficiency in SQL for data management tasks.
• Experience with Git-based version control systems.

Preferred Skills

• Exposure to Scala.
• Experience with configuration management tools such as Chef, Ansible, Puppet, or Salt.

Key Skills

Education

Not specified