Description

You will build and operationalize large-scale enterprise data solutions.

Responsibilities

• Build production data pipelines from data ingestion through consumption in a hybrid big data architecture.
• Develop and maintain data solutions utilizing technologies like SQL, Python, BigQuery, Cloud Composer, and Cloud Pub/Sub.
• Implement CI/CD pipelines using Gitlab/GitHub and deployment tools like Cloud Build or Gitlab Runners.
• Manage data infrastructure components including Hadoop, Hive, Apache Spark, and Cloud SQL.

Required Skills

• Minimum 5 years of experience in IT building enterprise data solutions.
• Minimum 5 years experience building production data pipelines in a hybrid big data architecture.
• Minimum 4 years experience with CI/CD pipelines (Gitlab/GitHub, Cloud Build/Gitlab Runners).
• Minimum 3 years combined experience in PySpark, Cloud Dataproc, Cloud Dataflow, Terraform, Hadoop, Hive, Apache Spark, Cloud Spanner, Cloud SQL, and Data Fusion.
• Experience with GCP, Azure, or AWS, with preference for GCP.
• Bachelor's or Master's degree in Statistics, Mathematics, Data Science, Engineering, or related quantitative field.
• Experience with data ingestion to consumption within a hybrid big data architecture.

Key Skills

Education

Bachelor's or Master's degrees