Description

You will design, develop, and maintain data pipelines for data products.

Responsibilities

• Collaborate with business, IT, analyst, and data science teams to gather requirements.
• Design, develop, deploy, and maintain high-performance inbound and outbound data pipelines.
• Model data platforms using business logic and build semantic layer objects.
• Implement CI/CD pipelines for continuous deployment and delivery of data products.
• Ensure data quality by identifying critical data elements and preparing remediation plans.

Required Skills

• Minimum 6 years of experience in data engineering, covering data platforms, ingestion, management, and analytics.
• 2+ years of experience with Google Cloud services: BigQuery, Composer, GCS, DataStream, and Dataflow.
• Proficiency in SQL and Python programming.
• Experience with Airflow for workflow management and custom operator development.
• Skilled in query tuning for performance and scalability.
• Experience with real-time data ingestion using GCP Pub/Sub, Kafka, or Spark.
• Experience developing REST APIs to expose data to internal teams.
• Proven experience working in an agile development environment.

Key Skills

Education

Bachelor's degree