← Back to jobs

IBU Consulting Ltd Logo
Data Engineer

IBU Consulting Ltd

 

Bengaluru, KA, India

Posted On: Just posted
Experience: 5+ years
Availability: Onsite
Openings: 1
Category: Data Engineer
Tenure: No Preference/Any
Related Jobs

No related jobs found

Description

Design, develop, and maintain scalable ETL/ELT pipelines and cloud-native data solutions on Google Cloud Platform.

This role is on-site.

Responsibilities

  • Build and optimize data pipelines using Python, PySpark, and BigQuery.
  • Develop orchestration workflows with Cloud Composer (Apache Airflow) and event-driven ingestion via Pub/Sub.
  • Optimize Spark applications on Dataproc and write efficient SQL for data transformation.
  • Ensure data quality through validation, reconciliation, and monitoring frameworks.
  • Collaborate with Product Owners and Engineering teams to deliver scalable solutions.

Required Skills

  • 5+ years of experience in Data Engineering with production-grade pipelines.
  • Advanced proficiency in Python, SQL, and PySpark.
  • Hands-on experience with Google Cloud Platform (GCP), BigQuery, Dataproc, Pub/Sub, and Cloud Storage.
  • Experience with Cloud Composer (Apache Airflow) for workflow orchestration.
  • Strong understanding of data warehouse concepts, including Star/Snowflake schemas, SCD, partitioning, and clustering.
  • Proficiency in Git/GitHub and CI/CD concepts.
  • Experience collaborating with Product Owners and Business SMEs.

Preferred Skills

  • Experience with enterprise-scale cloud migration initiatives.
  • Knowledge of data governance, data quality practices, and reusable framework development.
  • Exposure to AI-assisted development tools like GitHub Copilot.

Education

Bachelor's degree

Related Jobs

No related jobs found

← Back to jobs