Description

 Key Responsibilities

Design, develop, and maintain scalable ETL/ELT pipelines using Python and PySpark.Build and optimize cloud-native data solutions on Google Cloud Platform (GCP).Develop optimized SQL queries and transformation logic in BigQuery.Design and maintain orchestration workflows using Cloud Composer (Apache Airflow).Develop, optimize, and troubleshoot Spark applications running on Dataproc.Design event-driven data ingestion pipelines using Google Pub/Sub.Ensure data quality through validation, reconciliation, monitoring, and exception handling.Develop reusable frameworks and utilities to improve engineering productivity.Collaborate with Product Owners, Business SMEs, and Engineering teams to understand requirements and deliver scalable solutions.Participate in code reviews and contribute to engineering best practices.Create and maintain technical documentation for data pipelines and architecture.Required Technical SkillsProgrammingPython (Advanced)SQL (Advanced)PySparkGoogle Cloud Platform (GCP)BigQueryCloud Composer (Apache Airflow)DataprocPub/SubCloud StorageData EngineeringETL / ELT DevelopmentBatch & Streaming Data ProcessingData Validation & ReconciliationPipeline Performance OptimizationLogging & MonitoringData Warehouse ConceptsData Warehouse ArchitectureStar & Snowflake SchemaFact & Dimension ModelingSlowly Changing Dimensions (SCD)Partitioning & ClusteringData Lake vs Data Warehouse conceptsDevOps & Version ControlGit / GitHubCI/CD ConceptsDocker (Preferred)Preferred QualificationsExperience working on enterprise-scale cloud migration initiatives.Experience building reusable data engineering frameworks.Knowledge of data governance and data quality practices.Exposure to AI-assisted development tools (e.g., GitHub Copilot) is an advantage.Strong analytical and problem-solving skills.Excellent communication and stakeholder management skills.Experience5+ years of experience in Data Engineering.Strong hands-on experience with SQL, Python, and PySpark.Experience building production-grade data pipelines.Hands-on experience working with Google Cloud Platform (GCP)

Key Skills

Education

Bachelor's degree