Description

Responsibilities

• Design and implement robust data pipelines using Google Cloud Platform (GCP) services such as BigQuery, Cloud Storage, and Pub/Sub
• Develop and manage workflows using Cloud Composer (Apache Airflow) for efficient scheduling and orchestration
• Write clean, efficient, and scalable code in Python, leveraging advanced programming techniques
• Craft complex SQL queries in BigQuery, including window functions, CTEs, and performance tuning strategies
• Build and maintain real-time data processing systems using Apache Kafka
• Model and manage NoSQL databases, particularly MongoDB, with a focus on scalable schema design
• Utilize Shell scripting and perform Linux system administration tasks to support data infrastructure
• Conduct data profiling and implement validation techniques to ensure data quality and integrity
• Develop and maintain API integration scripts for seamless service automation and data exchange
• Troubleshoot and resolve data-related issues with strong analytical and problem-solving skills
• Create and maintain data flow diagrams to clearly communicate architecture and pipeline logic to stakeholders

Qualifications

• Required: Minimum 7 years of proven hands-on experience with GCP data services
• Required: Minimum 3 years of proven hands-on experience with the core technical responsibilities listed above (data pipelines, Cloud Composer, Python, BigQuery SQL, Apache Kafka, MongoDB, Shell scripting, data profiling, API integration, and data flow documentation)
• Required: Demonstrated experience with AI tools such as GPT and Gemini, with clear business use cases highlighted on resume
• Required: Strong understanding of distributed systems and data engineering best practices
• Required: Excellent communication and documentation skills
• Required: Ability to work independently and collaboratively in a fast-paced environment
• Required: Experience in Agile methodology
• Required: Experience working in an onshore/offshore support model, collaborating with offshore teams
• Preferred: GCP Google Cloud Professional Data Engineer Certification
• Preferred: Experience with SAFe methodology
• Preferred: Experience working with GitHub, RTC, and automation tools

Key Skills

Education

Bachelor's degree