Description

Key Skills: GCP, Bigdata, Airflow, CICD

Roles and Responsibilities:

• Build and manage big data processing solutions to support large-scale data ingestion and transformation needs.
• Develop and maintain data pipelines and data workflows on GCP, ensuring performance, reliability, and scalability.
• Implement robust data engineering practices to support end-to-end data flow from source to consumption.
• Collaborate with stakeholders to understand data requirements and translate them into technical designs and deliverables.
• Support automated deployment and release processes using CI/CD practices (nice-to-have).
• Orchestrate and schedule data workflows using Airflow where applicable (nice-to-have).
• Monitor pipeline health, troubleshoot issues, and optimize data processing for stable operations.

Skills Required:

• Strong experience in building and managing big data solutions using GCP services such as BigQuery, Dataflow, Dataproc, and Cloud Storage.
• Proven ability to design and develop scalable data pipelines for large-scale data ingestion, processing, and transformation.
• Hands-on experience with distributed data processing frameworks and big data technologies.
• Experience in workflow orchestration using tools like Apache Airflow for scheduling and monitoring data pipelines.
• Familiarity with CI/CD practices and tools to automate deployment and data pipeline releases.
• Strong understanding of data modeling, data warehousing, and data lifecycle management concepts.
• Ability to monitor, troubleshoot, and optimize data pipelines for performance and reliability.
• Experience in handling structured and unstructured data across different data sources.
• Knowledge of data security, governance, and access control within GCP environments.
• Strong analytical and problem-solving skills to resolve complex data engineering challenges.
• Effective communication and stakeholder management skills to collaborate with cross-functional teams.

Additional Note:

• Looking for a GCP Data Engineer with strong experience in Big Data and GCP.

Education: Bachelor's Degree in related field

Key Skills

Education

Any Graduate