Description

You will build and maintain data pipelines using Scala and Python within a GCP environment.

Responsibilities

• Develop and manage data workflows using Airflow.
• Implement and maintain Lambda Architecture to handle diverse data processing needs.
• Optimize SQL queries and data structures for efficient processing.
• Integrate Kafka for real-time data streaming and ingestion.
• Build data visualizations and reports using Tableau or Power BI.

Required Skills

• 5+ years of experience in data engineering roles.
• Proficiency in Scala and Python (PySpark).
• Hands-on experience with GCP tools, specifically BigQuery and Hive.
• Strong expertise in SQL optimization.
• Experience implementing Lambda Architecture.
• Practical knowledge of Kafka for stream processing.
• Experience with workflow orchestration using Airflow.
• Ability to use Tableau or Power BI for data visualization.
• Any Graduate degree.

Key Skills

Education

Any Graduate