Description

You will design, build, and optimize data pipelines on Google Cloud Platform using PySpark and Spark SQL.

Responsibilities

• Build and manage ETL pipelines using GCP tools and existing platforms.
• Develop and optimize Spark SQL queries and PySpark scripts for data processing and validation.
• Tune Spark clusters to improve data processing performance and resource utilization.
• Write complex SQL queries for data analysis and ensure data quality.
• Support client-facing delivery by providing data-driven solutions.

Required Skills

• 8+ years of experience in data engineering or analytics.
• Strong proficiency in SQL for complex querying and analysis.
• Hands-on experience with Google Cloud Platform (GCP) environments.
• Experience with Unix/Linux command line tools and scripting.
• Proficiency in PySpark for data processing tasks.
• Experience with Spark SQL and cluster optimization techniques.
• Understanding of data engineering and analytics concepts.

Preferred Skills

• Experience working in client-facing delivery environments.

Key Skills

Education

Any Gradute