Description

Key Skills: Pyspark, Bigquery, SQL, Python, Data Flow, Airflow

Roles and Responsibilities:

• Design and develop scalable data pipelines using Pyspark and BigQuery.
• Implement data processing workflows and ETL processes.
• Collaborate with data scientists and analysts to understand data requirements.
• Optimize data storage and retrieval processes for performance.
• Ensure data quality and integrity throughout the data lifecycle.

Skills Required:

• 7+ years of relevant experience in data engineering.
• Strong experience with Pyspark for data processing.
• Proficiency in BigQuery for data warehousing.
• Expertise in Python for data manipulation and analysis.
• Solid understanding of SQL for querying databases.

Good to Have:

• Familiarity with Airflow for workflow orchestration.
• Experience with Data Flow for stream processing.

Education: Bachelor's degree in Computer Science or related field

Key Skills

Education

Any Graduate