Description

Big Data Engineer responsible for designing, building, and maintaining distributed data systems and data warehousing solutions on Google Cloud Platform.

Responsibilities

• Design and develop scalable ETL pipelines using Pyspark and Hive for large-scale data processing.
• Manage Big Data and NoSQL environments, ensuring high availability, replication, and performance tuning.
• Optimize SQL queries and database structures for online transaction and analytical processing systems.
• Collaborate within Agile/Scrum teams to deliver software using Test-Driven Development practices.

Required Skills

• 5+ years of experience in Big Data, Data Warehousing, and ETL development.
• Strong proficiency in SQL, Hive, and Pyspark for data manipulation and analysis.
• Hands-on experience with Google Cloud Platform (GCP) data tools and infrastructure.
• Deep knowledge of distributed systems, NoSQL design, and various data stores (document, column, graph).
• Experience with database administration, including encryption, maintenance, and schema development.
• Proficiency in Python or Java for data engineering tasks.
• Familiarity with data modeling tools such as ErStudio or ErWin.
• Experience working in Agile or Scrum software delivery methodologies.

Key Skills

Education

Any Gradute