Description

You will design, develop, and maintain scalable big data solutions.

This role is on-site.

Responsibilities

• Implement data processing pipelines using PySpark and Apache Spark.
• Collaborate with cross-functional teams to define data requirements and deliver optimized solutions.
• Optimize data storage and retrieval processes within Google Cloud Platform (GCP).
• Troubleshoot and resolve issues across big data applications and distributed systems.

Required Skills

• 5+ years of experience in large-scale data processing using Scala and Python.
• Hands-on expertise with Apache Spark and PySpark for distributed data transformations.
• Solid understanding of big data architecture and distributed computing principles.
• Experience utilizing Google Cloud Platform (GCP) services for data engineering.
• Proficiency in SQL and relational databases, including MS SQL and MySQL.
• Experience implementing streaming technologies, specifically Kafka.
• Familiarity with data manipulation libraries such as NumPy and Pandas.

Preferred Skills

• Ability to troubleshoot performance issues and optimize Spark jobs for scalability.

Key Skills

Education

Any Graduate