Description

Design and implement large-scale data processing systems using Spark and Jenkins.

This role is on-site.

Responsibilities

• Design and implement large-scale data processing systems using Spark and Jenkins.
• Develop and maintain large-scale data warehouses and data lakes.
• Collaborate with data scientists to translate business requirements into technical solutions.
• Troubleshoot and optimize data processing workflows to improve efficiency and reduce costs.
• Ensure the scalability, reliability, and performance of data systems.

Required Skills

• 6 to 25 years of experience in big data development.
• Proficiency in PySpark, Scala, or Java.
• Hands-on experience with AWS (EMR, S3, Glue, Airflow, RDS, DynamoDB).
• Experience with Hadoop and Spark frameworks.
• Working knowledge of NoSQL databases and relational databases.
• Experience implementing CI/CD pipelines using Jenkins.
• Strong understanding of data processing and analysis.
• Any Graduate degree.

Key Skills

Education

Any Graduate