Description

You will design, develop, and maintain scalable data pipelines and back-end systems using Spark, Python, Scala, and Java.

This role is on-site.

Responsibilities

• Build and optimize ETL processes using SQL and DataFrames for large-scale data manipulation.
• Implement real-time data streaming pipelines using MSK or Kafka.
• Develop data storage and processing solutions on AWS or GCP, leveraging S3 and Iceberg.
• Troubleshoot pipeline issues and ensure data quality, integrity, and security.
• Collaborate with data scientists and engineers to define requirements and improve development processes.

Required Skills

• 5+ years of back-end development experience with a focus on data engineering.
• Strong proficiency in Spark, Python, Scala, and Java.
• Expertise in SQL and relational databases.
• Experience with cloud technologies (AWS or GCP) and object storage (S3).
• Hands-on experience with message streaming platforms like MSK/Kafka.
• Familiarity with data lake technologies such as Iceberg.
• Bachelor’s degree in Computer Science or equivalent experience.

Preferred Skills

• Experience with data modelling, schema design, and data governance.
• Knowledge of CI/CD pipelines and DevOps practices.

Key Skills

Education

Any Graduate