Description

You will design, develop, and maintain scalable data pipelines and backend systems using Spark, Python, and Scala.

This role is on-site.

Responsibilities

• Build and maintain real-time data streaming pipelines using MSK/Kafka and batch processing with Spark.
• Write optimized SQL queries and manage data storage using S3 and Iceberg data lake technologies.
• Ensure data quality, integrity, and security across all backend systems and pipelines.
• Collaborate with data scientists and engineers to deliver data-driven solutions and participate in code reviews.
• Troubleshoot issues in data pipelines and contribute to improving development processes.

Required Skills

• 5+ years of experience in backend development with a focus on data engineering.
• Strong proficiency in Spark, Python, Scala, and Java.
• Expertise in SQL and working with relational databases.
• Experience with cloud technologies, preferably AWS or GCP.
• Hands-on experience with message streaming platforms like MSK/Kafka.
• Experience with S3 or similar object storage and data lake technologies like Iceberg.
• Understanding of data warehousing concepts and DataFrames for data manipulation.

Preferred Skills

• Experience with data modeling, schema design, and data governance.
• Familiarity with DevOps practices and CI/CD pipelines.

Key Skills

Education

Any Graduate