Description

You will design, develop, and maintain scalable and reliable data pipelines and back-end systems.

Responsibilities

• Design, develop, and maintain data pipelines using Spark, Python, Scala, and Java.
• Write efficient and optimized SQL queries for ETL processes.
• Work with DataFrame structures to manipulate and analyze large datasets.
• Implement data storage and processing solutions using cloud technologies (AWS or GCP).
• Build and maintain real-time data streaming pipelines using MSK/Kafka.
• Ensure data quality, integrity, and security across systems.

Required Skills

• 5+ years of experience in back-end development focused on data engineering.
• Strong proficiency in Spark, Python, Scala, and Java.
• Expertise in SQL and relational database operations.
• Experience with DataFrame manipulation for data analysis.
• Experience with AWS or GCP cloud technologies.
• Experience with message streaming platforms like MSK/Kafka.
• Familiarity with object storage solutions like S3.
• Experience with data lake technologies such as Iceberg.
• Solid understanding of data warehousing concepts.

Key Skills

Education

Any Graduate