Description

You will design and develop data ingestion, processing, and storage solutions for big data environments.

Responsibilities

• Develop scalable data pipelines for the ingestion, transformation, and storage of large datasets.
• Build efficient data processing solutions optimized for both real-time and batch processing.
• Design and implement data storage solutions to support big data requirements.
• Implement data validation and monitoring strategies to ensure data quality and integrity throughout the pipeline.

Required Skills

• 5-8 years of experience designing and implementing ETL solutions.
• Strong expertise in Apache Spark and big data technologies.
• Proficiency in Python, Scala, and Java.
• Experience with AWS data ingestion and processing tools including FluentBit, Kinesis, and Glue.
• Hands-on experience with AWS storage solutions such as S3, Redshift, Iceberg, and Aurora.
• Advanced SQL skills.
• Experience managing real-time data streams.
• Bachelor's degree or higher in Computer Science, Engineering, or a related field.

Preferred Skills

• AWS data services certification or direct hands-on experience.

Key Skills

Education

Bachelor's degree or higher in Computer Science, Engineering