Description

You will develop and maintain data pipelines for real-time and hourly batch processing.

Responsibilities

• Develop and maintain data pipelines for real-time and hourly batch processing.
• Support scalable data flows for internal and external stakeholders, including reporting teams and media partners.
• Implement new data exports and update business logic for various feeds.
• Contribute to the development of billion-scale streaming data pipeline infrastructure.
• Ensure data quality and integrity throughout ingestion and processing.

Required Skills

• 5+ years of professional development experience with Scala.
• Practical knowledge of Apache Kafka for data streaming.
• Familiarity with Spark Structured Streaming in Scala.
• Understanding of AWS cloud technologies (S3, Athena, EC2, IAM, EMR).
• Experience with Git for version control.
• Experience with Data Streaming concepts.

Key Skills

Education

Any Gradute