Description

You will lead the design, implementation, and optimization of large-scale data systems, real-time streaming solutions, and cloud-based data platforms.

Responsibilities

• Lead the design and development of scalable data architectures on AWS using S3, EMR, Glue, Redshift, Lambda, and Kinesis.
• Build and optimize complex batch and real-time data pipelines using Apache Spark (Scala and PySpark) and Kafka Streams (Java).
• Architect and manage Data Lakes using S3, Glue, and Apache Iceberg to ensure ACID compliance and schema evolution.
• Manage Snowflake for cloud data warehousing, focusing on data integration and query optimization.
• Develop data models and Tableau dashboards to provide actionable insights for business stakeholders.
• Mentor junior engineers and collaborate with Data Scientists, Product Managers, and DevOps teams to deliver data solutions.

Required Skills

• 5+ years of experience in Data Engineering or designing large-scale distributed data systems.
• Proficiency in Apache Spark using both Scala and PySpark.
• Hands-on experience with Kafka Streams using Java for real-time processing.
• Deep knowledge of AWS services including S3, Glue, EMR, Lambda, Redshift, and Athena.
• Experience implementing Apache Iceberg within Data Lake architectures.
• Proficiency in Snowflake for data modeling and warehousing.
• Advanced SQL skills for querying relational and NoSQL databases.
• Bachelor's or Master's degree in a relevant field.

Preferred Skills

• Experience with CI/CD implementation.

Key Skills

Education

Bachelor's or Master's degrees