Description

You will design and maintain large-scale data pipelines using modern big data technologies.

This role is on-site.

Responsibilities

• Develop and optimize data workflows using PySpark and Python.
• Manage data ingestion and transformation processes with Apache NiFi and Kafka.
• Query and analyze structured data using SQL and Hive.
• Implement data lake architectures leveraging Apache Iceberg and Starburst.
• Maintain Hadoop cluster operations and ensure data integrity.

Required Skills

• 5+ years of experience in data engineering.
• Strong proficiency in PySpark and Python.
• Expert-level SQL skills.
• Hands-on experience with Apache NiFi, Starburst, and Apache Iceberg.
• Experience with Kafka for streaming data.
• Deep understanding of Hadoop and Hive ecosystems.

Preferred Skills

• Bachelor's degree in Computer Science or related field.
• 3+ years tenure in previous data engineering roles.

Key Skills

Education

Bachelor's degree