Description

Design and manage complex data integration workflows and real-time processing systems.

Responsibilities

• Integrate data from diverse sources including APIs, databases, and streaming platforms.
• Build and maintain scalable data pipelines using Apache Airflow, Apache NiFi, or custom solutions.
• Develop real-time processing workflows using stream processing frameworks.
• Ensure data consistency and quality across all integration points.

Required Skills

• 10+ years of experience in data integration.
• 3+ years of experience with Apache Flink.
• 3+ years of experience with Apache Beam.
• 3+ years of experience with Apache NiFi.
• 3+ years of experience with Apache Hudi.
• 6+ years of experience with ZooKeeper.
• Proficiency with Apache Kafka.
• Experience working with APIs and SQL.

Preferred Skills

• Bachelor's degree.

Key Skills

Education

Bachelor's degree