Description

You will design, build, and maintain large-scale data processing systems and pipelines.

This role is remote.

Responsibilities

• Design and optimize data pipelines using Hadoop, Hive, Spark, and NiFi.
• Manage data ingestion, transformation, and orchestration workflows.
• Develop and execute complex SQL queries for data processing, validation, and troubleshooting.
• Implement event streaming and message processing using Kafka.
• Troubleshoot production issues and perform performance tuning for jobs and pipelines.

Required Skills

• 8+ years of experience in data engineering.
• Proficiency with Hadoop, Hive, and Spark.
• Strong command of SQL.
• Experience with GCP and AWS cloud platforms.
• Familiarity with Python.
• Working knowledge of CI/CD practices and DevOps principles.
• Experience with Kafka for event streaming.
• Proficiency with Jenkins, Bitbucket, and Jira.

Preferred Skills

• Bachelor's degree in Computer Science or related field.
• Experience with NiFi for data flow orchestration.

Key Skills

Education

Bachelor's degree