Description

Senior Big Data Engineer for a 12-month engagement in Sunnyvale, CA.

Responsibilities

• Design and develop big data applications using open-source technologies.
• Automate workflows and create data pipelines using Apache Airflow, Hive, Spark, and Kafka.
• Develop logical and physical data models for big data platforms and data lakes.
• Lead daily standups, design reviews, and backlog grooming using JIRA.
• Mentor junior engineers and provide on-call support for system maintenance.

Required Skills

• 5+ years of hands-on experience developing data warehouse solutions and data products.
• 5+ years of experience with distributed data processing platforms (Hadoop, Hive, Spark) and workflow orchestration (Airflow).
• 2+ years of experience modeling and designing schemas for data lakes or RDBMS platforms.
• Proficiency in Python, Java, and Scala.
• Experience with scripting languages Perl and Shell.
• Proven ability to process and manage large datasets (multi-TB/PB scale).
• Bachelor's Degree in Computer Science or equivalent experience.

Key Skills

Education

Any Graduate