Description

You will design and develop big data applications and scalable data pipelines using open-source technologies.

Responsibilities

• Design and develop logical and physical data models for big data platforms.
• Automate workflows using Apache Airflow and create pipelines with Apache Hive, Spark, and Kafka.
• Lead data engineering efforts in an offshore model, including leading daily standups and design reviews.
• Mentor junior engineers, conduct root cause analysis, and ensure adherence to coding standards.
• Manage complex user stories, prioritize backlogs in JIRA, and estimate resource efforts.

Required Skills

• 12+ years of hands-on experience in data engineering and data warehouse solutions.
• 10+ years of experience acting as a Scrum Master.
• 6+ years of experience with Hadoop, Hive, Spark, or Airflow.
• 5+ years of experience with GCP (Dataproc, GCS, BigQuery).
• 5+ years of experience in PySpark and Scala development.
• 5+ years of experience in schema modeling and design for data lakes or RDBMS.
• Proficiency in Python, Java, Scala, Perl, and Shell scripting.
• Experience working with multi-TB/PB scale datasets.
• Experience with Gitflow, BitBucket, JIRA, Confluence, and CI/CD tools like Bamboo, Jenkins, or TFS.
• Bachelor's degree in Computer Science or equivalent experience.

Key Skills

Education

Bachelor's degree