Description

You will design, develop, and maintain big data applications and distributed data processing workflows.

Responsibilities

• Design and develop big data applications using Apache Hive, Spark, and Kafka.
• Automate workflows and build data pipelines using Apache Airflow.
• Develop logical and physical data models for big data platforms.
• Build data pipelines in GCP using Dataproc, GCS, and BigQuery.
• Lead daily standups, design reviews, and backlog grooming using JIRA.
• Mentor junior engineers on development standards and best practices.
• Implement CI/CD pipelines and follow Gitflow development practices.

Required Skills

• 10+ years of experience in data engineering and data warehouse development.
• 6+ years of hands-on experience with Hadoop, Hive, Spark, or Airflow.
• 4+ years of recent experience working with GCP.
• Strong proficiency in Python, Java, Scala, and scripting languages like Perl or Shell.
• Experience with schema design and data modeling for data lakes and RDBMS.
• Expertise in managing large datasets at multi-TB/PB scale.
• Hands-on experience with Kubernetes, Docker, and containerized environments.
• Proficiency with Bitbucket, JIRA, and Confluence.
• Experience with Agile, Scrum, or Kanban methodologies.

Preferred Skills

• Experience with CI/CD tools such as Bamboo, Jenkins, or TFS.
• Knowledge of Gitflow branching strategies and test-driven development.

Key Skills

Education

Any Gradute