Description

Design system solutions and develop custom applications to transition legacy batch-based platforms to event-driven, microservices-based architectures. You will build and execute data pipelines for batch and real-time data processing.

Responsibilities

• Translate user needs into system architecture and develop software applications.
• Build and execute data pipelines for batch and real-time data processing.
• Collaborate with SRE, testing, and production support teams to ensure smooth development life-cycle transitions.
• Perform coding, debugging, documentation, and performance validation of component systems.
• Test the engineering resilience of software and automation tools.

Required Skills

• 5+ years of software development experience, including leading engineering and scrum teams.
• 3+ years of hands-on experience with Map-Reduce, Hive, and Spark (core, SQL, and PySpark).
• Expertise in writing and optimizing complex SQL, Hive, and PySpark dataframes for large-scale data processing.
• Proficiency in UNIX shell scripting.
• Strong understanding of Hadoop and Spark architecture principles.
• Ability to design and develop optimized data pipelines.
• Experience in the analysis, design, testing, and implementation of system applications.
• Hands-on experience with NoSQL databases including HBase, Couchbase, or MongoDB.

Preferred Skills

• Knowledge of cloud platforms such as GCP or AWS.
• 2+ years of experience designing solutions using Kafka streams or queues.
• Experience with GitHub and CI/CD pipelines.

Key Skills

Education

Any Graduate