Description

You will design, develop, and maintain software solutions, focusing on backend services, APIs, and database integration. You will build custom rule engines and implement data processing pipelines using Python and PySpark.

You will gather data from HBase, apply business transformations via Apache Spark DataFrames and RDDs, and perform read/write operations using Hive Context objects. You will optimize applications for performance, security, and maintainability while troubleshooting and resolving software defects.

Responsibilities

• Design and maintain backend services, APIs, and database integrations.
• Develop custom rule engines and Python-based data analysis programs.
• Implement data pipelines using PySpark, Spark DataFrames, and RDDs.
• Execute data extraction from HBase and manage Hive Context operations.
• Optimize application performance, security, and code maintainability.

Required Skills

• 5+ years of professional experience with Python development.
• Strong proficiency in PySpark and Apache Spark (DataFrames/RDDs).
• Experience designing custom frameworks or rule engines.
• Hands-on experience with HBase for data gathering.
• Ability to integrate databases and build scalable APIs.
• Bachelor's degree in a relevant technical field.

Key Skills

Education

Bachelor's degree