Description

Build and maintain PySpark applications for batch and streaming data processing within Hadoop environments.

Responsibilities

• Optimize Spark application performance using Spark Context, Spark-SQL, DataFrames, and Pair RDDs.
• Design and implement web applications using Streamlit to visualize complex data processing tasks.
• Convert Java-based legacy scenarios to Python frameworks and decommission old controls.
• Collaborate with stakeholders to translate customer needs into technical solutions.
• Process huge datasets using APIs and open-source frameworks in the cloud.

Required Skills

• 8+ years of experience in Python and PySpark.
• Extensive experience with Hadoop Big Data ecosystems and NoSQL databases.
• Strong backend development experience in Python.
• Proficiency with data libraries: Pandas, Polars, NumPy, Matplotlib, Plotly, Seaborn, and Altair.
• Experience building web interfaces and custom widgets using Streamlit.
• Ability to process huge datasets using APIs and open-source frameworks in the cloud.
• Experience working within a financial organization or familiarity with trading hub concepts.

Preferred Skills

• Bachelor's Degree in Computer Science or related field.

Key Skills

Education

Bachelor's Degree