Description

Build and optimize PySpark applications for batch and streaming data processing within Hadoop environments, while designing web visualizations using Streamlit and migrating legacy Java controls to Python.

This role is on-site.

Responsibilities

• Build and maintain PySpark applications for batch and streaming data processing.
• Optimize Spark application performance using Spark Context, Spark-SQL, DataFrames, and Pair RDDs.
• Design and implement web applications using Streamlit to visualize complex data processing tasks.
• Convert Java-based legacy scenarios to Python frameworks and decommission old controls.
• Collaborate with stakeholders to translate customer needs into technical solutions.

Required Skills

• 8+ years of experience in Python and PySpark.
• Extensive experience with Hadoop Big Data ecosystems and NoSQL databases.
• Strong backend development experience in Python.
• Proficiency with data libraries: Pandas, Polars, NumPy, Matplotlib, Plotly, Seaborn, and Altair.
• Experience building web interfaces and custom widgets using Streamlit.
• Ability to process huge datasets using APIs and open-source frameworks in the cloud.
• Experience working within a financial organization or familiarity with trading hub concepts.

Preferred Skills

• Bachelor's Degree in Computer Science or related field.

Key Skills

Education

Bachelor's Degree