Description

You will design and maintain data pipelines using Python and PySpark to transform structured and unstructured data into standardized outputs for quantitative analysis. You are responsible for integrating Large Language Models into data workflows and ensuring operational support. You will develop utility tools to automate software development, testing, and deployment. You will participate in code reviews, adhere to coding standards, and collaborate with cross-functional teams to achieve project goals.

Responsibilities

• Build and maintain data pipelines using Python and PySpark.
• Transform structured and unstructured data for financial engineering and quantitative analysis.
• Integrate LLMs into data processing workflows.
• Develop automation tools for testing and deployment.
• Conduct code reviews and enforce coding standards.

Required Skills

• 5+ years of experience as a Data Engineer or similar role.
• Strong Python programming skills for data engineering and operational support.
• Solid SQL skills for querying and manipulating structured data.
• Experience with PySpark for data processing.
• Knowledge of Large Language Models (LLM) and their integration with data pipelines.
• Familiarity with AWS for deployment and scaling.
• Proficiency with Git and scripting.
• Experience with C++ is desirable.

Key Skills

Education

Any Graduate