Description

You will build high-volume data pipelines and design end-to-end data architecture, from ingestion to serving.

This role is on-site.

Responsibilities

• Build and maintain batch and near real-time data pipelines using Python and Kafka.
• Develop LLM-based extraction and processing workflows for production use.
• Implement validation layers to reduce hallucinations and ensure data quality.
• Design and manage embeddings, vector search, and retrieval systems.
• Ensure system reliability, observability, and effective failure handling in distributed environments.

Required Skills

• 5+ years of experience in data engineering or backend development.
• Strong proficiency in Python for building ETL and streaming pipelines.
• Hands-on experience with AWS cloud services.
• Deep knowledge of SQL and NoSQL databases.
• Experience deploying LLMs in production, including prompting, validation, and monitoring.
• Familiarity with distributed systems and Kafka for streaming data.

Preferred Skills

• Experience with RAG architectures and vector search.
• Exposure to financial domain data or agentic AI workflows.

Key Skills

Education

Bachelor's degree