Description

Design, build, and deploy scalable ETL/ELT pipelines for large-scale data processing using PySpark and Apache Spark.

This role is on-site.

Responsibilities

• Build and maintain complex logical and physical databases for high performance and security.
• Design and implement data solutions that adhere to strict financial industry regulations and governance policies.
• Provide technical guidance and serve as the escalation point for complex data issues.
• Collaborate with Scrum masters and product owners to deliver project objectives using Agile methodologies.

Required Skills

• 5+ years of experience with compute engines, specifically Apache Spark and PySpark.
• Strong knowledge of data layers including Lakehouse/Objectstore, Iceberg, and Delta Lake.
• Proficiency in SQL for data manipulation and querying.
• Experience with streaming technologies, particularly Kafka.
• Ability to work effectively in a financial industry environment with strict compliance requirements.

Preferred Skills

• Familiarity with agentic AI tools such as LangGraph, LangChain, and A2A.
• Previous experience working in the financial sector.

Key Skills

Education

Any Graduate