Description

Design and implement scalable data processing pipelines and AI agents using Python, PySpark, and Databricks on AWS. You own the lifecycle from POC to production for GenAI solutions.

This role is on-site.

Responsibilities

• Build and maintain data lakes and processing workflows on Databricks to support ML operations.
• Design and implement prompt and context engineering techniques to enhance LLM-based solution performance.
• Integrate AI/ML models into data pipelines and build AI agents using agentic frameworks and MCP servers.
• Collaborate with cross-functional teams to identify business requirements and deliver data-driven solutions.

Required Skills

• 5+ years of applied experience in data science, machine learning, or adjacent disciplines.
• Hands-on expertise with Python, PySpark, and Spark SQL for data processing.
• Proven experience with LLM orchestration, GenAI models, and AI agents.
• Strong proficiency with Databricks for building scalable data solutions.
• Experience with AWS services (S3, Lambda, Redshift, Athena, Step Functions, MSK, EKS) and data lake architectures.
• Ability to take solutions from POC to production quickly.

Preferred Skills

• Full stack development experience including Java/J2EE microservices and React UI.
• AWS certification and experience with agentic frameworks, MCP servers, and prompt engineering.

Key Skills

Education

Any Graduate