Description

You will design and deliver high-quality data products, ETL/ELT pipelines, and analytics-ready solutions to support enterprise functions. You will own the development of data engineering systems, APIs, and AI integrations on AWS and Databricks.

This role is on-site.

Responsibilities

• Build and optimize ETL/ELT pipelines using PySpark, Delta Lake, and Databricks with medallion architecture.
• Design and develop RESTful APIs to expose data products and analytics capabilities to business applications.
• Integrate LLM APIs (OpenAI, AWS Bedrock, Anthropic) and build RAG pipelines using vector databases.
• Collaborate with data architects to maintain data models and ensure data governance, accuracy, and security.
• Optimize data storage and retrieval for efficient performance and scalability within AWS cloud-native ecosystems.

Required Skills

• 3-5 years of experience in IT, developing AWS cloud-native data lakes and ecosystems.
• Proficiency in Python, SQL, and PySpark for data engineering.
• Experience with Databricks, Delta Lake, and ETL/ELT processes.
• Strong understanding of data architecture, including medallion architecture and lakehouse concepts.
• Experience designing and implementing RESTful APIs.
• At least 1-2 years of experience in an onshore-offshore delivery model.
• Knowledge of data governance policies and procedures.

Preferred Skills

• Experience with LLM API integration, Vector databases (pgvector, Pinecone, OpenSearch), and RAG pipelines.
• Familiarity with orchestration frameworks, AI agents, and frameworks like LangChain or LlamaIndex.
• Domain knowledge in Enterprise functions such as HR, Finance, Compliance, or Procurement.

Key Skills

Education

Any Graduate