Description

Build and scale batch and streaming data pipelines for the Ignite AI advisor project, integrating AI-driven automation and engineering standards.

This role is hybrid.

Responsibilities

• Design and implement scalable data pipelines using Java, Python, SQL, and JavaScript to ingest data from upstream cloud systems.
• Develop data engineering frameworks to streamline the development and deployment of pipelines across the Ignite landscape.
• Create and refine prompts for Large Language Models (LLMs) to automate data cleansing, transformation logic, and documentation.
• Explore and implement AI agents to monitor data quality, orchestrate workflows, and automate repetitive data management tasks.
• Establish engineering best practices and leverage AI-powered coding assistants to optimize code and accelerate development.

Required Skills

• 5+ years of experience in data engineering, data architecture, or a related data field.
• Strong proficiency in Java, Python, SQL, and JavaScript.
• Experience with build management tools such as Maven or Gradle.
• Hands-on experience with cloud platforms: GCP, AWS, or Azure.
• Proficiency with CI/CD tooling including Jenkins, Docker, and Kubernetes.
• Solid understanding of data modeling, warehousing, and integration principles.

Preferred Skills

• Demonstrated experience with prompt engineering and AI-assisted coding tools.
• Familiarity with AI application frameworks like LangChain or LlamaIndex.
• DevOps experience with Terraform for IaC and GCP infrastructure management.

Key Skills

Education

Any Graduate