Description

You will design and implement scalable data architectures and ingestion frameworks for structured and unstructured data.

This role is on-site.

Responsibilities

• Build data ingestion pipelines for sources like SharePoint, Confluence, and Jira into the OneData platform.
• Design scalable architectures to process document content including XML, HTML, DOC, PDF, XLS, JPEG, TIFF, and PPT.
• Implement vector databases for efficient storage and retrieval of high-dimensional vectors.
• Develop automated data quality, security, and validation checks to ensure data integrity.
• Monitor pipeline performance, resolve bottlenecks, and manage access policies for cloud storage and vector databases.

Required Skills

• 10+ years of experience in database and cloud-based technologies.
• Proficiency in Python, SQL, and PySpark.
• Hands-on experience with Databricks and Snowflake.
• Experience with Machine Learning and AI concepts, specifically RAG architecture, LLMs, embeddings, and vector data stores.
• Proven track record building pipelines for both structured and unstructured data.
• Experience with Cloud data stores, NoSQL, Graph, and Vector databases.
• Expertise in CI/CD workflows using Jenkins and Terraform.
• Proficiency with Splunk and Dynatrace for monitoring.
• Proficiency with GitHub, Jira, and Confluence.

Key Skills

Education

Any Graduate