Description

You will lead the design and implementation of scalable data architectures and ingestion frameworks for structured and unstructured data.

Responsibilities

• Build data ingestion frameworks and pipelines to move data from sources like SharePoint, Confluence, and Jira into the OneData platform.
• Design scalable target state architectures for processing document content including XML, HTML, DOC, PDF, XLS, JPEG, TIFF, and PPT.
• Implement vector databases for efficient storage and retrieval of high-dimensional vectors.
• Develop automated data quality, security, and validation checks to ensure data integrity and maintainability.
• Monitor pipeline performance, resolve bottlenecks, and manage access policies for cloud storage and vector databases.

Required Skills

• 10+ years of experience in database and cloud-based technologies.
• Proficiency in Python, SQL, and PySpark.
• Hands-on experience with Databricks and Snowflake.
• Experience with Machine Learning and AI concepts, specifically RAG architecture, LLMs, embeddings, and vector data stores.
• Proven track record building pipelines for both structured and unstructured data.
• Experience with Cloud data stores, NoSQL, Graph, and Vector databases.
• Expertise in CI/CD workflows using Jenkins, Terraform, Splunk, and Dynatrace.
• Proficiency with GitHub, Jira, and Confluence.

Preferred Skills

• Master’s degree in a technical field.

Key Skills

Education

Any Graduate