Description

You will design, manage, and optimize data architectures to support AI and machine learning workflows.

This role is remote.

Responsibilities

• Build scalable, automated data pipelines and integrate large-scale datasets.
• Develop and deploy machine learning models using AWS infrastructure.
• Implement data classification processes to identify PII entities.
• Optimize LLM outputs using re-ranking techniques, index stores, and vector stores.
• Manage complex data processing tasks using AWS ETL services.

Required Skills

• 5+ years of professional experience in data engineering.
• Proficiency in Python or Scala.
• Hands-on experience with TensorFlow and PyTorch.
• Practical knowledge of Retrieval-Augmented Generation (RAG) and agent-based workflows.
• Experience with AWS Glue, Lambda, and Data Pipeline.
• Expertise in AWS core services: IAM, VPC, EC2, S3, RDS, CloudWatch, and CloudTrail.
• Proven ability to leverage AWS SageMaker, Comprehend, and Entity Resolution.
• Deep understanding of index and vector stores for improving LLM outputs.

Preferred Skills

• Experience managing data privacy and compliance requirements for PII data.
• Familiarity with advanced data indexing techniques and vector databases.

Key Skills

Education

Not specified