Description

You will design, manage, and optimize data architectures specifically built to support AI and Machine Learning workflows.

Responsibilities

• Build scalable, automated data pipelines and integrate large-scale datasets.
• Develop and deploy machine learning models and frameworks at scale using AWS infrastructure.
• Implement data classification processes, specifically for identifying PII data entities.
• Optimize LLM outputs using re-ranking techniques, index stores, and vector stores.
• Utilize AWS ETL services to manage complex data processing and integration tasks.

Required Skills

• 5+ years of professional experience in data engineering or related fields.
• Proficiency in Python or Scala.
• Hands-on experience with machine learning frameworks including TensorFlow and PyTorch.
• Practical knowledge of Retrieval-Augmented Generation (RAG) and agent-based workflows.
• Experience with AWS ETL services such as AWS Glue, Lambda, and Data Pipeline.
• Expertise in AWS core services: IAM, VPC, EC2, S3, RDS, Lambda, CloudWatch, and CloudTrail.
• Proven ability to leverage AWS SageMaker, Comprehend, and Entity Resolution.
• Deep understanding of index and vector stores for improving LLM outputs.

Preferred Skills

• Experience managing data privacy and compliance requirements for PII data.
• Familiarity with advanced data indexing techniques and vector databases.

Key Skills

Education

Not specified