Description

Build and deploy entity resolution systems that serve as the authoritative source of truth for customer data.

Responsibilities

• Develop probabilistic matching and ML models (gradient boosting, neural classifiers) for record linkage across large-scale datasets.
• Build candidate blocking pipelines using phonetic algorithms, token similarity, and LSH to handle billions of potential match pairs.
• Implement fuzzy matching and clustering algorithms to create unified golden customer profiles.
• Integrate LLMs for classification, disambiguation, and structured data extraction from unstructured sources.
• Build RAG pipelines and NLQ-to-SQL systems to enable semantic search and natural language querying of customer data.

Required Skills

• Python programming with scikit-learn, HuggingFace Transformers, RapidFuzz, and jellyfish.
• 3+ years of experience in ML/AI engineering with at least 1 year in entity resolution or record linkage.
• Strong SQL skills and experience with Spark or Dask for distributed data processing.
• Experience with LLM APIs (OpenAI, Anthropic) and prompt engineering pipelines.
• Knowledge of embedding-based similarity systems (Sentence-BERT) and ANN/KNN retrieval (FAISS, Annoy).
• Familiarity with vector databases (Pinecone, pgvector, Qdrant) and ML lifecycle tools (MLflow).
• Bachelor's or Master's degree in Computer Science, Data Science, or related field.

Preferred Skills

• Experience handling messy, real-world datasets spanning hundreds of millions of records.
• Understanding of precision/recall tradeoffs in identity resolution and data quality metrics.

Key Skills

Education

Bachelor's or Master's degrees