Inverting Multi-Vector Visual Document Indices
Paper • 2610.09920 • Published • 18
👋 Open to Work
Zhuchenyang Liu
Ryenhails
P(doom) 10%
AI & ML interests
None yet
Recent Activity
authored a paper about 16 hours ago
Inverting Multi-Vector Visual Document Indices upvoted a paper about 22 hours ago
Inverting Multi-Vector Visual Document Indices submitted a paper about 22 hours ago
Inverting Multi-Vector Visual Document IndicesOrganizations
authored a
paper about 16 hours ago
upvoted a paper about 22 hours ago
submitted a
paper to Daily Papers about 22 hours ago
updated a
Space 1 day ago
published a
model 1 day ago
updated a
model 1 day ago
CPU-friendly query encoder for EVIE-4.5B
1
#2 opened 5 days ago
by Ryenhails
published a
model 6 days ago
updated 7
models 6 days ago
nanovdr/ColNanoVDR-Q-Ettin150M-EVIE45B-2048-ML
Sentence Similarity • 0.1B • Updated • 29 • 1
nanovdr/ColNanoVDR-Q-Ettin400M-ColQwen35-320-ML
Sentence Similarity • 0.4B • Updated • 73 • 2
nanovdr/ColNanoVDR-Q-Ettin150M-Vultron45B-320-ML
Sentence Similarity • 0.1B • Updated • 62 • 2
nanovdr/ColNanoVDR-Q-Ettin150M-Tomoro8B-320-ML
Sentence Similarity • 0.1B • Updated • 67 • 2
nanovdr/ColNanoVDR-Q-Ettin150M-ColVec8B-640-ML
Sentence Similarity • 0.1B • Updated • 75 • 4
nanovdr/ColNanoVDR-Q-Ettin150M-ColVec4B-640-ML
Sentence Similarity • 0.1B • Updated • 119 • 3
nanovdr/ColNanoVDR-Q-Ettin150M-ColQwen35-320-ML
Sentence Similarity • 0.1B • Updated • 112 • 2
Lightweight text-only query encoder for webAI-ColVec1.1-4b (CPU-friendly)
#3 opened 7 days ago
by Ryenhails
Lightweight text-only query encoders for colqwen3.5-4.5B-v3 (CPU-friendly)
#4 opened 7 days ago
by Ryenhails
reacted to their post with 🔥 9 days ago
Post
3705
🚀 NanoVDR goes multi-vector: meet ColNanoVDR!
Multi-vector VLM retrievers lead visual document retrieval, but every search runs a multi-billion-parameter query encoder. We distill that encoder into a 149M text-only student that queries the teacher's existing page index directly. No re-indexing, and no pages during training.
🧠 How: OTW (Optimal Transport with Learned Weights) aligns the student's query tokens with the teacher's, even though the two tokenize differently (e.g. 17 vs 29 tokens). We prove the alignment cost bounds the MaxSim score gap on every page, so training only needs cached teacher query tokens.
📊 Five teachers → five 149M students, ViDoRe v3 NDCG@5:
- ColVec1.1-8b: 62.6 → 60.1 (96.0%)
- ColVec1.1-4b: 61.6 → 59.1 (95.8%)
- Vultron-4.5B: 61.0 → 58.3 (95.5%)
- ColQwen3.5-4.5B: 58.7 → 55.1 (93.8%)
- Tomoro-ColQwen3-8B: 59.0 → 54.9 (93.0%)
⚡ 26× faster query encoding on a single CPU thread (87 ms vs 2.3 s)
💾 Matches score distillation while reading 12.6× less cached teacher data
📄 Paper: ColNanoVDR: Document-Free Query Distillation for Multi-Vector Visual Document Retrieval via Optimal Transport (2609.34899)
🤗 Checkpoints (all five students): nanovdr
💻 Code: https://github.com/Ryenhails/NanoVDR
🧩 Single-vector predecessor, NanoVDR: https://arxiv.org/abs/2603.12824
If you already serve one of these teachers, swap in the matching student and keep your index as is. Feedback and upvotes welcome! 🙌
Multi-vector VLM retrievers lead visual document retrieval, but every search runs a multi-billion-parameter query encoder. We distill that encoder into a 149M text-only student that queries the teacher's existing page index directly. No re-indexing, and no pages during training.
🧠 How: OTW (Optimal Transport with Learned Weights) aligns the student's query tokens with the teacher's, even though the two tokenize differently (e.g. 17 vs 29 tokens). We prove the alignment cost bounds the MaxSim score gap on every page, so training only needs cached teacher query tokens.
📊 Five teachers → five 149M students, ViDoRe v3 NDCG@5:
- ColVec1.1-8b: 62.6 → 60.1 (96.0%)
- ColVec1.1-4b: 61.6 → 59.1 (95.8%)
- Vultron-4.5B: 61.0 → 58.3 (95.5%)
- ColQwen3.5-4.5B: 58.7 → 55.1 (93.8%)
- Tomoro-ColQwen3-8B: 59.0 → 54.9 (93.0%)
⚡ 26× faster query encoding on a single CPU thread (87 ms vs 2.3 s)
💾 Matches score distillation while reading 12.6× less cached teacher data
📄 Paper: ColNanoVDR: Document-Free Query Distillation for Multi-Vector Visual Document Retrieval via Optimal Transport (2609.34899)
🤗 Checkpoints (all five students):
💻 Code: https://github.com/Ryenhails/NanoVDR
🧩 Single-vector predecessor, NanoVDR: https://arxiv.org/abs/2603.12824
If you already serve one of these teachers, swap in the matching student and keep your index as is. Feedback and upvotes welcome! 🙌
authored a
paper 10 days ago
updated a
Space 10 days ago