Visual document retrieval (late interaction)
Search scanned forms, slides, drawings and table-heavy PDFs as images, with no OCR pipeline to maintain.
Use it when
Corpora where layout carries meaning: engineering drawings, slide decks, scanned contracts, invoices and financial tables that OCR-based pipelines flatten and lose.
The parts, top to bottom
across every level
Hover or tap any part to see what it does. The light shows the order a request moves through.
What happens, in order
- 1Rasterise each page
- 2Encode into patch embeddings (ColPali / ColQwen)
- 3Store multi-vectors with MaxSim support
- 4Query encoded token by token
- 5Top pages passed to a vision model
- 6Answer with page citations
What we typically build it with
Trade-offs
Higher storage per page (about a thousand patch vectors) and heavier queries, but ColPali scored nDCG@5 of 81.3 on the ViDoRe benchmark versus roughly 65–75 for OCR pipelines. Worth it when documents are visual; unnecessary for clean text.
Problems this architecture solves
Generic problem statements with the flow and the outcomes the industry has documented.
Documents keyed in by hand
Invoices, KYC files, claims and contracts arrive as PDFs and photos. People retype them, errors slip through, and the backlog grows every month-end.
How the system works
- Ingest PDF / image / email
- Classify document type
- Extract to a schema
- Validate & score confidence
- Auto-post or route to review
Outcome: Documented deployments classify documents in under a second, turn day-long claims backlogs into minutes, and reach 85% no-touch invoice processing within six months with a seven-month payback. DXC with Claude; Vic.ai
ArchitectureAnswers locked in scans, slides and drawings
Engineering drawings, slide decks, scanned contracts and table-heavy PDFs lose their meaning when flattened to text, so search returns nothing useful.
How the system works
- Rasterise pages
- Patch embeddings (ColPali)
- Multi-vector index
- Retrieve pages, not chunks
- Vision model answers with citations
Outcome: Late-interaction retrieval over page images scored 81.3 nDCG@5 on the ViDoRe benchmark against roughly 65–75 for OCR pipelines, with no OCR stack to maintain. ColPali (ICLR 2025)
ArchitectureOther retrieval patterns
Hybrid RAG with reranking and citations
Keyword plus vector search, fused and reranked, answered with sources.
Use it when: Questions over a body of documents, tickets or records where exact terms matter as much as meaning, and where every answer must be checkable and permission-aware.
Flow
- 1Ingest & chunk by structure
- 2Enrich with context & metadata
- 3Embed + keyword index
- 4Query: BM25 + vectors, fuse (RRF)
- 5Rerank top candidates
Agentic and graph-based retrieval
The agent decides how to search; a knowledge graph answers the global questions.
Use it when: Multi-hop questions, comparisons across many documents, or questions about themes across a whole corpus where chunk retrieval alone returns fragments.
Flow
- 1Question analysed
- 2Agent chooses strategy: search, graph, SQL, or ask
- 3Runs several retrievals
- 4Verifies & consolidates evidence
- 5Answers with a reasoning trail
Let's build intelligent systems that drive growth
Tachyon is the engineering partner for teams that need AI in production, not in a deck. Start with a free 60-minute discovery call.
Tell us the problem, we will map it to the architecture
Every great partnership begins with a conversation. Whether you are exploring possibilities or ready to scale, tell us what you are actually trying to build.
- ubheshubham.37@gmail.com
- +91 84592 96471
- Clients worldwide · English
- Pune, India · Headquarters