Nvidia released five Nemotron 3.5 models for retrieving, ranking and extracting information from documents on 21 October, extending its agent-model lineup into the search stage that precedes an answer. In a release posted on Hugging Face, the company reported that adding its new reranker reduced agent inference-token use per query by approximately 13% or 14%, depending on which embedding model supplied the initial results, while improving retrieval accuracy in both tests.
Key points
- Three Embed variants retrieve text and visual evidence, while an 8B Rerank model orders candidates and a 631M-parameter Parse model handles document extraction.
- Nvidia reports approximately 14% fewer agent inference tokens per query with Embed 8B and Rerank, or 13% fewer with Embed 1B and Rerank.
- The agent tests covered a mixed text-and-image benchmark and two text-only benchmarks, with the retrieval configuration changed between runs.
Five Nemotron 3.5 models divide the search
The release comprises Nemotron 3.5 Embed 8B BF16, Embed 1B BF16, Embed 1B NVFP4, Rerank 8B BF16 and Parse at 631M parameters. Nvidia describes them as open, commercially available models. The Embed variants find candidate evidence across document text and images, Rerank orders candidates by relevance to a query, and Parse is for document extraction. Those are separate jobs in a workflow that may have to locate information before an agent can reason over it.
Embed can retrieve material carried by tables, charts, infographics, datasheets and schematics as well as by written passages. Rerank is an 8B cross-encoder: it evaluates each query together with a candidate document, using text and image inputs to refine the order of the results. The distinction is like finding a stack of potentially relevant files, then checking each file against the question before deciding which ones to read first. Nvidia says that getting more relevant evidence into the agent’s context can reduce repeated searches and the tokens spent examining less useful documents.
Checking whether stock could run out next quarter would call for comparing demand projections in charts, available quantities in tables and how long deliveries take according to written documents. Retrieving those visual and written details together could give an agent more relevant material to inspect before it reasons about whether replenishment would arrive in time. Ranking the candidates could also reduce the material it would search through again.
The release follows a different addition to the family on 15 October: Nvidia introduced Nemotron 3.5 Super VL, a 120-billion-parameter model for multimodal agent tasks, and two Bolt models aimed at local deployment, according to its announcement. AI Affairs reported that earlier Super VL and Bolt release. The new Embed and Rerank models address the evidence an agent receives, rather than the generation model that works through it.
Nvidia holds the agent fixed in retrieval tests
Nvidia tested Embed and Rerank on ViDoRe v3, which combines text and image inputs, and on BRIGHT and BrowseComp-Plus, which use text-only inputs. Its test agent searched a collection of documents, inspected what it found and could revise its search. Nvidia changed the search setup between runs — embedding alone, or embedding followed by reranking, with different models in those roles — while keeping the language model, tools, tasks, document collection and agent execution policy fixed.
For each task, Nvidia measured whether the agent found the relevant information and counted its input and output inference tokens across all turns. On that setup, the company reports approximately 14% fewer agent tokens per query when Rerank followed Embed 8B, and approximately 13% fewer when it followed Embed 1B. Retrieval accuracy improved in both configurations. The percentages concern the agent tokens counted in Nvidia’s tests, with the search configuration changed and the other stated elements held constant.
Nvidia also compared its reranker with Qwen3-VL-Reranker-8B when Embed 8B provided the initial candidates. It reports approximately 0.9 points higher nDCG@10 for Nemotron 3.5 Rerank at a similar agent token cost. nDCG@10 measures how well a ranking places relevant results among its first ten entries; here the reported advantage is a ranking result under the company’s chosen retriever and agent setup.
For companies building agents, the tested change sits in the search tool rather than in the agent’s underlying language model. The three evaluated task sets include both visual-document retrieval and text-only search, while the reported token reductions come from the complete agent runs, including searches it chose to repeat. Nvidia’s comparison therefore puts a count on the work an agent did after receiving results from each retrieval configuration.
Embed spans 34 text-retrieval languages
Nvidia evaluated text retrieval across 34 languages. For multilingual visual retrieval it used MIRACL Vision, covering 18 languages, and KoViDoRe. The company says the model may work in additional languages beyond those evaluated. These tests matter to the document search described in the release because the evidence can be carried by an image as well as by text, and either may need to be found in more than one language.
The two smaller Embed variants offer different deployment formats. Nvidia describes Embed 1B BF16 as a compact version distilled from the 8B model and says the Embed 1B NVFP4 version applies quantisation to both its text and image encoders for retrieval on supported hardware. Nvidia also says its embedding fine-tuning recipes support adaptation to custom document collections.
