Databricks optimizes multi-hop agent retrieval with adaptive search model
The new adaptive retrieval architecture reduces computational overhead for AI agents by dynamically adjusting search steps based on query complexity.
Databricks Inc. introduced an expansion to its Adaptive Instructed-Retriever search model on September 9, 2026, designed to accelerate response times for artificial intelligence agents that require iterative retrieval processes. The model serves as a core component for the company’s Genie Code, Genie One, and Genie Agents, providing a mechanism to optimize how agents navigate complex, multi-source data environments.
The architecture builds upon the earlier Instructed-Retriever-1, which introduced the capability to carry user instructions, data-source schemas, and examples through the retrieval and generation pipeline. While the predecessor focused on parallel, single-step searches for routine queries, the new adaptive model is engineered specifically for tasks that demand evidence from multiple sources or rely on findings from previous search steps.
Databricks reports that the model matches the retrieval quality of several prominent third-party and open-source systems while operating at twice the speed. Comparative tests utilized seven internal and external benchmarks, measuring performance against Anthropic PBC’s Claude Sonnet 5, OpenAI Group PBC’s GPT-5.6 Luna, and Hangzhou DeepSeek Artificial Intelligence Co. Ltd.’s V4-Flash. These results indicate that the model can achieve high recall scores in fewer steps than competing frontier models.
The operational logic of the model centers on a dynamic management system for computational costs. Developers define a maximum number of sequential search steps, and the model autonomously determines the necessary iterations for each specific request. It terminates the search process once it concludes the available evidence is sufficient, or it continues if further rounds are projected to improve the final output.
Training for the model involved synthetic enterprise retrieval environments and multi-hop question sets, incorporating data from its predecessor to maintain single-step efficiency. Databricks applied online reinforcement learning using a Clipped Importance Sampling Policy Optimization technique. This specific technique ensures that the model updates its retrieval policy without deviating too far from previous successful strategies, preventing the instability often associated with rapid policy shifts in reinforcement learning. The reward system prioritizes accurate search trajectories while imposing penalties on additional steps that fail to yield corresponding quality gains.
Varying the penalty weight allows users to select from a family of model checkpoints tailored to specific latency-quality curves. A heavier penalty forces the agent to favor fewer steps for rapid, interactive applications, whereas a lighter penalty permits more extensive searching for complex, offline workloads. This flexibility allows engineers to tune the retrieval behavior based on the specific requirements of their data agents.
The shift toward adaptive retrieval reflects a broader industry focus on reducing the computational overhead inherent in agentic workflows. By learning not only how to search but also when to stop searching, the model addresses the inefficiency of brute-force exploration in large, dynamic workspaces. This capability is particularly relevant for enterprise environments where information is fragmented across disparate systems and requires multi-hop reasoning to synthesize accurate answers.
The performance claims remain subject to verification, as they rely on proprietary benchmarks developed by Databricks rather than independent, third-party evaluations. The company has not yet released specific details regarding parameter counts, pricing structures, or general availability for the new model. The integration of this model into the Databricks AI Runtime suggests that customers will retain the ability to specialize the architecture for their own unique data and performance constraints.
Engineers must evaluate how these adaptive search trajectories perform in production environments with high-concurrency requirements. The ability to minimize unnecessary turns in a search process could significantly reduce the total cost of ownership for organizations scaling agentic systems. Observers will monitor whether this approach to dynamic step-counting becomes a standard feature in future retrieval-augmented generation frameworks, as the industry moves toward more efficient, agent-driven data discovery and automated reasoning pipelines.
