Laravel AI: Load Tools On Demand With ToolSearch

Every tool an agent exposes was previously sent to the provider on every request. For an agent with four tools that is fine. For an agent with thirty, the definitions become a large fixed cost on each call, and the model has more options to choose between, which makes its choice less accurate.

Laravel AI v0.11.0 adds a way to defer part of that catalog. @behzadsp contributed a ToolSearch wrapper in #697 that maps onto the hosted tool search OpenAI and Anthropic already offer: the provider receives a search entry plus deferred definitions, and loads a tool's full definition only when it decides to use it.

Wrapping the Tools You Want Deferred

ToolSearch takes the tools to defer as its first argument. Anything left outside the wrapper is sent the way it always was:

use Laravel\Ai\Providers\Tools\ToolSearch;
 
public function tools(): iterable
{
    return [
        new LookupAccount,
        new ToolSearch(tools: [
            new IssueRefund,
            new ChangePlan,
            new ResendInvoice,
            new TransferSeat,
            // ...the rest of the back office catalog
        ]),
    ];
}

The tools themselves need no changes. There is no interface to implement and no per-tool provider option, so the same tool class can be deferred in one agent and sent up front in another.

Anthropic's search strategy is the second constructor argument, regex or bm25, validated when you build the object:

new ToolSearch(tools: [new IssueRefund, new ChangePlan], strategy: 'bm25');

Anything else throws an InvalidArgumentException where you wrote it, and omitting it sends regex. Other fields on the provider's search entry, cache_control among them, pass through withProviderOptions() on the wrapper. On Anthropic the wrapper is the only place a cache breakpoint can go, since their API returns a 400 for any tool carrying both defer_loading: true and cache_control. The package puts provider options on the search entry alone and never on the deferred definitions, so that combination is not one you can build here.

What Each Provider Receives

On OpenAI the wrapper becomes a {"type": "tool_search"} entry. On Anthropic it becomes a versioned type, tool_search_tool_regex_20251119 or the bm25 equivalent. In both cases the deferred tools follow as ordinary definitions carrying defer_loading: true, which mirrors how the two providers model this natively: one search tool, plus per-tool deferral.

When the model calls a deferred tool, the gateway resolves it by name from inside the wrapper exactly as it resolves a top-level tool, so execution and results behave the same.

The two providers report the search differently in the response. Anthropic returns a server_tool_use block holding the pattern or query Claude searched with, then a tool_search_tool_result block listing tool_reference entries for what it found, and expands those into full definitions before Claude sees them. That block is not a tool call you answer: returning a tool_result for its srvtoolu_ ID makes Anthropic reject the request. OpenAI resolves hosted search inside the same call and reports it as tool_search_call and tool_search_output items. Laravel AI maps the request side only, so neither shape is parsed into anything the SDK exposes.

Nothing has to stay outside the wrapper. A ToolSearch holding every one of an agent's tools maps the same way, with the search entry as the only definition sent that is not deferred.

Checking What Was Sent

Fake the HTTP call and inspect the tools array. A deferred definition carries defer_loading: true and a top-level one does not:

use Illuminate\Http\Client\Request;
use Illuminate\Support\Facades\Http;
 
Http::fake(['*' => Http::response([
    'id' => 'resp_123',
    'status' => 'completed',
    'model' => 'gpt-5.4',
    'output' => [[
        'type' => 'message',
        'status' => 'completed',
        'content' => [['type' => 'output_text', 'text' => 'ok']],
    ]],
    'usage' => ['input_tokens' => 1, 'output_tokens' => 1],
])]);
 
(new SupportAgent)->prompt('Refund order 41');
 
Http::assertSent(function (Request $request) {
    $tools = collect(data_get($request->data(), 'tools'));
 
    $deferred = $tools->firstWhere('name', 'IssueRefund');
    $plain = $tools->firstWhere('name', 'LookupAccount');
 
    return $tools->contains(fn ($tool) => ($tool['type'] ?? null) === 'tool_search')
        && ($deferred['defer_loading'] ?? false) === true
        && ! isset($plain['defer_loading']);
});

The names come from ToolNameResolver, which returns the tool's class basename unless the class defines a name() method. On Anthropic the same assertion holds with tool_search_tool_regex_20251119 in place of tool_search, and the faked body is Anthropic's own response shape rather than the OpenAI one above.

An agent whose only tool is an empty wrapper sends no tools key at all, and no tool_choice.

Provider Support and Limits

Only OpenAiProvider and AnthropicProvider implement the SupportsToolSearch marker. Every other provider throws a LogicException before a request goes out, rather than dropping the wrapper's tools. That covers Azure, the chat-completions gateways (Groq, DeepSeek, Mistral, OpenRouter), Gemini, xAI, Bedrock, and Ollama. Whether a given model supports it is enforced by the provider's API, not locally.

Three more rules govern how the wrapper is used:

The support check runs whenever a wrapper is present, including an empty one, so a misconfigured provider throws in development instead of the first time the deferred list is non-empty in production.
Only one wrapper may be registered per request. A second ToolSearch throws.
On OpenAI, the package rejects a ToolSearch combined with store=false rather than letting the request fail at the API.

An empty wrapper on a supported provider emits nothing.

Step Budgeting

When you do not set maxSteps yourself, the SDK derives a default from how many tools the agent has. If a wrapper counted as one tool, wrapping twenty tools would cut the default step budget to a fraction of what the same agent had before, and runs would stop before the agent finished.

ToolSearch::budget() expands each wrapper into its deferred tool count, so the step budget does not change based on how the tools are packaged.

When to Use It

The generation loop counts one step per round-trip to the provider, and it only makes another round-trip when the model calls a tool the SDK executes locally. The provider loads a deferred definition inside its own call, so the search never adds a step of its own.

A deferred tool the model calls runs as an ordinary step, the same as it would have unwrapped. Wrapping changes what the provider is told about a tool, not how it runs.

You save prompt tokens and spend time inside the provider's call instead. Weigh that against the size of the catalog. Four tools do not send enough definitions to be worth deferring, and the model has to search before it can call one.

Two limits apply whatever the size of the catalog. Only OpenAI and Anthropic support the wrapper, so a failover chain throws when it reaches any other provider. OpenAI's hosted search also requires stored responses, so an application that sets store=false to keep prompts off the provider's servers cannot use it there.

Client-Side Tool Search

Ranking a catalog inside the library, which would work on every provider, is a separate feature. #805 proposed it alongside this pull request and was closed without merging, so it does not ship today. ToolSearch is the hosted wrapper: the provider does the searching, and only OpenAI and Anthropic support it.