Tool Calling & Programmatic Prompts
Tools let the LLM call methods in your application during a conversation — query a database, look up an order, send an email, or any other action you want the assistant to be able to perform. Vaadin supports two ways to expose tools: tool objects registered with the AI framework (covered on this page) and framework-agnostic Controllers.
A third source of tools is an MCP server, whose tools the AI framework’s own client contributes to each request. See Framework Features.
Tool Calling
A tool object has methods with the @Tool annotation of the AI framework. The framework offers these methods to the LLM as tools and runs them when the LLM calls them. Register the tool objects with the framework when you create the LLM provider. The annotation and the place to register depend on the framework.
For Spring AI, use @org.springframework.ai.tool.annotation.Tool with a description attribute. Add the tool objects to the ChatClient that you pass to the provider:
Source code
Java
public class WeatherTools {
// Spring AI: org.springframework.ai.tool.annotation.Tool
@Tool(description = "Get current weather for a city")
public String getWeather(String city) {
return weatherService.getCurrentWeather(city);
}
}
ChatClient chatClient = ChatClient.builder(chatModel)
.defaultTools(new WeatherTools())
.build();
SpringAILLMProvider provider = new SpringAILLMProvider(chatClient);For LangChain4j, use @dev.langchain4j.agent.tool.Tool, whose description is the annotation’s value.
Add the tool objects in a customizer of the provider. LangChain4JLLMProvider takes the customizer as an optional second constructor argument, and passes it the builder of the LangChain4j AI service that it runs each turn through:
Source code
Java
public class WeatherTools {
// LangChain4j: dev.langchain4j.agent.tool.Tool
@Tool("Get current weather for a city")
public String getWeather(String city) {
return weatherService.getCurrentWeather(city);
}
}
LangChain4JLLMProvider provider = new LangChain4JLLMProvider(chatModel,
service -> service.tools(new WeatherTools()));Every conversation that runs through the same ChatClient or provider calls the same tool objects. When a tool works with the state of one view, such as its components, create the client or the provider for that view.
withTools() on the orchestrator builder, and after reconnect(), takes the same tool objects, but it’s deprecated. Register the tool objects with the AI framework instead.
|
Note
|
Tool Threading
With a streaming provider or background execution, tool methods are invoked off the request thread, where UI.getCurrent() and other Vaadin thread locals are not available. Wrap component access in ui.access(), or capture the needed state before the turn starts.
|
|
Tip
|
Framework-Agnostic Tools via Controllers
For a reusable set of tools that does not depend on a specific LLM framework’s annotations, or when a lifecycle hook is needed after each LLM request cycle, implement AIController instead. GridAIController, ChartAIController, and FormAIController are built-in examples. A controller can be combined with tool objects registered with the AI framework.
|
|
Note
|
Tool Errors
Tool objects are executed by the AI framework, whose own error handling decides what the LLM sees when a tool throws — by default, both LangChain4j and Spring AI relay the raw message of any exception. To control what the LLM learns about failures, replace that default in the framework:
You can also define the tool through a controller instead, and throw a |
Tool Call Limits
A turn that uses tools is a loop: the model asks for tool calls, the provider runs them and calls the model again with the results, until the model answers. Both built-in providers bound that loop per turn. By default, the model may call any single tool at most 40 times and all tools together at most 150 times in one turn. Without a bound, a request the model can’t satisfy keeps calling the model and the tools until the application is stopped. The limits count every tool the model can call, whether it comes from a tool object, a controller, or an MCP server.
When a limit is exceeded, the turn fails with a ToolCallLimitExceededException, whichever provider runs it. The Message List shows its generic error message, and the exception reaches the response listener and AIController.onResponse() as the error of the turn. Its message names the limit that was exceeded and, for a per-tool limit, the tool:
Source code
Java
.withResponseListener(event -> {
event.getError()
.filter(ToolCallLimitExceededException.class::isInstance)
.ifPresent(error -> log.warn(
"Turn stopped: {}", error.getMessage()));
})LangChain4j
LangChain4JLLMProvider runs the loop itself and enforces the limits. Change them with setMaxCallsPerTool() and setMaxTotalToolCalls(). A value of 0 removes that limit, and a negative value throws an IllegalArgumentException. The values are read when a turn starts, so a change applies from the next prompt.
Source code
Java
LangChain4JLLMProvider provider = new LangChain4JLLMProvider(chatModel);
provider.setMaxCallsPerTool(10);
provider.setMaxTotalToolCalls(0); // No limit across all toolsThe provider checks the limits before running the tool calls of a response. When a call would take a count past its limit, none of the tool calls in that response run, and the model isn’t called again. The failed turn leaves only its prompt in the provider’s memory: tool calls and their results are sent to the model only within the turn they belong to and are never kept between turns, so the next prompt continues from where the last completed turn left the conversation. Metadata observed before the failure is still published, so the turn reports the token usage of the round trips that did complete.
Spring AI
SpringAILLMProvider adds no limit of its own. Spring AI bounds the loop with the same defaults, 40 calls per tool and 150 in total per turn, with either constructor of the provider. When Spring AI stops the loop, the provider fails the turn with the ToolCallLimitExceededException described above, carrying Spring AI’s own message about the limit. The finish reason toolCallLimitExceeded that Spring AI puts on that reply is still published in the response metadata, and the reply may remain in the chat memory, with either constructor: the provider doesn’t rewrite what Spring AI’s advisors stored.
The limits belong to the ChatClient, so tune or remove them there and pass the client to the ChatClient constructor of the provider; the ChatModel constructor builds a client with Spring AI’s defaults. In a Spring Boot application, the spring.ai.tools.limits properties configure them on the auto-configured ChatClient.Builder. How to configure them is described under Tool Call Limits in the Spring AI documentation.
|
Note
|
Custom Providers
A custom LLM provider that runs its own tool-calling loop decides itself whether and how to bound it. The ToolCallLimitExceededException constructor is public, so such a provider can end a turn with the same exception, and a ResponseListener or AIController can be unit-tested against it.
|
Programmatic Prompts
Send prompts without a Message Input component using prompt(). This is useful for triggering AI interaction from button clicks or other events:
Programmatic prompts go through the same pipeline as chat submits: a configured request interceptor can change or reject them before anything is sent.
|
Note
|
One Request at a Time
The orchestrator processes one prompt at a time. If prompt() is called while a previous request is still streaming, the new call is logged as a warning and silently dropped. Wait for the current response to complete before sending another prompt.
|
|
Important
|
UI Context Required
prompt() requires an active UI context. If called from a background thread or outside a Vaadin request, it throws an IllegalStateException. Always call prompt() from within a UI event handler or wrap the call in ui.access().
|