Advancing Tool Support - Part 3 #2121
Conversation
| /** | ||
| * Merge the given {@link ChatOptions} into this instance. | ||
| */ | ||
| public ToolCallingChatOptions merge(ChatOptions options) { |
There was a problem hiding this comment.
This method would not by called by any class since we always use the logic in ModelOptionsUtil to merge options, so I removed this.
| } | ||
|
|
||
| @Override | ||
| public String convert(ToolExecutionException exception) { |
There was a problem hiding this comment.
I added this simple logic to handle tool execution errors. We can evolve this in separate PRs. Users have the possibility to provide their own handler.
| * @author Thomas Vitale | ||
| * @since 1.0.0 | ||
| */ | ||
| public class SpringBeanToolCallbackResolver implements ToolCallbackResolver { |
There was a problem hiding this comment.
It's pretty much the same logic as DefaultFunctionCallbackResolver. I also added autotests since we didn't have any for this logic.
|
|
||
| private final Map<String, ToolCallback> toolCallbacks = new HashMap<>(); | ||
|
|
||
| public StaticToolCallbackResolver(List<ToolCallback> toolCallbacks) { |
There was a problem hiding this comment.
In the auto configuration, this can be populated by all the ToolCallbacks registered as beans in Spring (via @bean or @component).
| import static org.assertj.core.api.Assertions.assertThat; | ||
|
|
||
| /** | ||
| * Unit tests for {@link ChatResponse}. |
There was a problem hiding this comment.
I was surprised to see we didn't have any auto-tests for ChatResponse. It's worth adding them (in a separate task) while also making it null-safe, because there are a lot of paths ending in NullPointerException inside this class
| * | ||
| * @author Thomas Vitale | ||
| */ | ||
| class SpringBeanToolCallbackResolverKotlinTests { |
There was a problem hiding this comment.
Added extra tests to make sure the resolution via Spring beans works correctly also in Kotlin
| runExplicitToolCallingExecutionWithOptionsStream(chatOptions, prompt); | ||
| } | ||
|
|
||
| private void runExplicitToolCallingExecutionWithOptions(ChatOptions chatOptions, Prompt prompt) { |
There was a problem hiding this comment.
This shows how to do the tool execution on the client-side. It should be very straightforward using the new ToolCallingManager.
| @Test | ||
| void explicitToolCallingExecutionWithLegacyOptions() { | ||
| ChatOptions chatOptions = FunctionCallingOptions.builder() | ||
| .functionCallbacks(ToolCallbacks.from(tools)) |
There was a problem hiding this comment.
It's backward compatible with the deprecated FunctionCallingOptions
* Introduced ToolCallingManager to manage the tool calling activities for resolving and executing tools. A default implementation is provided. It can be used to handle explicit tool execution on the client-side, superseding the previous FunctionCallingHelper class. It’s ready to be instrumented via Micrometer, and support exception handling when tool calls fail. * Introduced ToolCallExceptionConverter to handle exceptions in tool calling, and provided a default implementation propagating the error message to the chat morel. * Introduced ToolCallbackResolver to resolve ToolCallback instances. A default implementation is provided (DelegatingToolCallbackResolver), capable of delegating the resolution to a series of resolvers, including static resolution (StaticToolCallbackResolver) and dynamic resolution from the Spring context (SpringBeanToolCallbackResolver). * Improved configuration in ToolCallingChatOptions to enable/disable the tool execution within a ChatModel (superseding the previous proxyToolCalls option). * Added unit and integration tests to cover all the new use cases and existing functionality which was not covered by autotests (tool resolution from Spring context). * Deprecated FunctionCallbackResolver, AbstractToolCallSupport, and FunctionCallingHelper. Relates to spring-projectsgh-2049 Signed-off-by: Thomas Vitale <[email protected]>
ThomasVitale
force-pushed
the
gh-2049-part-c
branch
from
449d6ec to
f678969
Compare
ToolCallingManagerto manage the tool calling activities for resolving and executing tools. A default implementation is provided. It can be used to handle explicit tool execution on the client-side, superseding the previousFunctionCallingHelperclass. It’s ready to be instrumented via Micrometer, and support exception handling when tool calls fail.ToolCallExceptionConverterto handle exceptions in tool calling, and provided a default implementation propagating the error message to the chat morel.ToolCallbackResolverto resolveToolCallbackinstances. A default implementation is provided (DelegatingToolCallbackResolver), capable of delegating the resolution to a series of resolvers, including static resolution (StaticToolCallbackResolver) and dynamic resolution from the Spring context (SpringBeanToolCallbackResolver).ToolCallingChatOptionsto enable/disable the tool execution within a ChatModel (superseding the previous proxyToolCalls option).FunctionCallbackResolver,AbstractToolCallSupport, andFunctionCallingHelper.Relates to gh-2049