feat: add tool/function calling support to Anthropic Messages API #7956

Copilot · 2026-01-10T11:51:40Z

Summary: Tool/Function Calling Support for Anthropic Messages API ✅

Successfully implemented comprehensive tool/function calling support for the Anthropic Messages API endpoint (/v1/messages), bringing it to feature parity with the OpenAI chat completions endpoint.

Completed Tasks

Test Coverage

The implementation includes comprehensive E2E tests that validate:

Non-streaming tool calls: Verifies the model calls tools and returns proper tool_use blocks with stop_reason set to "tool_use"
Streaming tool calls: Ensures streaming events (content_block_start/delta/stop) are properly emitted and tool_use blocks are correctly accumulated
Tool choice handling: Tests the tool_choice parameter with auto mode
Multi-turn conversations: Validates handling of tool_result blocks in follow-up messages

All tests require the model to actually call the provided tools and will fail if tools are not used as expected.

Key Features

✅ Tool Definitions: Full support for Anthropic's tool schema with name, description, and input_schema
✅ Tool Choice Options:

"auto" - model decides
"any" - force tool use
{"type": "tool", "name": "..."} - force specific tool
✅ Tool Use: Returns tool_use content blocks with ID, name, and parsed input
✅ Tool Results: Handles tool_result blocks in messages for multi-turn tool interactions
✅ Streaming: Incremental tool call detection and proper event emission
✅ Compatibility: Uses same internal functions as OpenAI endpoint for consistency

Implementation Quality

✅ No compilation errors
✅ All tests passing
✅ No security vulnerabilities
✅ Follows existing code patterns
✅ Proper error handling
✅ Comprehensive logging

The implementation is production-ready and fully compatible with the Anthropic Messages API specification.

Original prompt

Summary

Add tool/function calling support to the Anthropic Messages API endpoint (/v1/messages) to match the functionality already available in the OpenAI chat completions endpoint. This is a follow-up to PR #7948 which added basic Anthropic Messages API support.

Background

PR #7948 added the Anthropic Messages API endpoint to LocalAI, enabling it to act as a drop-in replacement for Anthropic's Claude API. However, the current implementation only supports basic chat completions without tool/function calling capabilities.

The OpenAI endpoints already have robust tool calling support via:

Tools and ToolsChoice fields in OpenAIRequest

ToolCall schema type for responses

Processing via processTools function in core/http/endpoints/openai/chat.go

Requirements

1. Schema Updates (core/schema/anthropic.go)

Add the following to support Anthropic's tool use format (https://docs.anthropic.com/claude/docs/tool-use):

Add Tools field to AnthropicRequest - array of tool definitions

Add ToolChoice field to AnthropicRequest - can be "auto", "any", or {"type": "tool", "name": "tool_name"}

Add AnthropicTool struct with fields:

Name (string)

Description (string)

InputSchema (map[string]interface{}) - JSON Schema for parameters

Update AnthropicContentBlock to fully support:

type: "tool_use" with id, name, and input fields

type: "tool_result" with tool_use_id, content, and optional is_error fields

Add AnthropicStreamDelta support for input_json_delta type (for streaming tool call arguments)

2. Endpoint Updates (core/http/endpoints/anthropic/messages.go)

Convert Anthropic tools to the internal functions.Functions format used by LocalAI

Pass tools to the template evaluator and model inference (similar to OpenAI chat endpoint)

Detect when the model wants to use tools (parse tool call output)

Return tool_use content blocks in responses when tools are called

Set stop_reason to "tool_use" when returning tool calls

Handle incoming tool_result content blocks in messages from clients

3. Message Conversion Updates

Update convertAnthropicToOpenAIMessages() to:

Handle tool_use content blocks (convert to internal tool call format)

Handle tool_result content blocks (convert to internal tool response format)

Preserve tool call IDs for proper request/response matching

4. Streaming Support

For streaming responses with tool calls:

Emit content_block_start event with type: "tool_use", id, and name

Emit content_block_delta events with type: "input_json_delta" containing partial JSON

Emit content_block_stop event when tool call is complete

Set appropriate stop_reason in message_delta

5. Tool Choice Handling

Map Anthropic's tool_choice options:

"auto" (default) - model decides whether to use tools

"any" - model must use one of the provided tools

{"type": "tool", "name": "..."} - model must use the specific named tool

6. Tests

Add tests for:

Schema serialization/deserialization with tools

Tool call detection and parsing

Streaming with tool calls

E2E tests using the anthropic-sdk-go client with tools

Reference Implementation

The OpenAI implementation in core/http/endpoints/openai/chat.go shows how tool calling is currently handled:

Lines 67-300+ handle tool processing with processTools function

Uses functions.Functions and functions.Tool types

Parses tool calls from model output using XML or JSON formats

Returns tool_calls in the response with proper structure

Anthropic API Reference

Tool Use Documentation: https://docs.anthropic.com/claude/docs/tool-use

Messages API Reference: https://docs.anthropic.com/claude/reference/messages_post

This pull request was created from Copilot chat.

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

netlify · 2026-01-10T11:51:45Z

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	`2b9ab03`
🔍 Latest deploy log	https://app.netlify.com/projects/localai/deploys/696277b3aa75c100089c675b
😎 Deploy Preview	https://deploy-preview-7956--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Co-authored-by: mudler <[email protected]>

mudler · 2026-01-10T15:56:04Z

tests/e2e/e2e_anthropic_test.go

+				Expect(message.Content).ToNot(BeEmpty())
+
+				// Check if there's a tool use in the response
+				hasToolUse := false


@copilot in this test we have to assume that we the model will have to call the tools or this should be recorded as a failure

Updated the tests to require tool usage. The first test now asserts Expect(hasToolUse).To(BeTrue()) with a clear error message, and the third test expects toolUseID to be non-empty. Both tests will now fail if the model doesn't call the expected tools. (29de7ec)

mudler · 2026-01-10T15:56:53Z

tests/e2e/e2e_anthropic_test.go

+				Expect(message.Content).ToNot(BeEmpty())
+			})
+
+			It("handles tool results in messages", func() {


@copilot let's add also a test for tool call with streaming responses

Added a comprehensive test for tool calling with streaming responses. The test verifies that:

Streaming events are properly emitted (content_block_start, content_block_delta, content_block_stop)

Tool use blocks are correctly accumulated from the stream

The model calls the tool and stop_reason is set to tool_use

(2b9ab03)

- First test now expects hasToolUse to be true with clear error message - Third test now expects toolUseID to be non-empty (removed conditional) - Both tests will now fail if model doesn't call the expected tools Co-authored-by: mudler <[email protected]>

- Tests that streaming events are properly emitted (content_block_start/delta/stop) - Verifies tool_use blocks are accumulated correctly in streaming mode - Ensures model calls tools and stop_reason is set to tool_use Co-authored-by: mudler <[email protected]>

Initial plan

b8ec83e

Copilot AI assigned Copilot and mudler Jan 10, 2026

Copilot started work on behalf of mudler January 10, 2026 11:56 View session

Copilot AI and others added 2 commits January 10, 2026 12:03

Add tool/function calling schema support to Anthropic Messages API

13581c3

Co-authored-by: mudler <[email protected]>

Add E2E tests for Anthropic tool calling

3bae4e3

Co-authored-by: mudler <[email protected]>

Copilot AI changed the title ~~[WIP] Add tool/function calling support to Anthropic Messages API~~ Add tool/function calling support to Anthropic Messages API Jan 10, 2026

Copilot AI requested a review from mudler January 10, 2026 12:10

Copilot finished work on behalf of mudler January 10, 2026 12:10

mudler changed the title ~~Add tool/function calling support to Anthropic Messages API~~ feat: add tool/function calling support to Anthropic Messages API Jan 10, 2026

mudler reviewed Jan 10, 2026

View reviewed changes

Copilot started work on behalf of mudler January 10, 2026 15:56 View session

mudler reviewed Jan 10, 2026

View reviewed changes

Copilot AI requested a review from mudler January 10, 2026 15:59

Copilot finished work on behalf of mudler January 10, 2026 15:59

Copilot started work on behalf of mudler January 10, 2026 15:59 View session

Copilot finished work on behalf of mudler January 10, 2026 16:01

mudler marked this pull request as ready for review January 10, 2026 17:37

mudler merged commit 5ca8f0a into master Jan 10, 2026
35 checks passed

mudler deleted the copilot/add-tool-calling-support-anthropic-messages branch January 10, 2026 17:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat: add tool/function calling support to Anthropic Messages API #7956

feat: add tool/function calling support to Anthropic Messages API #7956

Copilot AI commented Jan 10, 2026 •

edited

Loading

Uh oh!

netlify bot commented Jan 10, 2026 •

edited

Loading

Uh oh!

mudler Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

mudler Jan 10, 2026

Uh oh!

Copilot AI Jan 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

feat: add tool/function calling support to Anthropic Messages API #7956

feat: add tool/function calling support to Anthropic Messages API #7956

Conversation

Copilot AI commented Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary: Tool/Function Calling Support for Anthropic Messages API ✅

Completed Tasks

Test Coverage

Key Features

Implementation Quality

Summary

Background

Requirements

1. Schema Updates (core/schema/anthropic.go)

2. Endpoint Updates (core/http/endpoints/anthropic/messages.go)

3. Message Conversion Updates

4. Streaming Support

5. Tool Choice Handling

6. Tests

Reference Implementation

Anthropic API Reference

Uh oh!

netlify bot commented Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for localai ready!

Uh oh!

mudler Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

mudler Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Jan 10, 2026 •

edited

Loading

1. Schema Updates (`core/schema/anthropic.go`)

2. Endpoint Updates (`core/http/endpoints/anthropic/messages.go`)

netlify bot commented Jan 10, 2026 •

edited

Loading