Underspecified schema for SFT with tool-calling

[Kena-Njonge]

Describe the Bug

The tool-calling schema documentation for SFT in Microsoft Foundry (classic) is too underspecified relative to the actual validator.

A .jsonl training file can appear valid against the documented schema and examples, pass preview/preprocessing steps, and still fail validation:

contains invalid schema (20)

The error does not identify the offending line, field, or unsupported property, which makes it difficult to determine what part of the tool-calling payload is actually invalid.

Demo/Dataset Affected

• Other: SFT SCHEMA.md, specifically the tool-calling section

Steps to Reproduce

I FOUND THE ERROR: I was trying to train on both response and tool-calling but I didn't include the tool-call as an explicit step, but instead had the tool call and then the message, so I had two "assistant' messages following each other, that seems to have been the error. Nevertheless, I think there is a lot of room for improvement as I describe below,. Thank you.

1. Create an SFT .jsonl file using the documented tool-calling format.
2. Include an assistant tool-calling mssage such as:

{
  "messages": [
    {
      "role": "user",
      "content": "What is the current price of Microsoft stock?"
    },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "type": "function",
          "function": {
            "name": "get_current_stock_price",
            "arguments": "{\"symbol\": \"MSFT\"}",
            "output": ""
          }
        }
      ]
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_stock_price",
        "description": "Get the current stock price",
        "parameters": {
          "type": "object",
          "properties": {
            "symbol": {
              "type": "string",
              "description": "The stock symbol"
            }
          },
          "required": ["symbol"]
        }
      }
    }
  ]
}

3. Upload the file for fine-tuning in Microsoft Foundry (classic).
4. Observe that preview/preprocessing appears reasonable, but the upload ultimately fails validation (after 15 min).

Expected Behavior

One of the following should happen:

1. the file is accepted if this shape is supported, or
2. validation fails with a specific error that identifies the exact unsupported field and line number.

For example:

• messages[1].tool_calls[0].function.output is not allowed
• tools[0].function.parameters.required is missing
• assistant tool_calls messages must omit/include content

Actual Behavior

The upload fails with a generic schema error:

status : training file: Preprocessing Summary:
The provided data failed validation due to: contains invalid schema (20). Please visit our docs to learn how to resolve these issues, and try again.

This message is not actionable because it does not identify:

• which record failed
• which field failed
• whether undocumented restrictions apply to tool-calling messages

Why This Is a Documentation/Validation Problem

The current documentation does not make the validator behavior clear enough for tool-calling datasets.

In particular, it is unclear whether the following are allowed, required, or forbidden:

• content: null on assistant tool-calling messages
• omission of content on assistant tool-calling messages
• extra properties such as function.output
• whether tools must use type: "function" only or if they can be of another type e.g. openapi as is returned by the azure.ai.agents.models library
• whether parameters.required must always be present, even when empty

If these are intentionally restricted, the documentation should state that explicitly.
If they are unsupported, the validation error should report the exact offending field.

Environment

• Azure region: Sweden Central
• Model: gpt-4.1
• Experience: Microsoft Foundry (classic)

Additional information