stopAtToolNames does not return finalOutput

[mhashas]

export const createGetActionLogsAgent = (
  model: Model,
  adminApiToolkit: AdminApiToolkit,
) =>
  new Agent({
    name: 'get_action_logs',
    instructions: buildInstruction(),
    model: model,
    tools: [
      createGetActionLogsFiltersTool(),
      createQueryActionLogsTool(adminApiToolkit),
    ],
    toolUseBehavior: { stopAtToolNames: ['query_action_logs'] },
  });

const actionLogsAgent = createGetActionLogsAgent(model, adminApiToolkit);

const tools = [
  actionLogsAgent.asTool({
    toolName: 'get_action_logs',
    toolDescription: GET_ACTION_LOGS_TOOL_DESCRIPTION,
    // in theory should not be needed, but without it the content is '', might be a bug in the library
    customOutputExtractor: (output: RunResult<any, any>) => {
      return output.finalOutput;
    },
  }),

I would expect to not have to write this customOutputExtractor. queryActionLogs returns a very long string in case it matters.

I also have an extra question, is there an easy way to unit tests the tools?

  describe('createGetActionLogsFiltersTool', () => {
    const tool = createGetActionLogsFiltersTool();
    test('should have correct name and description', () => {
      expect(tool.name).toBe('get_action_logs_filters');
      expect(tool.description).toBe(
        'Generate valid filters for action logs query.',
      );
    });
  });
});

Basically how can I call the tool.execute method. I found a function (forgot the name) but requires passing contexts, etc.

[vrtnis]

Hey @mhashas – good catch.

The reason, as you probably have determined, you're seeing an empty result (unless you use customOutputExtractor) is that when you use stopAtToolNames, the sub-agent halts before emitting a final assistant message, and asTool() by default just returns that last message. So technically, it's not a bug per se, but something that could be enhanced.

There’s a draft PR open #249 which introduces an opt-in returnRunResult: true flag to agent.asTool(). With that flag set, you’ll get the full RunResult (including finalOutput, interruptions like approval requests, etc.), without needing to write a custom extractor.

So your example could be simplified to:

const tool = actionLogsAgent.asTool({
  toolName: 'get_action_logs',
  returnRunResult: true,
});

BTW you can also unit test the tool by calling .invoke() directly. It takes a RunContext (which can be empty unless you rely on extra context), and a json string as input:

const res = await tool.invoke(new RunContext(), '{"input": "your filters"}');
expect(res.finalOutput).toContain('expected log data');

Feel free to give the PR #249 a look and lmk if it fits your use case or if there are other improvements you’d like to see there.

[mhashas]

Hey, thanks for the answer, I commented in your MR.

[vrtnis]

Moving this #249 (comment) here.

Ok, so as I understand, there's a bit of incompatibility between the two options:

stopAtToolNames

• The final output will be the output of the first tool in the list that was called.

asTool

• If not provided, the last message from the agent will be used.

When we talk about the last message from the agent, we're talking about the last message from the history that comes from the assistant, not the last message from the history? Any reason why we want the last message to strictly be from the assistant?

Yes. asTool() intentionally looks for the last assistant_message because that’s the only history item guaranteed to be authored by the model and suitable for human reading (including any output validation/guardrails you configured). Tool calls/results and user/system messages are inputs or internal artifacts, not finished answers.

If the agent terminates on a tool call (e.g., via stopAtToolNames), there may be no trailing assistant message. In that case, either extract run.finalOutput with a customOutputExtractor, or (once available) opt in to the proposed returnRunResult mode from PR #249 to surface it directly.
e.g.,

const tool = actionLogsAgent.asTool({
  toolName: 'get_action_logs',
  // today:
  customOutputExtractor: (r) => r.finalOutput,
  // or, when merged:
  // returnRunResult: true,
});