[draft] Responses API proxy server

[Wauplin]

⚠️⚠️ Codebase moved to its own repository: https://github.com/huggingface/responses.js ⚠️⚠️

---

Very early draft. Goal is to run a server that implements OpenAI's Responses API on top of Inference Providers.

cc @Vaibhavs10 @julien-c @hanouticelina

No need to review for now. Mostly to start as a base + try things.

Quick Start

Start server

cd packages/responses-server
pnpm install
pnpm dev

Run examples

# Text input
pnpm run example text

# Multi-turn text input
pnpm run example multi_turn

# Text + image input
pnpm run example image

# Streaming
pnpm run example streaming

# Structured output
pnpm run example structured_output
pnpm run example structured_output_streaming

# Function calling
pnpm run example function
pnpm run example function_streaming

TODOs

This is a POC. Most features are currently not implemented. Check in schemas.ts all fields that exist in official API but are not supported yet.

Here is a list of the main features of the API in no particular order. We don't need all of them right now:

• use openai types for output [ResponsesAPI] return openai-defined types #1580
• streaming mode [ResponsesAPI] Implement streaming mode #1582
• structured output [Responses API] Structured output #1586
• function calling [Responses API] Function calling #1587
• [ResponsesAPI] Provider in model + clean input #1588
• background mode
• get, delete, cancel, list responses
• conversation state
• reasoning
• tools execution (web search, file search, image generation, MCP, code interpreter, etc.)

Example

import OpenAI from "openai";

const openai = new OpenAI({ baseURL: "http://localhost:3000/v1", apiKey: process.env.HF_TOKEN });

const response = await openai.responses.create({
	model: "Qwen/Qwen2.5-VL-7B-Instruct",
	instructions: "You are a helpful assistant.",
	input: "Tell me a three sentence bedtime story about a unicorn.",
});

console.log(response);
console.log(response.output_text);

pnpm run example text_single

> @huggingface/responses-server@0.1.0 example /home/wauplin/projects/huggingface.js/packages/responses-server
> node scripts/run-example.js text_single

{
  object: 'response',
  id: 'resp_ff152426c1c1ad544392e8ff512ef73dedc3a9cbab60e65a',
  status: 'completed',
  error: null,
  instructions: 'You are a helpful assistant.',
  model: 'Qwen/Qwen2.5-VL-7B-Instruct',
  temperature: 1,
  top_p: 1,
  created_at: 1751359624,
  output: [
    {
      id: 'msg_0d05372ec992169434028304842f1123b195e9bebdf1305f',
      type: 'message',
      role: 'assistant',
      status: 'completed',
      content: [Array]
    }
  ],
  output_text: "Once upon a time, in a faraway realm, lived a young unicorn named Starlight. Every night, Starlight would Sadly come down from a hill and heatpocumentt the Sleeping Forest. One Moonlight night, Starlight accidentally stumbled upon a secluded glade filled with silver stars and glowing pixies. Dazzled by the magical light, Starlight decided to let the pixies light Starlight. With the pixies' help, Starlight found a way to make the Severnvalsea never gonna gonna gonna stop glittering magnificently. As the last stars disappeared into the night, Starlight felt a sense of peace, knowing that the Gadondo's secret was now safe, forever encompassed in the magic of their icy depiction."
}

[Wauplin]

closing in favor of its own repository: https://github.com/huggingface/responses.js