From 2f9d0322a014bb8178d01874b51a4105efa60e8e Mon Sep 17 00:00:00 2001 From: Linda Li <139801625+lindazqli@users.noreply.github.com> Date: Tue, 23 Jun 2026 08:13:52 -0700 Subject: [PATCH 1/4] samples(python/hosted-agents): add Foundry IQ knowledge base agent (Responses) (#570) * samples(python/hosted-agents): add Foundry IQ knowledge base agent (Responses) Add an Agent Framework hosted-agent sample that answers questions from a Foundry IQ knowledge base (Azure AI Search agentic retrieval), reached through a Foundry Toolbox using the agent's managed identity (Agentic Identity) for keyless Entra ID auth. - provision_kb.py: creates the search index, knowledge source, and knowledge base (answer synthesis via a keyless Azure OpenAI model) and prints the KB MCP endpoint. - main.py: FoundryChatClient + MCPStreamableHTTPTool over the toolbox; injects a fresh bearer token and the Toolboxes=V1Preview feature header. - agent.manifest.yaml: RemoteTool connection (AgenticIdentity) + toolbox exposing the knowledge_base_retrieve MCP tool. - README with azd and VS Code (Foundry Toolkit) quick starts and RBAC table. Validated end-to-end on a Foundry project: deploy -> grant Search Index Data Reader -> grounded, cited answers from the knowledge base. * samples(python/foundry-iq): use TOOLBOX_ENDPOINT env var instead of TOOLBOX_NAME Align with the sibling 04-foundry-toolbox sample: the agent reads the full toolbox MCP endpoint from TOOLBOX_ENDPOINT, falling back to building it from FOUNDRY_PROJECT_ENDPOINT + TOOLBOX_NAME. Update main.py, agent.manifest.yaml, agent.yaml, .env.example, and the README accordingly. * samples(python/foundry-iq): create connection + toolbox via azd ai CLI Match the sibling 04-foundry-toolbox pattern instead of declaring the connection/toolbox as manifest resources (the manifest's AgenticIdentity connection block hits the azd Bicep authType mapping). The manifest now only declares the model resource; a new toolbox.yaml defines the KB MCP tool, and the README walks through 'azd ai connection create' + 'azd ai toolbox create'. * samples(python/foundry-iq): drop dead TOOLBOX_NAME fallback TOOLBOX_ENDPOINT is now the single source for the toolbox MCP endpoint; the sample never sets TOOLBOX_NAME, and the old URL-deriving fallback produced the version number as the tool name for versioned endpoints. Require TOOLBOX_ENDPOINT and use a stable 'knowledge_base' tool name. * samples(python/foundry-iq): one-command provisioning via azd postprovision hook Add hooks/postprovision.{sh,ps1} that run provision_kb.py, create the knowledge-base-mcp connection and knowledge-base toolbox, and set TOOLBOX_ENDPOINT, so a single azd provision does all the manual steps. provision_kb.py now stores the KB MCP endpoint as KB_MCP_ENDPOINT in the azd env (best-effort). README documents wiring the hook into azure.yaml, with the manual flow kept as a fallback. * samples(python/foundry-iq): add server_url to toolbox MCP tool The MCP tool needs server_url pointing at the knowledge base's MCP endpoint alongside project_connection_id (which supplies Agentic Identity auth), matching the connection+server_url pattern used by the other toolbox samples. toolbox.yaml uses a \ placeholder that the postprovision hook substitutes before azd ai toolbox create. * Fix Foundry IQ KB postprovision hooks: idempotent, self-locating, .yaml temp file * Exclude foundry-iq-knowledge-base from cloud e2e (requires external provisioning) The sample's postprovision hook provisions an Azure AI Search KB and creates the knowledge-base toolbox/TOOLBOX_ENDPOINT. The generic Cloud E2E harness never runs the hook, leaving TOOLBOX_ENDPOINT empty so main.py raises ValueError on startup and every invoke returns HTTP 500. Add .ci-skip per the documented requires-external-credentials opt-out. * Rename foundry-iq sample to 17-foundry-iq-toolbox so cloud e2e injects a real toolbox endpoint The Cloud E2E harness only injects a pre-provisioned TOOLBOX_ENDPOINT (from TOOLBOX_ENDPOINT_NCUS) into samples it classifies as toolbox samples, which it detects by the substring 'toolbox' in the path. The previous name left TOOLBOX_ENDPOINT empty so main.py raised ValueError and every invoke returned HTTP 500. Renaming to include 'toolbox' opts the sample into the toolbox-endpoint matrix expansion (same path as the passing 04-foundry-toolbox sibling); drop the .ci-skip and fix the README init path. * Update README azd init path to 17-foundry-iq-toolbox The rename commit moved README.md but a failed multi-pathspec git add dropped the in-file path edit, so line 66 still referenced the old 17-foundry-iq-knowledge-base directory. Point it at 17-foundry-iq-toolbox. * Fall back to FOUNDRY_PROJECT_ENDPOINT/TOOLBOX_NAME when TOOLBOX_ENDPOINT is unset * Use narrow agent-framework-foundry subpackages; keep strict TOOLBOX_ENDPOINT (e2e provides it) * Cloud E2E: pin 17-foundry-iq-toolbox to the foundry-iq-kb knowledge toolbox * ci(hosted-agents): scope foundry-iq-kb toolbox to sample 17 only --- .../17-foundry-iq-toolbox/.azdignore | 4 + .../17-foundry-iq-toolbox/.dockerignore | 8 + .../17-foundry-iq-toolbox/.env.example | 9 + .../17-foundry-iq-toolbox/Dockerfile | 16 ++ .../responses/17-foundry-iq-toolbox/README.md | 197 ++++++++++++++ .../17-foundry-iq-toolbox/agent.manifest.yaml | 42 +++ .../17-foundry-iq-toolbox/agent.yaml | 14 + .../hooks/postprovision.ps1 | 93 +++++++ .../hooks/postprovision.sh | 75 ++++++ .../responses/17-foundry-iq-toolbox/main.py | 89 +++++++ .../17-foundry-iq-toolbox/provision_kb.py | 249 ++++++++++++++++++ .../17-foundry-iq-toolbox/requirements.txt | 16 ++ .../17-foundry-iq-toolbox/toolbox.yaml | 30 +++ 13 files changed, 842 insertions(+) create mode 100644 samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/.azdignore create mode 100644 samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/.dockerignore create mode 100644 samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/.env.example create mode 100644 samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/Dockerfile create mode 100644 samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/README.md create mode 100644 samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/agent.manifest.yaml create mode 100644 samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/agent.yaml create mode 100644 samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/hooks/postprovision.ps1 create mode 100644 samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/hooks/postprovision.sh create mode 100644 samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/main.py create mode 100644 samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/provision_kb.py create mode 100644 samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/requirements.txt create mode 100644 samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/toolbox.yaml diff --git a/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/.azdignore b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/.azdignore new file mode 100644 index 000000000..301a106fc --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/.azdignore @@ -0,0 +1,4 @@ +agent.manifest.yaml +agent.yaml +.env.example +hooks/ diff --git a/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/.dockerignore b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/.dockerignore new file mode 100644 index 000000000..39a016fe8 --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/.dockerignore @@ -0,0 +1,8 @@ +.venv +__pycache__ +*.pyc +*.pyo +*.pyd +.Python +.env +hooks/ diff --git a/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/.env.example b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/.env.example new file mode 100644 index 000000000..1bea5f434 --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/.env.example @@ -0,0 +1,9 @@ +FOUNDRY_PROJECT_ENDPOINT="..." +AZURE_AI_MODEL_DEPLOYMENT_NAME="..." +# Full MCP endpoint of the toolbox the agent consumes, e.g. +# https://.services.ai.azure.com/api/projects//toolboxes/agent-tools/mcp?api-version=v1 +TOOLBOX_ENDPOINT="..." + +# Only needed when running provision_kb.py to create the knowledge base: +AZURE_SEARCH_ENDPOINT="https://.search.windows.net" +AZURE_OPENAI_ENDPOINT="https://.openai.azure.com" diff --git a/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/Dockerfile b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/Dockerfile new file mode 100644 index 000000000..0cc939d9b --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/Dockerfile @@ -0,0 +1,16 @@ +FROM python:3.12-slim + +WORKDIR /app + +COPY . user_agent/ +WORKDIR /app/user_agent + +RUN if [ -f requirements.txt ]; then \ + pip install -r requirements.txt; \ + else \ + echo "No requirements.txt found"; \ + fi + +EXPOSE 8088 + +CMD ["python", "main.py"] diff --git a/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/README.md b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/README.md new file mode 100644 index 000000000..0fa36d06a --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/README.md @@ -0,0 +1,197 @@ +# Foundry IQ Knowledge Base Agent (Responses Protocol) + +An [Agent Framework](https://github.com/microsoft/agent-framework) agent that answers questions from a **[Foundry IQ](https://learn.microsoft.com/azure/search/agentic-retrieval-concept) knowledge base** (Azure AI Search agentic retrieval), hosted on Microsoft Foundry using the **Responses protocol**. The agent reaches the knowledge base through a **Foundry Toolbox** that proxies the knowledge base's MCP endpoint. + +## How it works + +```mermaid +flowchart LR + U[User] --> A["Hosted agent
(Agent Framework)"] + A -->|MCP over toolbox| T["Foundry Toolbox
knowledge-base-mcp connection"] + T -->|Agent identity
(Entra ID)| KB["Knowledge base MCP
knowledge_base_retrieve"] + KB --> KS[Knowledge source] --> IDX[(Azure AI Search index)] + KB -.answer synthesis.-> M[[Azure OpenAI model]] +``` + +1. **Knowledge base (data plane).** [`provision_kb.py`](provision_kb.py) creates an Azure AI Search index, seeds it with the "Earth at night" documents, and builds a **knowledge source** and a **knowledge base**. The knowledge base synthesizes answers with an Azure OpenAI model and exposes an MCP endpoint (`{search}/knowledgebases/{kb}/mcp`) whose only tool is `knowledge_base_retrieve`. +2. **Toolbox connection.** A `RemoteTool` **connection** (`knowledge-base-mcp`) authenticates to the knowledge base's MCP endpoint with **Agentic Identity** — the agent's managed identity, keyless. A **toolbox** (defined in [`toolbox.yaml`](toolbox.yaml)) exposes that endpoint as an MCP tool: its `server_url` points at the knowledge base's MCP endpoint and `project_connection_id` supplies the connection's auth. Both are created for you by the `azd provision` `postprovision` hook (see [Provision and run the agent](#provision-and-run-the-agent)). +3. **Agent.** [`main.py`](main.py) uses `FoundryChatClient` and connects to the toolbox's MCP endpoint with `MCPStreamableHTTPTool`. The agent discovers `knowledge_base_retrieve` at runtime and grounds its answers in the retrieved sources. + +### Model Integration + +The agent uses `FoundryChatClient` from the Agent Framework to create an OpenAI-compatible Responses client, and connects to the toolbox over MCP via `MCPStreamableHTTPTool`. It reads the toolbox's MCP endpoint from the `TOOLBOX_ENDPOINT` environment variable. See [main.py](main.py) for the full implementation. + +## Prerequisites + +- An Azure AI Foundry project with a deployed chat model (e.g. `gpt-4.1-mini`). +- An **Azure AI Search** service ([create one](https://learn.microsoft.com/azure/search/search-create-service-portal)) with a **system-assigned managed identity** and **RBAC** enabled (Portal → search service → **Keys** → **API Access control** → "Both" or "Role-based access control"). +- Azure CLI logged in (`az login`). + +### Required RBAC + +| Identity | Role | Scope | Why | +| --- | --- | --- | --- | +| Search service managed identity | **Cognitive Services User** | Foundry account | Knowledge base calls the Azure OpenAI model for answer synthesis (keyless) | +| You (provisioning) | **Search Service Contributor** | Search service | Create the index, knowledge source, and knowledge base | +| You (provisioning) | **Search Index Data Contributor** | Search service | Upload the seed documents | +| The agent's managed identity | **Search Index Data Reader** | Search service | The deployed agent retrieves from the knowledge base at query time | + +> The agent's managed identity is created when you deploy the agent. Grant it **Search Index Data Reader** on the search service after the first `azd deploy` (see [Grant the agent access](#grant-the-agent-access-to-the-knowledge-base)). + +## Provision and run the agent + +### Option 1: Azure Developer CLI (`azd`) + +With the bundled `postprovision` hook, a single `azd provision` builds the knowledge base, creates the toolbox connection and toolbox, and sets `TOOLBOX_ENDPOINT` for you. + +#### 1. Install prerequisites + +1. **Azure Developer CLI (`azd`)** — [Install azd](https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/install-azd) (1.25 or later) +2. Install the unified Foundry CLI extension bundle: + ```bash + azd ext install microsoft.foundry + ``` +3. Authenticate: + ```bash + azd auth login + ``` + +#### 2. Initialize the agent project + +No cloning required. Create a new folder and initialize from the manifest: + +```bash +mkdir my-foundry-iq-agent && cd my-foundry-iq-agent + +azd ai agent init -m https://github.com/microsoft-foundry/foundry-samples/blob/main/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/agent.manifest.yaml +``` + +Follow the prompts to configure your Foundry project and model deployment. If you don't have an existing Foundry project, `azd ai agent init` will guide you through creating one. Initializing also sets the selected project as the active project, and copies this sample's files into a new service directory `src//` — including [`provision_kb.py`](provision_kb.py), [`toolbox.yaml`](toolbox.yaml), and the [`hooks/`](hooks/) scripts. + +#### 3. Enable one-command provisioning (`postprovision` hook) + +Wire the bundled hook into the `azure.yaml` that `azd ai agent init` generated, so the knowledge base, connection, and toolbox are created automatically every time you run `azd provision`. `postprovision` must be registered at the **top level** of `azure.yaml` (service-scoped hooks only support the package/deploy lifecycle), and the `run:` path must point at the hook inside the generated service directory. Add this top-level block, replacing `` with the service folder `azd ai agent init` created under `src/`: + +```yaml +hooks: + postprovision: + posix: + shell: sh + run: ./src//hooks/postprovision.sh + windows: + shell: pwsh + run: ./src//hooks/postprovision.ps1 +``` + +The hook ([`hooks/postprovision.sh`](hooks/postprovision.sh) / [`hooks/postprovision.ps1`](hooks/postprovision.ps1)) runs everything the [manual steps](#provision-manually-without-the-hook) below would, in one shot. It locates its own directory, so it works no matter where `azd` runs it from. + +#### 4. Provision + +Point the hook at your existing Azure AI Search service, then provision: + +```bash +azd env set AZURE_SEARCH_ENDPOINT "https://.search.windows.net" +azd provision +``` + +`azd provision` creates (or reuses) your Foundry project and model deployment, then the `postprovision` hook: + +1. Runs [`provision_kb.py`](provision_kb.py) to build the search index, knowledge source, and knowledge base, and stores the KB's MCP endpoint as `KB_MCP_ENDPOINT`. +2. Creates the `knowledge-base-mcp` `RemoteTool` connection (Agentic Identity, keyless) targeting that endpoint. +3. Creates the `knowledge-base` toolbox from [`toolbox.yaml`](toolbox.yaml). +4. Sets `TOOLBOX_ENDPOINT` so the agent connects to the toolbox. + +> The hook derives `AZURE_OPENAI_ENDPOINT` from your Foundry project endpoint for answer synthesis. To use a different Azure OpenAI resource, set it explicitly first: `azd env set AZURE_OPENAI_ENDPOINT "https://.openai.azure.com"`. + +#### 5. Run the agent locally + +```bash +azd ai agent run +``` + +The agent host will start on `http://localhost:8088`. + +#### Invoke the local agent + +In a separate terminal, from the project directory: + +```bash +azd ai agent invoke --local "What can you tell me about the Earth at night?" +``` + +#### Deploy to Foundry + +```bash +azd deploy +``` + +For the full deployment guide, see [Deploy a hosted agent](https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/deploy-hosted-agent). + +#### Grant the agent access to the knowledge base + +After the first deploy, grant the agent's managed identity **Search Index Data Reader** on the search service so it can retrieve at query time: + +```powershell +$searchId = az search service show -n -g --query id -o tsv +# Find the agent identity object id in the Foundry portal (Agents → your agent → Identity), +# or via the deployment output, then: +az role assignment create --assignee-object-id --assignee-principal-type ServicePrincipal ` + --role "Search Index Data Reader" --scope $searchId +``` + +#### Invoke the deployed agent + +```bash +azd ai agent invoke "What can you tell me about the Earth at night?" +``` + +#### Provision manually (without the hook) + +Prefer to run the steps yourself (or skip the hook)? Provision the knowledge base directly from the project directory, with `az login` done: + +```bash +pip install requests azure-identity python-dotenv + +export AZURE_SEARCH_ENDPOINT="https://.search.windows.net" +export AZURE_OPENAI_ENDPOINT="https://.openai.azure.com" +export AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4.1-mini" +python provision_kb.py +``` + +In PowerShell, use `$env:NAME="value"` instead of `export`. The script prints the knowledge base's **MCP endpoint** and is safe to re-run. Then create the connection, point the toolbox's `server_url` at that endpoint, create the toolbox, and store the toolbox endpoint: + +```bash +azd ai connection create knowledge-base-mcp --kind remote-tool \ + --target "" \ + --auth-type agentic-identity --audience https://search.azure.com --metadata "ApiType=Azure" + +# toolbox.yaml uses ${KB_MCP_ENDPOINT} for the tool's server_url — replace it with +# the endpoint provision_kb.py printed (or export KB_MCP_ENDPOINT and envsubst it). +azd ai toolbox create knowledge-base --from-file ./toolbox.yaml + +azd env set TOOLBOX_ENDPOINT "https://.services.ai.azure.com/api/projects//toolboxes/knowledge-base/mcp?api-version=v1" +``` + +### Option 2: VS Code (Foundry Toolkit) + +> The VS Code flow doesn't run the `azd` hook — provision the knowledge base, connection, and toolbox first with [Provision manually](#provision-manually-without-the-hook). + +1. Open the Command Palette (`Ctrl+Shift+P`) and run **Foundry Toolkit: Create Hosted Agent**, then select this sample. +2. Press **F5** to run and debug the agent locally on `http://localhost:8088`. +3. Run **Foundry Toolkit: Open Agent Inspector** to chat with the running agent. +4. Run **Foundry Toolkit: Deploy Hosted Agent** to build the image, register the version, and assign RBAC. + +## Try it + +```bash +azd ai agent invoke --local "What can you tell me about the Earth at night?" +azd ai agent invoke --local "Why do some lights appear over the open ocean?" +azd ai agent invoke --local "How is nighttime imagery used to study light pollution?" +``` + +## Next steps + +- [Agentic retrieval in Azure AI Search](https://learn.microsoft.com/azure/search/agentic-retrieval-concept) — knowledge bases and knowledge sources +- [Foundry Toolbox](https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/tools/toolbox) — managed tool registry +- [Add a Foundry Toolbox](../04-foundry-toolbox/) — the general toolbox sample this one builds on +- [Azure AI Search RAG](../11-azure-search-rag/) — classic RAG with a context provider instead of a knowledge base diff --git a/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/agent.manifest.yaml b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/agent.manifest.yaml new file mode 100644 index 000000000..9bd1cf878 --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/agent.manifest.yaml @@ -0,0 +1,42 @@ +# yaml-language-server: $schema=https://raw.githubusercontent.com/microsoft/AgentSchema/refs/heads/main/schemas/v1.0/AgentManifest.yaml +name: agent-framework-foundry-iq-knowledge-base-responses +displayName: "Foundry IQ Knowledge Base Agent" +description: > + An Agent Framework agent that answers questions from a Foundry IQ knowledge + base (Azure AI Search agentic retrieval), reached through a Foundry toolbox. + The toolbox connects to the knowledge base's MCP endpoint using the agent's + managed identity (Agentic Identity) for keyless Entra ID authentication. +metadata: + tags: + - Agent Framework + - AI Agent Hosting + - Azure AI AgentServer + - Responses Protocol + - Foundry IQ + - Knowledge Base + - Agentic Retrieval + - Azure AI Search +template: + name: agent-framework-foundry-iq-knowledge-base-responses + kind: hosted + protocols: + - protocol: responses + version: 1.0.0 + environment_variables: + - name: AZURE_AI_MODEL_DEPLOYMENT_NAME + value: "{{AZURE_AI_MODEL_DEPLOYMENT_NAME}}" + # Full MCP endpoint of the toolbox the agent consumes. The postprovision hook + # creates the connection and toolbox from the bundled toolbox.yaml and sets the + # TOOLBOX_ENDPOINT azd environment variable; ${TOOLBOX_ENDPOINT} resolves from it + # at deploy time. To wire it up by hand instead, run: + # azd ai connection create knowledge-base-mcp --kind remote-tool \ + # --target "" --auth-type agentic-identity \ + # --audience https://search.azure.com --metadata "ApiType=Azure" + # azd ai toolbox create knowledge-base --from-file ./toolbox.yaml + # azd env set TOOLBOX_ENDPOINT "" + - name: TOOLBOX_ENDPOINT + value: "${TOOLBOX_ENDPOINT}" +resources: + - kind: model + id: gpt-4.1-mini + name: AZURE_AI_MODEL_DEPLOYMENT_NAME diff --git a/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/agent.yaml b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/agent.yaml new file mode 100644 index 000000000..a1f447ea2 --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/agent.yaml @@ -0,0 +1,14 @@ +# yaml-language-server: $schema=https://raw.githubusercontent.com/microsoft/AgentSchema/refs/heads/main/schemas/v1.0/ContainerAgent.yaml +kind: hosted +name: agent-framework-foundry-iq-knowledge-base-responses +protocols: + - protocol: responses + version: 1.0.0 +resources: + cpu: "0.25" + memory: "0.5Gi" +environment_variables: + - name: AZURE_AI_MODEL_DEPLOYMENT_NAME + value: ${AZURE_AI_MODEL_DEPLOYMENT_NAME} + - name: TOOLBOX_ENDPOINT + value: ${TOOLBOX_ENDPOINT} diff --git a/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/hooks/postprovision.ps1 b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/hooks/postprovision.ps1 new file mode 100644 index 000000000..6cc5b7a19 --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/hooks/postprovision.ps1 @@ -0,0 +1,93 @@ +#!/usr/bin/env pwsh +# azd postprovision hook (Windows / pwsh). +# +# Runs automatically after `azd provision`. It provisions the Foundry IQ +# knowledge base, creates the toolbox connection and toolbox, and stores +# TOOLBOX_ENDPOINT so the agent can reach the toolbox — collapsing every manual +# provisioning step into a single `azd provision`. +# +# Wire it up by adding this to the azure.yaml generated by `azd ai agent init`: +# +# hooks: +# postprovision: +# posix: +# shell: sh +# run: ./src//hooks/postprovision.sh +# windows: +# shell: pwsh +# run: ./src//hooks/postprovision.ps1 + +$ErrorActionPreference = "Stop" + +# PowerShell does not stop on a non-zero exit code from a native command (like +# azd), so check $LASTEXITCODE after each azd call and fail loudly. +function Invoke-Checked { + param([scriptblock] $Script, [string] $What) + & $Script + if ($LASTEXITCODE -ne 0) { throw "$What failed (exit $LASTEXITCODE)." } +} + +# Run from the sample directory (the parent of hooks/) so provision_kb.py and +# toolbox.yaml resolve no matter which directory azd invokes the hook from. +Set-Location (Split-Path -Parent $PSScriptRoot) + +# AZURE_SEARCH_ENDPOINT is a prerequisite (your existing Azure AI Search service). +if (-not $env:AZURE_SEARCH_ENDPOINT) { + throw "AZURE_SEARCH_ENDPOINT is not set. Run: azd env set AZURE_SEARCH_ENDPOINT ""https://.search.windows.net""" +} + +# Derive the Azure OpenAI endpoint from the Foundry project endpoint when unset. +if (-not $env:AZURE_OPENAI_ENDPOINT -and $env:FOUNDRY_PROJECT_ENDPOINT) { + $env:AZURE_OPENAI_ENDPOINT = ($env:FOUNDRY_PROJECT_ENDPOINT -replace '/api/projects/.*$', '') +} + +Write-Host "Provisioning the Foundry IQ knowledge base..." +Invoke-Checked { python -m pip install -q requests azure-identity python-dotenv } "pip install" +Invoke-Checked { python provision_kb.py } "provision_kb.py" # stores the KB MCP endpoint as KB_MCP_ENDPOINT + +$kb = azd env get-value KB_MCP_ENDPOINT +if (-not $kb) { throw "provision_kb.py did not set KB_MCP_ENDPOINT." } + +Write-Host "Creating the knowledge-base-mcp connection..." +# --force upserts so the hook is safe to re-run on every azd provision. +Invoke-Checked { + azd ai connection create knowledge-base-mcp --kind remote-tool --force ` + --target $kb --auth-type agentic-identity ` + --audience https://search.azure.com --metadata "ApiType=Azure" +} "connection create" + +Write-Host "Creating the knowledge-base toolbox..." +# Toolbox versions are immutable and 'create' has no upsert flag, so skip it if +# the toolbox already exists (e.g. on a repeat azd provision). 'toolbox show' +# exits 0 when the toolbox exists and non-zero when it does not. azd writes +# diagnostics to stderr, which PowerShell turns into a terminating error under +# "Stop", so run the probe with errors non-terminating and decide on the exit code. +$prevEap = $ErrorActionPreference +$ErrorActionPreference = "Continue" +azd ai toolbox show knowledge-base *> $null +$toolboxExists = ($LASTEXITCODE -eq 0) +$ErrorActionPreference = $prevEap +if ($toolboxExists) { + Write-Host "Toolbox knowledge-base already exists; skipping create." +} +else { + # toolbox.yaml references ${KB_MCP_ENDPOINT} for the tool's server_url; resolve + # it into a temp file so the toolbox points at this deployment's KB endpoint. + # The temp file must keep a .yaml extension — azd ai toolbox create rejects others. + $resolved = (Get-Content ./toolbox.yaml -Raw).Replace('${KB_MCP_ENDPOINT}', $kb) + $tmp = Join-Path ([System.IO.Path]::GetTempPath()) "toolbox-$([System.Guid]::NewGuid()).yaml" + Set-Content -Path $tmp -Value $resolved -NoNewline + try { + Invoke-Checked { azd ai toolbox create knowledge-base --from-file $tmp } "toolbox create" + } + finally { + Remove-Item $tmp -ErrorAction SilentlyContinue + } +} + +# The toolbox's unversioned MCP alias is deterministic from the project endpoint. +$proj = (azd env get-value FOUNDRY_PROJECT_ENDPOINT).TrimEnd('/') +$toolbox = "$proj/toolboxes/knowledge-base/mcp?api-version=v1" +Invoke-Checked { azd env set TOOLBOX_ENDPOINT $toolbox } "env set TOOLBOX_ENDPOINT" + +Write-Host "Done. TOOLBOX_ENDPOINT = $toolbox" diff --git a/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/hooks/postprovision.sh b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/hooks/postprovision.sh new file mode 100644 index 000000000..99e1085ff --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/hooks/postprovision.sh @@ -0,0 +1,75 @@ +#!/usr/bin/env sh +# azd postprovision hook (POSIX / sh). +# +# Runs automatically after `azd provision`. It provisions the Foundry IQ +# knowledge base, creates the toolbox connection and toolbox, and stores +# TOOLBOX_ENDPOINT so the agent can reach the toolbox — collapsing every manual +# provisioning step into a single `azd provision`. +# +# Wire it up by adding this to the azure.yaml generated by `azd ai agent init`: +# +# hooks: +# postprovision: +# posix: +# shell: sh +# run: ./src//hooks/postprovision.sh +# windows: +# shell: pwsh +# run: ./src//hooks/postprovision.ps1 + +set -e + +# Run from the sample directory (the parent of hooks/) so provision_kb.py and +# toolbox.yaml resolve no matter which directory azd invokes the hook from. +cd "$(CDPATH= cd "$(dirname "$0")/.." && pwd)" + +# AZURE_SEARCH_ENDPOINT is a prerequisite (your existing Azure AI Search service). +if [ -z "$AZURE_SEARCH_ENDPOINT" ]; then + echo "AZURE_SEARCH_ENDPOINT is not set. Run: azd env set AZURE_SEARCH_ENDPOINT \"https://.search.windows.net\"" >&2 + exit 1 +fi + +# Derive the Azure OpenAI endpoint from the Foundry project endpoint when unset. +if [ -z "$AZURE_OPENAI_ENDPOINT" ] && [ -n "$FOUNDRY_PROJECT_ENDPOINT" ]; then + export AZURE_OPENAI_ENDPOINT="$(printf '%s' "$FOUNDRY_PROJECT_ENDPOINT" | sed -E 's#/api/projects/.*$##')" +fi + +echo "Provisioning the Foundry IQ knowledge base..." +python -m pip install -q requests azure-identity python-dotenv +python provision_kb.py # stores the KB MCP endpoint as KB_MCP_ENDPOINT + +KB="$(azd env get-value KB_MCP_ENDPOINT)" +if [ -z "$KB" ]; then + echo "provision_kb.py did not set KB_MCP_ENDPOINT." >&2 + exit 1 +fi + +echo "Creating the knowledge-base-mcp connection..." +# --force upserts so the hook is safe to re-run on every azd provision. +azd ai connection create knowledge-base-mcp --kind remote-tool --force \ + --target "$KB" --auth-type agentic-identity \ + --audience https://search.azure.com --metadata "ApiType=Azure" + +echo "Creating the knowledge-base toolbox..." +# Toolbox versions are immutable and 'create' has no upsert flag, so skip it if +# the toolbox already exists (e.g. on a repeat azd provision). 'toolbox show' +# exits 0 when the toolbox exists and non-zero when it does not. +if azd ai toolbox show knowledge-base >/dev/null 2>&1; then + echo "Toolbox knowledge-base already exists; skipping create." +else + # toolbox.yaml references ${KB_MCP_ENDPOINT} for the tool's server_url; resolve + # it into a temp file so the toolbox points at this deployment's KB endpoint. + # The temp file must keep a .yaml extension — azd ai toolbox create rejects others. + TMPDIR_KB="$(mktemp -d)" + trap 'rm -rf "$TMPDIR_KB"' EXIT + TMP="$TMPDIR_KB/toolbox.yaml" + KB="$KB" awk '{ gsub(/\$\{KB_MCP_ENDPOINT\}/, ENVIRON["KB"]); print }' ./toolbox.yaml > "$TMP" + azd ai toolbox create knowledge-base --from-file "$TMP" +fi + +# The toolbox's unversioned MCP alias is deterministic from the project endpoint. +PROJ="$(azd env get-value FOUNDRY_PROJECT_ENDPOINT | sed 's#/*$##')" +TOOLBOX="$PROJ/toolboxes/knowledge-base/mcp?api-version=v1" +azd env set TOOLBOX_ENDPOINT "$TOOLBOX" + +echo "Done. TOOLBOX_ENDPOINT = $TOOLBOX" diff --git a/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/main.py b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/main.py new file mode 100644 index 000000000..38ef0ed00 --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/main.py @@ -0,0 +1,89 @@ +# Copyright (c) Microsoft. All rights reserved. + +import asyncio +import os +from collections.abc import Callable + +import httpx +from agent_framework import Agent, MCPStreamableHTTPTool +from agent_framework.foundry import FoundryChatClient +from agent_framework_foundry_hosting import ResponsesHostServer +from azure.identity import DefaultAzureCredential, get_bearer_token_provider +from dotenv import load_dotenv + +# Load environment variables from .env file +load_dotenv() + + +def resolve_toolbox_endpoint() -> str: + """Resolve the toolbox MCP endpoint URL from the ``TOOLBOX_ENDPOINT`` env var. + + Set it to the versioned endpoint printed by ``azd ai toolbox create`` (see + README.md / toolbox.yaml). + """ + endpoint = os.environ.get("TOOLBOX_ENDPOINT") + if not endpoint: + raise ValueError("TOOLBOX_ENDPOINT is not set") + return endpoint + + +class ToolboxAuth(httpx.Auth): + """Injects a fresh bearer token on every request.""" + + def __init__(self, token_provider: Callable[[], str]): + self._get_token = token_provider + + def auth_flow(self, request: httpx.Request): + request.headers["Authorization"] = f"Bearer {self._get_token()}" + yield request + + +async def main(): + credential = DefaultAzureCredential() + + # Token for the Foundry toolbox MCP endpoint. The toolbox proxies the call to + # the Azure AI Search knowledge base using the agent's managed identity, which + # is configured on the `knowledge-base-mcp` connection in agent.manifest.yaml. + token_provider = get_bearer_token_provider(credential, "https://ai.azure.com/.default") + + toolbox_endpoint = resolve_toolbox_endpoint() + + async with httpx.AsyncClient( + auth=ToolboxAuth(token_provider), + headers={"Foundry-Features": "Toolboxes=V1Preview"}, + timeout=120.0, + ) as http_client: + toolbox = MCPStreamableHTTPTool( + name="knowledge_base", + url=toolbox_endpoint, + http_client=http_client, + load_prompts=False, + ) + + client = FoundryChatClient( + project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], + model=os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"], + credential=credential, + ) + + agent = Agent( + client=client, + instructions=( + "You are a helpful assistant. Use the knowledge base tool to answer " + "user questions. If the knowledge base doesn't contain the answer, " + "respond with 'I don't know'. When you use information from the " + "knowledge base, include citations to the retrieved sources." + ), + tools=toolbox, + # History will be managed by the hosting infrastructure, thus there + # is no need to store history by the service. Learn more at: + # https://developers.openai.com/api/reference/resources/responses/methods/create + default_options={"store": False}, + ) + + server = ResponsesHostServer(agent) + await server.run_async() + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/provision_kb.py b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/provision_kb.py new file mode 100644 index 000000000..8afb5094b --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/provision_kb.py @@ -0,0 +1,249 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Provision the Foundry IQ knowledge base used by this sample. + +Runs once, before you deploy the agent. It creates (or updates) four things in +your Azure AI Search service: + + 1. A search index (``foundry-iq-index``) with a semantic configuration. + 2. Seed documents (the "Earth at night" corpus) uploaded into that index. + 3. A knowledge source over the index. + 4. A knowledge base that orchestrates the knowledge source and synthesizes + answers with an Azure OpenAI model. + +The knowledge base exposes an MCP endpoint +(``{search}/knowledgebases/{kb}/mcp``) with a single ``knowledge_base_retrieve`` +tool. The hosted agent reaches that endpoint through a Foundry toolbox. + +Usage (from this directory, with the venv activated and ``az login`` done): + + python provision_kb.py + +Required env vars (also read from a local ``.env`` file if present): + + AZURE_SEARCH_ENDPOINT e.g. https://.search.windows.net + AZURE_OPENAI_ENDPOINT e.g. https://.openai.azure.com + AZURE_AI_MODEL_DEPLOYMENT_NAME e.g. gpt-4.1-mini + +Optional env vars (sensible defaults shown): + + AZURE_SEARCH_INDEX_NAME foundry-iq-index + KNOWLEDGE_SOURCE_NAME foundry-iq-ks + KNOWLEDGE_BASE_NAME foundry-iq-kb + AZURE_AI_MODEL_NAME (defaults to AZURE_AI_MODEL_DEPLOYMENT_NAME) + +Your identity needs ``Search Service Contributor`` (to create the index and the +knowledge source/base) and ``Search Index Data Contributor`` (to upload +documents) on the search service. +""" + +import json +import os +import shutil +import subprocess +import sys + +import requests +from azure.identity import DefaultAzureCredential +from dotenv import load_dotenv + +SEARCH_SCOPE = "https://search.azure.com/.default" +API_VERSION = "2026-05-01-preview" +SEMANTIC_CONFIG_NAME = "default-semantic-config" + +DOCUMENTS: list[dict[str, str]] = [ + { + "id": "1", + "title": "City lights from space", + "content": ( + "At night, Earth's most populated regions glow with artificial light. " + "The brightest clusters trace major cities and coastlines, while large " + "dark areas correspond to oceans, deserts, and sparsely populated " + "regions. Satellite sensors such as the VIIRS Day/Night Band capture " + "these patterns with remarkable detail." + ), + }, + { + "id": "2", + "title": "Aurora and natural light", + "content": ( + "Not all light seen at night is artificial. Auroras near the poles, " + "lightning in storm systems, and moonlight reflecting off clouds all " + "contribute to the planet's nighttime glow. Wildfires and gas flares " + "also appear as bright points in nighttime imagery." + ), + }, + { + "id": "3", + "title": "Measuring light pollution", + "content": ( + "Nighttime satellite imagery is used to study light pollution, urban " + "growth, and energy use. Researchers compare brightness over time to " + "track economic activity, the spread of electrification, and the impact " + "of power outages following natural disasters." + ), + }, + { + "id": "4", + "title": "Shipping lanes and fishing fleets", + "content": ( + "Lights at sea reveal human activity far from land. Concentrations of " + "light over the open ocean often come from fishing fleets that use bright " + "lamps to attract catch. Oil platforms and busy shipping lanes are also " + "visible in nighttime imagery." + ), + }, +] + + +def _require(name: str) -> str: + value = os.environ.get(name, "").strip() + if not value: + print(f"ERROR: required environment variable '{name}' is not set.", file=sys.stderr) + sys.exit(1) + return value + + +class SearchClient: + def __init__(self, endpoint: str, token: str) -> None: + self._endpoint = endpoint.rstrip("/") + self._headers = { + "Authorization": f"Bearer {token}", + "Content-Type": "application/json", + } + + def put(self, path: str, body: dict) -> None: + url = f"{self._endpoint}/{path}?api-version={API_VERSION}" + response = requests.put(url, headers=self._headers, json=body, timeout=120) + if response.status_code not in (200, 201, 204): + print(f"ERROR: PUT {path} failed ({response.status_code}): {response.text}", file=sys.stderr) + sys.exit(1) + + def post(self, path: str, body: dict) -> dict: + url = f"{self._endpoint}/{path}?api-version={API_VERSION}" + response = requests.post(url, headers=self._headers, json=body, timeout=120) + if response.status_code not in (200, 201): + print(f"ERROR: POST {path} failed ({response.status_code}): {response.text}", file=sys.stderr) + sys.exit(1) + return response.json() if response.content else {} + + +def create_index(client: SearchClient, index_name: str) -> None: + print(f"Creating index '{index_name}'...") + body = { + "name": index_name, + "fields": [ + {"name": "id", "type": "Edm.String", "key": True, "filterable": True}, + {"name": "title", "type": "Edm.String", "searchable": True, "retrievable": True}, + {"name": "content", "type": "Edm.String", "searchable": True, "retrievable": True}, + ], + "semantic": { + "configurations": [ + { + "name": SEMANTIC_CONFIG_NAME, + "prioritizedFields": { + "titleField": {"fieldName": "title"}, + "prioritizedContentFields": [{"fieldName": "content"}], + }, + } + ] + }, + } + client.put(f"indexes/{index_name}", body) + + +def upload_documents(client: SearchClient, index_name: str) -> None: + print(f"Uploading {len(DOCUMENTS)} document(s) to '{index_name}'...") + actions = [{"@search.action": "mergeOrUpload", **doc} for doc in DOCUMENTS] + client.post(f"indexes/{index_name}/docs/index", {"value": actions}) + + +def create_knowledge_source(client: SearchClient, ks_name: str, index_name: str) -> None: + print(f"Creating knowledge source '{ks_name}'...") + body = { + "name": ks_name, + "kind": "searchIndex", + "searchIndexParameters": { + "searchIndexName": index_name, + "semanticConfigurationName": SEMANTIC_CONFIG_NAME, + "sourceDataFields": [{"name": "title"}, {"name": "content"}], + "searchFields": [], + }, + } + client.put(f"knowledgesources/{ks_name}", body) + + +def create_knowledge_base(client: SearchClient, kb_name: str, ks_name: str) -> None: + print(f"Creating knowledge base '{kb_name}'...") + aoai_endpoint = _require("AZURE_OPENAI_ENDPOINT").rstrip("/") + deployment = _require("AZURE_AI_MODEL_DEPLOYMENT_NAME") + model_name = os.environ.get("AZURE_AI_MODEL_NAME", "").strip() or deployment + body = { + "name": kb_name, + "description": "Foundry IQ knowledge base for the hosted-agent sample.", + "knowledgeSources": [{"name": ks_name}], + "outputMode": "answerSynthesis", + "retrievalReasoningEffort": {"kind": "low"}, + "models": [ + { + "kind": "azureOpenAI", + "azureOpenAIParameters": { + # Keyless: the search service managed identity has the + # Cognitive Services User role on the Foundry account. + "resourceUri": aoai_endpoint, + "deploymentId": deployment, + "modelName": model_name, + }, + } + ], + } + client.put(f"knowledgebases/{kb_name}", body) + + +def _set_azd_env(name: str, value: str) -> bool: + """Best-effort: store ``value`` in the active azd environment. + + Returns ``True`` when ``azd env set`` succeeds. Falls back to ``False`` when + azd isn't installed or there's no active environment, so the script still + works when run standalone. + """ + azd = shutil.which("azd") + if not azd: + return False + try: + subprocess.run([azd, "env", "set", name, value], check=True) + return True + except (OSError, subprocess.CalledProcessError): + return False + + +def main() -> None: + load_dotenv() + + endpoint = _require("AZURE_SEARCH_ENDPOINT") + index_name = os.environ.get("AZURE_SEARCH_INDEX_NAME", "").strip() or "foundry-iq-index" + ks_name = os.environ.get("KNOWLEDGE_SOURCE_NAME", "").strip() or "foundry-iq-ks" + kb_name = os.environ.get("KNOWLEDGE_BASE_NAME", "").strip() or "foundry-iq-kb" + + token = DefaultAzureCredential().get_token(SEARCH_SCOPE).token + client = SearchClient(endpoint, token) + + create_index(client, index_name) + upload_documents(client, index_name) + create_knowledge_source(client, ks_name, index_name) + create_knowledge_base(client, kb_name, ks_name) + + mcp_endpoint = f"{endpoint.rstrip('/')}/knowledgebases/{kb_name}/mcp?api-version={API_VERSION}" + print() + print(f"Knowledge base '{kb_name}' is ready.") + print(f"MCP endpoint: {mcp_endpoint}") + print() + if _set_azd_env("KB_MCP_ENDPOINT", mcp_endpoint): + print("Stored the MCP endpoint as KB_MCP_ENDPOINT in your azd environment.") + else: + print("Next: point the toolbox connection at this MCP endpoint. Set") + print(' azd env set KB_MCP_ENDPOINT "' + mcp_endpoint + '"') + + +if __name__ == "__main__": + main() diff --git a/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/requirements.txt new file mode 100644 index 000000000..e9d0f0f2a --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/requirements.txt @@ -0,0 +1,16 @@ +# Install the narrow Foundry subpackages directly required by this sample +# (Agent, MCPStreamableHTTPTool, FoundryChatClient, ResponsesHostServer) instead +# of the bare ``agent-framework`` meta-package. The meta-package pulls in +# ``agent-framework-orchestrations``, whose latest release requires +# ``agent-framework-core>=1.9.0`` while ``agent-framework`` pins +# ``agent-framework-core==1.2.2`` — an unsatisfiable combination that fails the +# remote build with ``ResolutionImpossible``. The narrow ``agent-framework-foundry`` +# subpackage still provides ``agent_framework`` and ``agent_framework.foundry``. +agent-framework-foundry +agent-framework-foundry-hosting +# mcp backs MCPStreamableHTTPTool's runtime client and is imported by +# agent-framework-foundry-hosting (``from mcp import McpError``); it is needed at +# runtime but not declared as a required dependency of agent-framework-core. Pin +# below 2.0: the 2.0 line renamed ``McpError`` to ``MCPError`` and breaks startup. +mcp>=1.24.0,<2 +httpx diff --git a/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/toolbox.yaml b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/toolbox.yaml new file mode 100644 index 000000000..eb719cf03 --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/17-foundry-iq-toolbox/toolbox.yaml @@ -0,0 +1,30 @@ +# toolbox.yaml +# Defines the toolbox this agent consumes. It exposes the Foundry IQ knowledge +# base's MCP endpoint as a single tool (knowledge_base_retrieve), reached through +# the `knowledge-base-mcp` connection. +# +# Create the connection (once) from the KB MCP endpoint printed by provision_kb.py: +# azd ai connection create knowledge-base-mcp --kind remote-tool \ +# --target "" --auth-type agentic-identity \ +# --audience https://search.azure.com --metadata "ApiType=Azure" +# +# Then create the toolbox from this file: +# azd ai toolbox create knowledge-base --from-file ./toolbox.yaml +# +# `server_url` is the knowledge base's MCP endpoint and `project_connection_id` +# supplies the keyless Agentic Identity auth for it. The postprovision hook +# substitutes ${KB_MCP_ENDPOINT} (set by provision_kb.py) before creating the +# toolbox; when running by hand, replace it with the endpoint provision_kb.py +# printed. +# +# Copy the versioned MCP endpoint the create command prints and set it as the +# TOOLBOX_ENDPOINT environment variable. The agent connects to that endpoint +# at runtime using its managed identity (Agentic Identity). +description: Foundry IQ knowledge base behind one endpoint +tools: + - type: mcp + name: knowledge_base + server_label: knowledge_base + server_url: ${KB_MCP_ENDPOINT} + project_connection_id: knowledge-base-mcp + require_approval: never From 82d51af0fa097f7b13fc5e8d69c8dbf7d239b964 Mon Sep 17 00:00:00 2001 From: Linda Li <139801625+lindazqli@users.noreply.github.com> Date: Tue, 23 Jun 2026 08:16:18 -0700 Subject: [PATCH 2/4] samples(python/13-foundry-memory): one-command azd provisioning via postprovision hook (#576) Add a postprovision hook (.ps1/.sh) that creates the Foundry Memory Store and wires MEMORY_STORE_NAME so a single 'azd provision' is enough. The hook also patches the init'd agent.yaml so the resolved store name reaches the deployed container (azd ai agent init resolves \ to empty at init time). Pin mcp<2 (agent-framework-foundry-hosting imports McpError, renamed in mcp 2.x). Document the project-scope Cognitive Services OpenAI User role needed for the embedding call. Exclude hooks/ from container/azd packaging. --- .../responses/13-foundry-memory/.azdignore | 1 + .../responses/13-foundry-memory/.dockerignore | 1 + .../responses/13-foundry-memory/README.md | 122 +++++++++++------- .../13-foundry-memory/hooks/postprovision.ps1 | 69 ++++++++++ .../13-foundry-memory/hooks/postprovision.sh | 62 +++++++++ .../13-foundry-memory/requirements.txt | 6 +- 6 files changed, 210 insertions(+), 51 deletions(-) create mode 100644 samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/hooks/postprovision.ps1 create mode 100644 samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/hooks/postprovision.sh diff --git a/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/.azdignore b/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/.azdignore index 4a74eabf4..301a106fc 100644 --- a/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/.azdignore +++ b/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/.azdignore @@ -1,3 +1,4 @@ agent.manifest.yaml agent.yaml .env.example +hooks/ diff --git a/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/.dockerignore b/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/.dockerignore index 084806822..3e5da2b03 100644 --- a/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/.dockerignore +++ b/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/.dockerignore @@ -6,3 +6,4 @@ __pycache__ .Python .env provision_memory_store.py +hooks/ diff --git a/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/README.md b/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/README.md index f948c40c1..c54dee42e 100644 --- a/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/README.md +++ b/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/README.md @@ -33,47 +33,19 @@ The agent is hosted using the [Agent Framework](https://github.com/microsoft/age ### Required RBAC -Your identity (or the Managed Identity running the container in production) needs **Azure AI User** on the Foundry project scope. This single role covers both provisioning the memory store with `provision_memory_store.py` and reading/writing memories from `main.py`. +Your identity (or the Managed Identity running the container in production) needs **Azure AI User** on the Foundry project scope. This role covers provisioning the memory store with `provision_memory_store.py` and reading/writing memories from `main.py`. -## Provisioning the memory store (one time) +The memory store embeds and retrieves memories through the project's inference endpoint, so the same identity also needs **Cognitive Services OpenAI User** on the Foundry project scope to call the embedding deployment. Without it, memory writes fail with a `401` (`Authentication to the Azure OpenAI resource failed`) and the store stays empty. When deploying, grant both roles to the hosted agent's runtime identity (the `…-AgentIdentity` service principal) at the project scope. -[`provision_memory_store.py`](provision_memory_store.py) creates a Foundry Memory Store with the user-profile capability enabled (and chat-summary disabled) using `AIProjectClient.beta.memory_stores.create`. It is safe to re-run: if a store with the same name already exists, the script leaves it alone. - -From this directory, with the venv activated and `az login` done: - -```bash -export FOUNDRY_PROJECT_ENDPOINT="https://.services.ai.azure.com/api/projects/" -export AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4.1-mini" -export AZURE_AI_EMBEDDING_MODEL_DEPLOYMENT_NAME="text-embedding-3-small" -export MEMORY_STORE_NAME="agent_framework_memory" -python provision_memory_store.py -``` - -Or in PowerShell: - -```powershell -$env:FOUNDRY_PROJECT_ENDPOINT="https://.services.ai.azure.com/api/projects/" -$env:AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4.1-mini" -$env:AZURE_AI_EMBEDDING_MODEL_DEPLOYMENT_NAME="text-embedding-3-small" -$env:MEMORY_STORE_NAME="agent_framework_memory" -python provision_memory_store.py -``` - -Expected output (first run): - -```text -Creating memory store 'agent_framework_memory'... -Created memory store 'agent_framework_memory' (id=memstore_...). -``` - -> To delete the store manually, call `project.beta.memory_stores.delete("")` on an `AIProjectClient` constructed with `allow_preview=True`. ## Option 1: Azure Developer CLI (`azd`) -### Prerequisites +With the bundled `postprovision` hook, a single `azd provision` creates the Foundry Memory Store and sets `MEMORY_STORE_NAME` for you. + +### 1. Install prerequisites -1. **Azure Developer CLI (`azd`)** — [Install azd](https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/install-azd) -2. Install the AI agent extension: +1. **Azure Developer CLI (`azd`)** — [Install azd](https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/install-azd) (1.25 or later) +2. Install the unified Foundry CLI extension bundle: ```bash azd ext install microsoft.foundry ``` @@ -82,7 +54,7 @@ Created memory store 'agent_framework_memory' (id=memstore_...). azd auth login ``` -### Initialize the agent project +### 2. Initialize the agent project No cloning required. Create a new folder and initialize from the manifest: @@ -92,17 +64,42 @@ mkdir my-memory-agent && cd my-memory-agent azd ai agent init -m https://github.com/microsoft-foundry/foundry-samples/blob/main/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/agent.manifest.yaml ``` -Follow the prompts to configure your Foundry project and model deployment. If you don't have an existing Foundry project, `azd ai agent init` will guide you through creating one. +Follow the prompts to configure your Foundry project and model deployment. If you don't have an existing Foundry project, `azd ai agent init` will guide you through creating one. Initializing also sets the selected project as the active project, and copies this sample's files into a new service directory `src//` — including [`provision_memory_store.py`](provision_memory_store.py) and the [`hooks/`](hooks/) scripts. -### Provision Azure resources (if needed) +### 3. Enable one-command provisioning (`postprovision` hook) -If you don't already have a Foundry project and model deployment: +Wire the bundled hook into the `azure.yaml` that `azd ai agent init` generated, so the memory store is created automatically every time you run `azd provision`. `postprovision` must be registered at the **top level** of `azure.yaml` (service-scoped hooks only support the package/deploy lifecycle), and the `run:` path must point at the hook inside the generated service directory. Add this top-level block, replacing `` with the service folder `azd ai agent init` created under `src/`: + +```yaml +hooks: + postprovision: + posix: + shell: sh + run: ./src//hooks/postprovision.sh + windows: + shell: pwsh + run: ./src//hooks/postprovision.ps1 +``` + +The hook ([`hooks/postprovision.sh`](hooks/postprovision.sh) / [`hooks/postprovision.ps1`](hooks/postprovision.ps1)) runs everything the [manual steps](#provision-manually-without-the-hook) below would, in one shot. It locates its own directory, so it works no matter where `azd` runs it from. + +### 4. Provision + +Point the hook at an embedding model deployment in your Foundry project (it powers the store's semantic memory, not the agent at runtime), then provision: ```bash +azd env set AZURE_AI_EMBEDDING_MODEL_DEPLOYMENT_NAME "text-embedding-3-small" azd provision ``` -### Run the agent locally +`azd provision` creates (or reuses) your Foundry project and chat model deployment, then the `postprovision` hook: + +1. Runs [`provision_memory_store.py`](provision_memory_store.py) to create the Foundry Memory Store (user-profile capability enabled, chat-summary disabled) and verifies it on the service. +2. Sets `MEMORY_STORE_NAME` so the agent reads and writes that store. It persists the name both to the `azd` environment (for `azd ai agent run`) and into the generated `src//agent.yaml` (so `azd deploy` ships it to the container — `azd ai agent init` resolves `${MEMORY_STORE_NAME}` to an empty value at init time, before the store name is known). + +> The hook defaults `MEMORY_STORE_NAME` to `agent_framework_memory`. To use a different name, set it first: `azd env set MEMORY_STORE_NAME ""`. + +### 5. Run the agent locally ```bash azd ai agent run @@ -115,35 +112,60 @@ The agent host will start on `http://localhost:8088`. In a separate terminal, from the project directory: ```bash -azd ai agent invoke --local "Hi" +azd ai agent invoke --local "Hi, my name is Alex and I'm vegetarian." ``` ### Deploy to Foundry -Once tested locally, deploy to Microsoft Foundry: +```bash +azd deploy +``` + +For the full deployment guide, see [Deploy a hosted agent](https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/deploy-hosted-agent). + +The deployed agent's Managed Identity needs **Azure AI User** on the Foundry project to read and write memories at runtime. The `postprovision` hook already created the memory store against that same project. -Make sure `MEMORY_STORE_NAME` is set in your `azd` environment: +### Invoke the deployed agent ```bash -azd env set MEMORY_STORE_NAME "agent_framework_memory" +azd ai agent invoke "Do you remember my name and what I like to eat?" ``` +### Provision manually (without the hook) + +Prefer to run the step yourself (or skip the hook)? [`provision_memory_store.py`](provision_memory_store.py) creates a Foundry Memory Store with the user-profile capability enabled (and chat-summary disabled) using `AIProjectClient.beta.memory_stores.create`. It is safe to re-run: if a store with the same name already exists, the script leaves it alone. + +From the project directory, with the venv activated and `az login` done: + ```bash -azd deploy +pip install azure-ai-projects azure-identity aiohttp python-dotenv + +export FOUNDRY_PROJECT_ENDPOINT="https://.services.ai.azure.com/api/projects/" +export AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4.1-mini" +export AZURE_AI_EMBEDDING_MODEL_DEPLOYMENT_NAME="text-embedding-3-small" +export MEMORY_STORE_NAME="agent_framework_memory" +python provision_memory_store.py ``` -For the full deployment guide, see [Deploy a hosted agent](https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/deploy-hosted-agent). +In PowerShell, use `$env:NAME="value"` instead of `export`. Then point the agent at the same store name: -The deployed agent's Managed Identity needs **Azure AI User** on the Foundry project to read and write memories at runtime. Make sure you have run `provision_memory_store.py` against the same Foundry project before deploying. +```bash +azd env set MEMORY_STORE_NAME "agent_framework_memory" +``` -### Invoke the deployed agent +Expected output (first run): -```bash -azd ai agent invoke "Hi" +```text +Creating memory store 'agent_framework_memory'... +Created memory store 'agent_framework_memory' (id=memstore_...). ``` +> To delete the store manually, call `project.beta.memory_stores.delete("")` on an `AIProjectClient` constructed with `allow_preview=True`. + ## Option 2: VS Code (Foundry Toolkit) +> The VS Code flow doesn't run the `azd` hook — provision the memory store first with [Provision manually](#provision-manually-without-the-hook). + ### Prerequisites 1. **VS Code** with the **[Foundry Toolkit](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.azure-ai-foundry)** extension installed. diff --git a/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/hooks/postprovision.ps1 b/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/hooks/postprovision.ps1 new file mode 100644 index 000000000..a36f2e293 --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/hooks/postprovision.ps1 @@ -0,0 +1,69 @@ +#!/usr/bin/env pwsh +# azd postprovision hook (Windows / pwsh). +# +# Runs automatically after `azd provision`. It provisions the Azure AI Foundry +# Memory Store the agent uses and stores MEMORY_STORE_NAME so the agent can +# reach it — collapsing the manual provisioning step into a single `azd provision`. +# +# Wire it up by adding this to the azure.yaml generated by `azd ai agent init`: +# +# hooks: +# postprovision: +# posix: +# shell: sh +# run: ./src//hooks/postprovision.sh +# windows: +# shell: pwsh +# run: ./src//hooks/postprovision.ps1 + +$ErrorActionPreference = "Stop" + +# PowerShell does not stop on a non-zero exit code from a native command (like +# azd), so check $LASTEXITCODE after each azd call and fail loudly. +function Invoke-Checked { + param([scriptblock] $Script, [string] $What) + & $Script + if ($LASTEXITCODE -ne 0) { throw "$What failed (exit $LASTEXITCODE)." } +} + +# Run from the sample directory (the parent of hooks/) so provision_memory_store.py +# resolves no matter which directory azd invokes the hook from. +Set-Location (Split-Path -Parent $PSScriptRoot) + +# The memory store needs an embedding model deployment; it isn't part of the +# agent's own model resources, so it's a prerequisite you set once. +if (-not $env:AZURE_AI_EMBEDDING_MODEL_DEPLOYMENT_NAME) { + throw "AZURE_AI_EMBEDDING_MODEL_DEPLOYMENT_NAME is not set. Run: azd env set AZURE_AI_EMBEDDING_MODEL_DEPLOYMENT_NAME ""text-embedding-3-small""" +} + +# Default the memory store name and persist it so the agent (which reads +# MEMORY_STORE_NAME at deploy time) and the provisioning script agree. +if (-not $env:MEMORY_STORE_NAME) { + $env:MEMORY_STORE_NAME = "agent_framework_memory" + Invoke-Checked { azd env set MEMORY_STORE_NAME $env:MEMORY_STORE_NAME } "env set MEMORY_STORE_NAME" +} + +Write-Host "Provisioning the Foundry Memory Store '$($env:MEMORY_STORE_NAME)'..." +# provision_memory_store.py uses the async azure-ai-projects client, which needs +# aiohttp as its transport. Install the narrow set the script imports. +Invoke-Checked { python -m pip install -q azure-ai-projects azure-identity aiohttp python-dotenv } "pip install" +# Idempotent: leaves an existing store with the same name untouched. +Invoke-Checked { python provision_memory_store.py } "provision_memory_store.py" + +# `azd ai agent init` resolves ${MEMORY_STORE_NAME} in agent.yaml at init time — +# before this hook runs and sets the value — so the manifest is left with an empty +# value and `azd deploy` would ship MEMORY_STORE_NAME="" to the container, leaving +# the agent with no store to read or write. Write the resolved name into agent.yaml +# so the deployed agent gets it. Idempotent. +$manifest = Join-Path (Get-Location) "agent.yaml" +if (Test-Path $manifest) { + $content = [System.IO.File]::ReadAllText($manifest) + $pattern = '(?m)(^[ \t]*-[ \t]*name:[ \t]*MEMORY_STORE_NAME[ \t]*\r?\n[ \t]*value:[ \t]*).*$' + $updated = [System.Text.RegularExpressions.Regex]::Replace($content, $pattern, ('${1}' + $env:MEMORY_STORE_NAME)) + if ($updated -ne $content) { + [System.IO.File]::WriteAllText($manifest, $updated) + Write-Host "Set MEMORY_STORE_NAME in agent.yaml to '$($env:MEMORY_STORE_NAME)' for deploy." + } +} + +Write-Host "Done. MEMORY_STORE_NAME = $($env:MEMORY_STORE_NAME)" diff --git a/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/hooks/postprovision.sh b/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/hooks/postprovision.sh new file mode 100644 index 000000000..10247dda8 --- /dev/null +++ b/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/hooks/postprovision.sh @@ -0,0 +1,62 @@ +#!/usr/bin/env sh +# azd postprovision hook (POSIX / sh). +# +# Runs automatically after `azd provision`. It provisions the Azure AI Foundry +# Memory Store the agent uses and stores MEMORY_STORE_NAME so the agent can +# reach it — collapsing the manual provisioning step into a single `azd provision`. +# +# Wire it up by adding this to the azure.yaml generated by `azd ai agent init`: +# +# hooks: +# postprovision: +# posix: +# shell: sh +# run: ./src//hooks/postprovision.sh +# windows: +# shell: pwsh +# run: ./src//hooks/postprovision.ps1 + +set -e + +# Run from the sample directory (the parent of hooks/) so provision_memory_store.py +# resolves no matter which directory azd invokes the hook from. +cd "$(CDPATH= cd "$(dirname "$0")/.." && pwd)" + +# The memory store needs an embedding model deployment; it isn't part of the +# agent's own model resources, so it's a prerequisite you set once. +if [ -z "$AZURE_AI_EMBEDDING_MODEL_DEPLOYMENT_NAME" ]; then + echo "AZURE_AI_EMBEDDING_MODEL_DEPLOYMENT_NAME is not set. Run: azd env set AZURE_AI_EMBEDDING_MODEL_DEPLOYMENT_NAME \"text-embedding-3-small\"" >&2 + exit 1 +fi + +# Default the memory store name and persist it so the agent (which reads +# MEMORY_STORE_NAME at deploy time) and the provisioning script agree. +if [ -z "$MEMORY_STORE_NAME" ]; then + MEMORY_STORE_NAME="agent_framework_memory" + export MEMORY_STORE_NAME + azd env set MEMORY_STORE_NAME "$MEMORY_STORE_NAME" +fi + +echo "Provisioning the Foundry Memory Store '$MEMORY_STORE_NAME'..." +# provision_memory_store.py uses the async azure-ai-projects client, which needs +# aiohttp as its transport. Install the narrow set the script imports. +python -m pip install -q azure-ai-projects azure-identity aiohttp python-dotenv +# Idempotent: leaves an existing store with the same name untouched. +python provision_memory_store.py + +# `azd ai agent init` resolves ${MEMORY_STORE_NAME} in agent.yaml at init time — +# before this hook runs and sets the value — so the manifest is left with an empty +# value and `azd deploy` would ship MEMORY_STORE_NAME="" to the container, leaving +# the agent with no store to read or write. Write the resolved name into agent.yaml +# so the deployed agent gets it. Idempotent. +if [ -f agent.yaml ]; then + awk -v val="$MEMORY_STORE_NAME" ' + prev ~ /name:[[:space:]]*MEMORY_STORE_NAME[[:space:]]*$/ && /value:/ { + sub(/value:.*/, "value: " val) + } + { print; prev = $0 } + ' agent.yaml > agent.yaml.tmp && mv agent.yaml.tmp agent.yaml + echo "Set MEMORY_STORE_NAME in agent.yaml to '$MEMORY_STORE_NAME' for deploy." +fi + +echo "Done. MEMORY_STORE_NAME = $MEMORY_STORE_NAME" diff --git a/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/requirements.txt index 265575ddd..4ae27c145 100644 --- a/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/requirements.txt @@ -1,5 +1,9 @@ # Use the narrow Foundry subpackages to keep dependencies light. agent-framework-foundry agent-framework-foundry-hosting -mcp<2,>=1.24.0 +# mcp is a transitive dependency of agent-framework-core[all] that is needed +# at runtime but not declared as a required dependency of agent-framework-core. +# Pin below 2.0: agent-framework-foundry-hosting imports ``McpError`` from mcp, +# which the 2.0 line renamed to ``MCPError`` — installing mcp 2.x breaks startup. +mcp>=1.24.0,<2 azure-ai-projects From 69d9017ce491d7aad7143724a6e40fc759852881 Mon Sep 17 00:00:00 2001 From: Hui Miao Date: Wed, 24 Jun 2026 13:11:59 +0800 Subject: [PATCH 3/4] fix: add debugpy to Python samples for Foundry Toolkit VS Code local debugging (#591) --- .../a2a/01-delegation/caller/requirements.txt | 3 +++ .../a2a/01-delegation/executor/requirements.txt | 3 +++ .../agent-framework/invocations/01-basic/requirements.txt | 5 ++++- .../agent-framework/responses/01-basic/requirements.txt | 5 ++++- .../agent-framework/responses/02-tools/requirements.txt | 5 ++++- .../agent-framework/responses/03-mcp/requirements.txt | 5 ++++- .../responses/04-foundry-toolbox/requirements.txt | 3 +++ .../agent-framework/responses/05-workflows/requirements.txt | 5 ++++- .../agent-framework/responses/06-files/requirements.txt | 5 ++++- .../agent-framework/responses/07-skills/requirements.txt | 3 +++ .../responses/07-teams-activity/requirements.txt | 3 +++ .../responses/08-observability/requirements.txt | 5 ++++- .../09-declarative-customer-support/requirements.txt | 3 +++ .../responses/10-downstream-azure/requirements.txt | 3 +++ .../responses/11-azure-search-rag/requirements.txt | 3 +++ .../responses/12-foundry-skills/requirements.txt | 3 +++ .../responses/13-foundry-memory/requirements.txt | 3 +++ .../responses/14-browser-automation-agent/requirements.txt | 3 +++ .../15-optimization-travel-approver/requirements.txt | 3 +++ .../responses/16-content-safety-guardrail/requirements.txt | 5 ++++- .../bring-your-own/invocations/ag-ui/requirements.txt | 3 +++ .../invocations/claude-agent-sdk/requirements.txt | 3 +++ .../invocations/diagnostic-agent/requirements.txt | 3 +++ .../invocations/event-grid-trigger/requirements.txt | 5 ++++- .../invocations/github-copilot/requirements.txt | 3 +++ .../bring-your-own/invocations/hello-world/requirements.txt | 3 +++ .../invocations/human-in-the-loop/requirements.txt | 3 +++ .../invocations/langgraph-chat/requirements.txt | 3 +++ .../invocations/notetaking-agent/requirements.txt | 3 +++ .../bring-your-own/invocations/toolbox/requirements.txt | 3 +++ .../invocations_ws/duplex-live-agent/requirements.txt | 3 +++ .../invocations_ws/hello-world/requirements.txt | 3 +++ .../invocations_ws/livekit-server/requirements.txt | 3 +++ .../invocations_ws/pipecat-webrtc/requirements.txt | 3 +++ .../invocations_ws/pipecat-ws-server/requirements.txt | 5 ++++- .../responses/background-agent/requirements.txt | 3 +++ .../responses/bring-your-own-toolbox/requirements.txt | 3 +++ .../responses/browser-automation/requirements.txt | 3 +++ .../bring-your-own/responses/env-vars-agent/requirements.txt | 3 +++ .../bring-your-own/responses/hello-world/requirements.txt | 3 +++ .../bring-your-own/responses/langgraph-chat/requirements.txt | 3 +++ .../langgraph-toolbox-user-identity/requirements.txt | 3 +++ .../responses/langgraph-toolbox/requirements.txt | 3 +++ .../responses/notetaking-agent/requirements.txt | 3 +++ .../responses/openai-agents-sdk/requirements.txt | 3 +++ .../responses/optimization-customer-support/requirements.txt | 3 +++ .../responses/optimization-hello-world/requirements.txt | 3 +++ .../requirements.txt | 3 +++ .../handoff-langgraph-responses-voicelive/requirements.txt | 3 +++ .../hello-world-invocations-voicelive/requirements.txt | 3 +++ .../hotel-booking-invocations-voicelive/requirements.txt | 3 +++ .../hosted-agents/langgraph/a2a/a2a-caller/requirements.txt | 3 +++ .../langgraph/a2a/a2a-executor/requirements.txt | 3 +++ .../langgraph/invocations/01-langgraph-chat/requirements.txt | 3 +++ .../langgraph/responses/01-langgraph-chat/requirements.txt | 3 +++ .../responses/02-langgraph-toolbox/requirements.txt | 3 +++ .../langgraph/responses/04-mcp/requirements.txt | 3 +++ .../langgraph/responses/05-workflows/requirements.txt | 3 +++ .../langgraph/responses/06-files/requirements.txt | 3 +++ .../responses/07-human-in-the-loop/requirements.txt | 3 +++ .../langgraph/responses/08-observability/requirements.txt | 3 +++ 61 files changed, 193 insertions(+), 10 deletions(-) diff --git a/samples/python/hosted-agents/agent-framework/a2a/01-delegation/caller/requirements.txt b/samples/python/hosted-agents/agent-framework/a2a/01-delegation/caller/requirements.txt index d029dbf93..0d3922941 100644 --- a/samples/python/hosted-agents/agent-framework/a2a/01-delegation/caller/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/a2a/01-delegation/caller/requirements.txt @@ -3,3 +3,6 @@ agent-framework-foundry agent-framework-foundry-hosting mcp<2,>=1.24.0 httpx + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/a2a/01-delegation/executor/requirements.txt b/samples/python/hosted-agents/agent-framework/a2a/01-delegation/executor/requirements.txt index 926e5079f..3b0ee9a88 100644 --- a/samples/python/hosted-agents/agent-framework/a2a/01-delegation/executor/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/a2a/01-delegation/executor/requirements.txt @@ -1,3 +1,6 @@ # Use the narrow Foundry subpackages to keep dependencies light. agent-framework-foundry agent-framework-foundry-hosting + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/invocations/01-basic/requirements.txt b/samples/python/hosted-agents/agent-framework/invocations/01-basic/requirements.txt index 0b85d57fa..03962b4c6 100644 --- a/samples/python/hosted-agents/agent-framework/invocations/01-basic/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/invocations/01-basic/requirements.txt @@ -1,2 +1,5 @@ agent-framework-foundry -agent-framework-foundry-hosting \ No newline at end of file +agent-framework-foundry-hosting + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/01-basic/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/01-basic/requirements.txt index 92b926abf..01f3e7bc9 100644 --- a/samples/python/hosted-agents/agent-framework/responses/01-basic/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/01-basic/requirements.txt @@ -1,4 +1,7 @@ # Use the narrow Foundry subpackages to keep dependencies light. agent-framework-foundry agent-framework-foundry-hosting -mcp<2,>=1.24.0 \ No newline at end of file +mcp<2,>=1.24.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/02-tools/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/02-tools/requirements.txt index 92b926abf..01f3e7bc9 100644 --- a/samples/python/hosted-agents/agent-framework/responses/02-tools/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/02-tools/requirements.txt @@ -1,4 +1,7 @@ # Use the narrow Foundry subpackages to keep dependencies light. agent-framework-foundry agent-framework-foundry-hosting -mcp<2,>=1.24.0 \ No newline at end of file +mcp<2,>=1.24.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/03-mcp/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/03-mcp/requirements.txt index 92b926abf..01f3e7bc9 100644 --- a/samples/python/hosted-agents/agent-framework/responses/03-mcp/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/03-mcp/requirements.txt @@ -1,4 +1,7 @@ # Use the narrow Foundry subpackages to keep dependencies light. agent-framework-foundry agent-framework-foundry-hosting -mcp<2,>=1.24.0 \ No newline at end of file +mcp<2,>=1.24.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/04-foundry-toolbox/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/04-foundry-toolbox/requirements.txt index 4d88caef9..01f3e7bc9 100644 --- a/samples/python/hosted-agents/agent-framework/responses/04-foundry-toolbox/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/04-foundry-toolbox/requirements.txt @@ -2,3 +2,6 @@ agent-framework-foundry agent-framework-foundry-hosting mcp<2,>=1.24.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/05-workflows/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/05-workflows/requirements.txt index 92b926abf..01f3e7bc9 100644 --- a/samples/python/hosted-agents/agent-framework/responses/05-workflows/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/05-workflows/requirements.txt @@ -1,4 +1,7 @@ # Use the narrow Foundry subpackages to keep dependencies light. agent-framework-foundry agent-framework-foundry-hosting -mcp<2,>=1.24.0 \ No newline at end of file +mcp<2,>=1.24.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/06-files/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/06-files/requirements.txt index 92b926abf..01f3e7bc9 100644 --- a/samples/python/hosted-agents/agent-framework/responses/06-files/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/06-files/requirements.txt @@ -1,4 +1,7 @@ # Use the narrow Foundry subpackages to keep dependencies light. agent-framework-foundry agent-framework-foundry-hosting -mcp<2,>=1.24.0 \ No newline at end of file +mcp<2,>=1.24.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/07-skills/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/07-skills/requirements.txt index 4d88caef9..01f3e7bc9 100644 --- a/samples/python/hosted-agents/agent-framework/responses/07-skills/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/07-skills/requirements.txt @@ -2,3 +2,6 @@ agent-framework-foundry agent-framework-foundry-hosting mcp<2,>=1.24.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/07-teams-activity/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/07-teams-activity/requirements.txt index 4d88caef9..01f3e7bc9 100644 --- a/samples/python/hosted-agents/agent-framework/responses/07-teams-activity/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/07-teams-activity/requirements.txt @@ -2,3 +2,6 @@ agent-framework-foundry agent-framework-foundry-hosting mcp<2,>=1.24.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/08-observability/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/08-observability/requirements.txt index 92b926abf..01f3e7bc9 100644 --- a/samples/python/hosted-agents/agent-framework/responses/08-observability/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/08-observability/requirements.txt @@ -1,4 +1,7 @@ # Use the narrow Foundry subpackages to keep dependencies light. agent-framework-foundry agent-framework-foundry-hosting -mcp<2,>=1.24.0 \ No newline at end of file +mcp<2,>=1.24.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/09-declarative-customer-support/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/09-declarative-customer-support/requirements.txt index b711255bc..c44de55a7 100644 --- a/samples/python/hosted-agents/agent-framework/responses/09-declarative-customer-support/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/09-declarative-customer-support/requirements.txt @@ -3,3 +3,6 @@ agent-framework-foundry agent-framework-declarative agent-framework-foundry-hosting mcp<2,>=1.24.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/10-downstream-azure/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/10-downstream-azure/requirements.txt index 01d403920..b77521a8f 100644 --- a/samples/python/hosted-agents/agent-framework/responses/10-downstream-azure/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/10-downstream-azure/requirements.txt @@ -4,3 +4,6 @@ agent-framework-foundry-hosting mcp<2,>=1.24.0 azure-storage-blob azure-servicebus + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/11-azure-search-rag/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/11-azure-search-rag/requirements.txt index c32a049ee..2eefb8749 100644 --- a/samples/python/hosted-agents/agent-framework/responses/11-azure-search-rag/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/11-azure-search-rag/requirements.txt @@ -3,3 +3,6 @@ agent-framework-foundry agent-framework-azure-ai-search agent-framework-foundry-hosting mcp<2,>=1.24.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/12-foundry-skills/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/12-foundry-skills/requirements.txt index 265575ddd..11e34099a 100644 --- a/samples/python/hosted-agents/agent-framework/responses/12-foundry-skills/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/12-foundry-skills/requirements.txt @@ -3,3 +3,6 @@ agent-framework-foundry agent-framework-foundry-hosting mcp<2,>=1.24.0 azure-ai-projects + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/requirements.txt index 4ae27c145..8294eab63 100644 --- a/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/13-foundry-memory/requirements.txt @@ -7,3 +7,6 @@ agent-framework-foundry-hosting # which the 2.0 line renamed to ``MCPError`` — installing mcp 2.x breaks startup. mcp>=1.24.0,<2 azure-ai-projects + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/14-browser-automation-agent/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/14-browser-automation-agent/requirements.txt index f57108c73..a3f2907cc 100644 --- a/samples/python/hosted-agents/agent-framework/responses/14-browser-automation-agent/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/14-browser-automation-agent/requirements.txt @@ -7,3 +7,6 @@ mcp<2,>=1.24.0 pydantic>=2.0.0 python-dotenv>=1.0.0 websockets>=14.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/15-optimization-travel-approver/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/15-optimization-travel-approver/requirements.txt index f43ba2962..7c9c9b254 100644 --- a/samples/python/hosted-agents/agent-framework/responses/15-optimization-travel-approver/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/15-optimization-travel-approver/requirements.txt @@ -4,3 +4,6 @@ agent-framework-foundry-hosting mcp<2,>=1.24.0 azure-identity>=1.15.0 azure-ai-agentserver-optimization>=1.0.0b1 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/agent-framework/responses/16-content-safety-guardrail/requirements.txt b/samples/python/hosted-agents/agent-framework/responses/16-content-safety-guardrail/requirements.txt index 92b926abf..01f3e7bc9 100644 --- a/samples/python/hosted-agents/agent-framework/responses/16-content-safety-guardrail/requirements.txt +++ b/samples/python/hosted-agents/agent-framework/responses/16-content-safety-guardrail/requirements.txt @@ -1,4 +1,7 @@ # Use the narrow Foundry subpackages to keep dependencies light. agent-framework-foundry agent-framework-foundry-hosting -mcp<2,>=1.24.0 \ No newline at end of file +mcp<2,>=1.24.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations/ag-ui/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations/ag-ui/requirements.txt index 609538a41..eb948ea6a 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations/ag-ui/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations/ag-ui/requirements.txt @@ -3,3 +3,6 @@ openai==2.32.0 azure-identity==1.25.3 ag-ui-protocol==0.1.16 azure-ai-agentserver-invocations==1.0.0b4 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations/claude-agent-sdk/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations/claude-agent-sdk/requirements.txt index 494144971..7cee8fd32 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations/claude-agent-sdk/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations/claude-agent-sdk/requirements.txt @@ -1,3 +1,6 @@ claude-agent-sdk>=0.1.74 azure-ai-agentserver-invocations==1.0.0b4 httpx<1.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations/diagnostic-agent/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations/diagnostic-agent/requirements.txt index f8acbf1a8..febb022af 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations/diagnostic-agent/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations/diagnostic-agent/requirements.txt @@ -1 +1,4 @@ azure-ai-agentserver-invocations==1.0.0b4 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations/event-grid-trigger/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations/event-grid-trigger/requirements.txt index a8a0c6a01..154288e5b 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations/event-grid-trigger/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations/event-grid-trigger/requirements.txt @@ -3,4 +3,7 @@ azure-ai-agentserver-invocations==1.0.0b4 azure-ai-projects==2.0.1 azure-identity==1.25.3 azure-storage-blob==12.27.0 -python-dotenv \ No newline at end of file +python-dotenv + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations/github-copilot/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations/github-copilot/requirements.txt index 90f2dbf71..b9c77e273 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations/github-copilot/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations/github-copilot/requirements.txt @@ -3,3 +3,6 @@ github-copilot-sdk>=0.2.0 azure-ai-agentserver-invocations==1.0.0b4 azure-identity>=1.17.0 python-dotenv==1.1.1 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations/hello-world/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations/hello-world/requirements.txt index 382ddc6f4..248529105 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations/hello-world/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations/hello-world/requirements.txt @@ -2,3 +2,6 @@ aiohttp==3.13.5 azure-ai-agentserver-invocations==1.0.0b4 azure-ai-projects==2.0.1 azure-identity==1.25.3 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations/human-in-the-loop/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations/human-in-the-loop/requirements.txt index a157cbcf2..516a8ba36 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations/human-in-the-loop/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations/human-in-the-loop/requirements.txt @@ -1,3 +1,6 @@ azure-ai-agentserver-invocations==1.0.0b4 azure-ai-projects==2.0.1 azure-identity==1.25.3 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations/langgraph-chat/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations/langgraph-chat/requirements.txt index 726ddc9df..a250b317e 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations/langgraph-chat/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations/langgraph-chat/requirements.txt @@ -4,3 +4,6 @@ langgraph==1.1.8 langgraph-prebuilt==1.0.10 langchain-core==1.3.0 langchain-azure-ai[opentelemetry]>=1.2.3 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations/notetaking-agent/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations/notetaking-agent/requirements.txt index a157cbcf2..516a8ba36 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations/notetaking-agent/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations/notetaking-agent/requirements.txt @@ -1,3 +1,6 @@ azure-ai-agentserver-invocations==1.0.0b4 azure-ai-projects==2.0.1 azure-identity==1.25.3 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations/toolbox/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations/toolbox/requirements.txt index 37b5a7d17..99c1cf791 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations/toolbox/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations/toolbox/requirements.txt @@ -3,3 +3,6 @@ azure-ai-projects==2.0.1 azure-identity==1.25.3 httpx python-dotenv==1.1.1 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations_ws/duplex-live-agent/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations_ws/duplex-live-agent/requirements.txt index e9c4ea780..9f2a563a2 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations_ws/duplex-live-agent/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations_ws/duplex-live-agent/requirements.txt @@ -9,3 +9,6 @@ agent-framework-foundry agent-framework-orchestrations github-copilot-sdk>=0.2.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations_ws/hello-world/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations_ws/hello-world/requirements.txt index ed1922f48..ed729f3dd 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations_ws/hello-world/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations_ws/hello-world/requirements.txt @@ -2,3 +2,6 @@ azure-ai-agentserver-invocations>=1.0.0b4 azure-ai-voicelive[aiohttp]>=1.2.0 azure-identity>=1.20.0 python-dotenv>=1.0.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations_ws/livekit-server/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations_ws/livekit-server/requirements.txt index 3df26edd0..0a94da0e2 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations_ws/livekit-server/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations_ws/livekit-server/requirements.txt @@ -7,3 +7,6 @@ fastapi>=0.115.0 uvicorn[standard]>=0.30.0 python-dotenv>=1.0.0 loguru>=0.7.2 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations_ws/pipecat-webrtc/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations_ws/pipecat-webrtc/requirements.txt index a365b0171..313cdbbaa 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations_ws/pipecat-webrtc/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations_ws/pipecat-webrtc/requirements.txt @@ -10,3 +10,6 @@ httpx # libs (libxcb1, libGL, ...) that aren't present in python:slim. Pin the # headless wheel here so pip won't install the GUI variant. opencv-python-headless + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/invocations_ws/pipecat-ws-server/requirements.txt b/samples/python/hosted-agents/bring-your-own/invocations_ws/pipecat-ws-server/requirements.txt index c98f4aca9..c73852789 100644 --- a/samples/python/hosted-agents/bring-your-own/invocations_ws/pipecat-ws-server/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/invocations_ws/pipecat-ws-server/requirements.txt @@ -2,4 +2,7 @@ python-dotenv uvicorn pipecat-ai[silero,websocket,azure]>=0.0.105 pipecat-ai-subagents>=0.3.0 -azure-ai-transcription \ No newline at end of file +azure-ai-transcription + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/responses/background-agent/requirements.txt b/samples/python/hosted-agents/bring-your-own/responses/background-agent/requirements.txt index 8a76697f8..e971f98c1 100644 --- a/samples/python/hosted-agents/bring-your-own/responses/background-agent/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/responses/background-agent/requirements.txt @@ -1,3 +1,6 @@ azure-ai-agentserver-responses==1.0.0b7 azure-ai-projects==2.0.1 azure-identity==1.25.3 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/responses/bring-your-own-toolbox/requirements.txt b/samples/python/hosted-agents/bring-your-own/responses/bring-your-own-toolbox/requirements.txt index 211c28b29..a5afb20ba 100644 --- a/samples/python/hosted-agents/bring-your-own/responses/bring-your-own-toolbox/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/responses/bring-your-own-toolbox/requirements.txt @@ -3,3 +3,6 @@ azure-ai-projects>=2.0.0 azure-identity>=1.25.0 httpx python-dotenv==1.1.1 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/responses/browser-automation/requirements.txt b/samples/python/hosted-agents/bring-your-own/responses/browser-automation/requirements.txt index 211c28b29..a5afb20ba 100644 --- a/samples/python/hosted-agents/bring-your-own/responses/browser-automation/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/responses/browser-automation/requirements.txt @@ -3,3 +3,6 @@ azure-ai-projects>=2.0.0 azure-identity>=1.25.0 httpx python-dotenv==1.1.1 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/responses/env-vars-agent/requirements.txt b/samples/python/hosted-agents/bring-your-own/responses/env-vars-agent/requirements.txt index 2e0e0045b..47c62eb11 100755 --- a/samples/python/hosted-agents/bring-your-own/responses/env-vars-agent/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/responses/env-vars-agent/requirements.txt @@ -1,3 +1,6 @@ azure-ai-agentserver-responses==1.0.0b7 azure-ai-projects==2.0.1 azure-identity==1.25.3 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/responses/hello-world/requirements.txt b/samples/python/hosted-agents/bring-your-own/responses/hello-world/requirements.txt index 8a76697f8..e971f98c1 100644 --- a/samples/python/hosted-agents/bring-your-own/responses/hello-world/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/responses/hello-world/requirements.txt @@ -1,3 +1,6 @@ azure-ai-agentserver-responses==1.0.0b7 azure-ai-projects==2.0.1 azure-identity==1.25.3 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/responses/langgraph-chat/requirements.txt b/samples/python/hosted-agents/bring-your-own/responses/langgraph-chat/requirements.txt index fdff33be3..2e70701cd 100644 --- a/samples/python/hosted-agents/bring-your-own/responses/langgraph-chat/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/responses/langgraph-chat/requirements.txt @@ -4,3 +4,6 @@ langgraph==1.1.8 langgraph-prebuilt==1.0.10 langchain-core==1.3.0 langchain-azure-ai[opentelemetry]>=1.2.3 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/responses/langgraph-toolbox-user-identity/requirements.txt b/samples/python/hosted-agents/bring-your-own/responses/langgraph-toolbox-user-identity/requirements.txt index 25f17ea88..3815c3eac 100644 --- a/samples/python/hosted-agents/bring-your-own/responses/langgraph-toolbox-user-identity/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/responses/langgraph-toolbox-user-identity/requirements.txt @@ -9,3 +9,6 @@ python-dotenv==1.1.1 starlette<1.0.0 aiohttp pydantic-core<2.46.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/responses/langgraph-toolbox/requirements.txt b/samples/python/hosted-agents/bring-your-own/responses/langgraph-toolbox/requirements.txt index 25f17ea88..3815c3eac 100644 --- a/samples/python/hosted-agents/bring-your-own/responses/langgraph-toolbox/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/responses/langgraph-toolbox/requirements.txt @@ -9,3 +9,6 @@ python-dotenv==1.1.1 starlette<1.0.0 aiohttp pydantic-core<2.46.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/responses/notetaking-agent/requirements.txt b/samples/python/hosted-agents/bring-your-own/responses/notetaking-agent/requirements.txt index 8a76697f8..e971f98c1 100644 --- a/samples/python/hosted-agents/bring-your-own/responses/notetaking-agent/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/responses/notetaking-agent/requirements.txt @@ -1,3 +1,6 @@ azure-ai-agentserver-responses==1.0.0b7 azure-ai-projects==2.0.1 azure-identity==1.25.3 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/responses/openai-agents-sdk/requirements.txt b/samples/python/hosted-agents/bring-your-own/responses/openai-agents-sdk/requirements.txt index 0d82e5d5a..857b24744 100644 --- a/samples/python/hosted-agents/bring-your-own/responses/openai-agents-sdk/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/responses/openai-agents-sdk/requirements.txt @@ -1,3 +1,6 @@ azure-ai-agentserver-responses==1.0.0b7 azure-identity==1.25.3 openai-agents>=0.1.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/responses/optimization-customer-support/requirements.txt b/samples/python/hosted-agents/bring-your-own/responses/optimization-customer-support/requirements.txt index 5c6438c0f..bbbe87265 100644 --- a/samples/python/hosted-agents/bring-your-own/responses/optimization-customer-support/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/responses/optimization-customer-support/requirements.txt @@ -2,3 +2,6 @@ azure-ai-agentserver-responses==1.0.0b7 azure-ai-projects==2.0.1 azure-identity>=1.25.0 azure-ai-agentserver-optimization>=1.0.0b1 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/responses/optimization-hello-world/requirements.txt b/samples/python/hosted-agents/bring-your-own/responses/optimization-hello-world/requirements.txt index 5c6438c0f..bbbe87265 100644 --- a/samples/python/hosted-agents/bring-your-own/responses/optimization-hello-world/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/responses/optimization-hello-world/requirements.txt @@ -2,3 +2,6 @@ azure-ai-agentserver-responses==1.0.0b7 azure-ai-projects==2.0.1 azure-identity>=1.25.0 azure-ai-agentserver-optimization>=1.0.0b1 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/voicelive/foreground-background-agents-responses-voicelive/requirements.txt b/samples/python/hosted-agents/bring-your-own/voicelive/foreground-background-agents-responses-voicelive/requirements.txt index f1c60c0e4..f78c510e6 100644 --- a/samples/python/hosted-agents/bring-your-own/voicelive/foreground-background-agents-responses-voicelive/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/voicelive/foreground-background-agents-responses-voicelive/requirements.txt @@ -1,3 +1,6 @@ azure-ai-agentserver-responses==1.0.0b6 azure-ai-projects==2.0.1 azure-identity==1.25.3 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/voicelive/handoff-langgraph-responses-voicelive/requirements.txt b/samples/python/hosted-agents/bring-your-own/voicelive/handoff-langgraph-responses-voicelive/requirements.txt index e9d09e6d0..8089c6a6b 100644 --- a/samples/python/hosted-agents/bring-your-own/voicelive/handoff-langgraph-responses-voicelive/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/voicelive/handoff-langgraph-responses-voicelive/requirements.txt @@ -5,3 +5,6 @@ langchain-openai>=0.3.0 langchain-core>=0.3.0 httpx[socks]>=0.28.1 python-dotenv>=1.0.0 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/voicelive/hello-world-invocations-voicelive/requirements.txt b/samples/python/hosted-agents/bring-your-own/voicelive/hello-world-invocations-voicelive/requirements.txt index a157cbcf2..516a8ba36 100644 --- a/samples/python/hosted-agents/bring-your-own/voicelive/hello-world-invocations-voicelive/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/voicelive/hello-world-invocations-voicelive/requirements.txt @@ -1,3 +1,6 @@ azure-ai-agentserver-invocations==1.0.0b4 azure-ai-projects==2.0.1 azure-identity==1.25.3 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/bring-your-own/voicelive/hotel-booking-invocations-voicelive/requirements.txt b/samples/python/hosted-agents/bring-your-own/voicelive/hotel-booking-invocations-voicelive/requirements.txt index 457989a49..3a961cbf8 100644 --- a/samples/python/hosted-agents/bring-your-own/voicelive/hotel-booking-invocations-voicelive/requirements.txt +++ b/samples/python/hosted-agents/bring-your-own/voicelive/hotel-booking-invocations-voicelive/requirements.txt @@ -1 +1,4 @@ azure-ai-agentserver-invocations==1.0.0b4 + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/langgraph/a2a/a2a-caller/requirements.txt b/samples/python/hosted-agents/langgraph/a2a/a2a-caller/requirements.txt index 03fa03f85..1f93f92bd 100644 --- a/samples/python/hosted-agents/langgraph/a2a/a2a-caller/requirements.txt +++ b/samples/python/hosted-agents/langgraph/a2a/a2a-caller/requirements.txt @@ -3,3 +3,6 @@ langchain langchain-mcp-adapters httpx python-dotenv + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/langgraph/a2a/a2a-executor/requirements.txt b/samples/python/hosted-agents/langgraph/a2a/a2a-executor/requirements.txt index 8220fa0f3..31bb7529c 100644 --- a/samples/python/hosted-agents/langgraph/a2a/a2a-executor/requirements.txt +++ b/samples/python/hosted-agents/langgraph/a2a/a2a-executor/requirements.txt @@ -1,3 +1,6 @@ langchain-azure-ai[hosting]>=1.2.4 langchain python-dotenv + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/langgraph/invocations/01-langgraph-chat/requirements.txt b/samples/python/hosted-agents/langgraph/invocations/01-langgraph-chat/requirements.txt index b5a161d6c..d83085e79 100644 --- a/samples/python/hosted-agents/langgraph/invocations/01-langgraph-chat/requirements.txt +++ b/samples/python/hosted-agents/langgraph/invocations/01-langgraph-chat/requirements.txt @@ -3,3 +3,6 @@ azure-ai-projects langchain langchain-openai python-dotenv + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/langgraph/responses/01-langgraph-chat/requirements.txt b/samples/python/hosted-agents/langgraph/responses/01-langgraph-chat/requirements.txt index b5a161d6c..d83085e79 100644 --- a/samples/python/hosted-agents/langgraph/responses/01-langgraph-chat/requirements.txt +++ b/samples/python/hosted-agents/langgraph/responses/01-langgraph-chat/requirements.txt @@ -3,3 +3,6 @@ azure-ai-projects langchain langchain-openai python-dotenv + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/langgraph/responses/02-langgraph-toolbox/requirements.txt b/samples/python/hosted-agents/langgraph/responses/02-langgraph-toolbox/requirements.txt index 2f322170f..812c22427 100644 --- a/samples/python/hosted-agents/langgraph/responses/02-langgraph-toolbox/requirements.txt +++ b/samples/python/hosted-agents/langgraph/responses/02-langgraph-toolbox/requirements.txt @@ -4,3 +4,6 @@ langchain langchain-openai langchain-mcp-adapters python-dotenv + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/langgraph/responses/04-mcp/requirements.txt b/samples/python/hosted-agents/langgraph/responses/04-mcp/requirements.txt index 2f322170f..812c22427 100644 --- a/samples/python/hosted-agents/langgraph/responses/04-mcp/requirements.txt +++ b/samples/python/hosted-agents/langgraph/responses/04-mcp/requirements.txt @@ -4,3 +4,6 @@ langchain langchain-openai langchain-mcp-adapters python-dotenv + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/langgraph/responses/05-workflows/requirements.txt b/samples/python/hosted-agents/langgraph/responses/05-workflows/requirements.txt index ba5000516..e55cc80a4 100644 --- a/samples/python/hosted-agents/langgraph/responses/05-workflows/requirements.txt +++ b/samples/python/hosted-agents/langgraph/responses/05-workflows/requirements.txt @@ -4,3 +4,6 @@ langchain langchain-openai langgraph python-dotenv + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/langgraph/responses/06-files/requirements.txt b/samples/python/hosted-agents/langgraph/responses/06-files/requirements.txt index 2f322170f..812c22427 100644 --- a/samples/python/hosted-agents/langgraph/responses/06-files/requirements.txt +++ b/samples/python/hosted-agents/langgraph/responses/06-files/requirements.txt @@ -4,3 +4,6 @@ langchain langchain-openai langchain-mcp-adapters python-dotenv + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/langgraph/responses/07-human-in-the-loop/requirements.txt b/samples/python/hosted-agents/langgraph/responses/07-human-in-the-loop/requirements.txt index b5a161d6c..d83085e79 100644 --- a/samples/python/hosted-agents/langgraph/responses/07-human-in-the-loop/requirements.txt +++ b/samples/python/hosted-agents/langgraph/responses/07-human-in-the-loop/requirements.txt @@ -3,3 +3,6 @@ azure-ai-projects langchain langchain-openai python-dotenv + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy diff --git a/samples/python/hosted-agents/langgraph/responses/08-observability/requirements.txt b/samples/python/hosted-agents/langgraph/responses/08-observability/requirements.txt index 9ec894674..e522c7a4c 100644 --- a/samples/python/hosted-agents/langgraph/responses/08-observability/requirements.txt +++ b/samples/python/hosted-agents/langgraph/responses/08-observability/requirements.txt @@ -3,3 +3,6 @@ azure-ai-projects langchain langchain-openai python-dotenv + +# debugpy enables local debugging of this agent with the Foundry Toolkit VS Code extension. +debugpy From f2642f98d11e198997a0e3833368d02c22a09a2a Mon Sep 17 00:00:00 2001 From: Mohit Pavan Kumar Gadamsetty <112494281+mohitpavan@users.noreply.github.com> Date: Wed, 24 Jun 2026 11:54:58 +0530 Subject: [PATCH 4/4] Adding csharp BYO agent. (#573) * Adding BYO agent. * Prompt updates. * remove. * Some changes. * Fixing form filler prompt to base prompt. * PR coomments and a bug fix. --- .../browser-automation/prompts/base.md | 1 + .../browser-automation/utils/Middlewares.cs | 3 +- .../browser-automation/.dockerignore | 34 ++ .../responses/browser-automation/.env.example | 14 + .../responses/browser-automation/Dockerfile | 23 + .../responses/browser-automation/Program.cs | 504 ++++++++++++++++++ .../browser-automation/agent.manifest.yaml | 58 ++ .../responses/browser-automation/agent.yaml | 24 + .../browser-automation.csproj | 30 ++ .../browser-automation/skills/form-filler.md | 30 ++ .../browser-automation/skills/web-scraper.md | 20 + .../utils/BrowserSession.cs | 123 +++++ .../browser-automation/utils/Constants.cs | 138 +++++ .../browser-automation/utils/Redaction.cs | 21 + .../browser-automation/utils/Skills.cs | 43 ++ .../browser-automation/utils/ToolboxClient.cs | 171 ++++++ .../prompts/base.md | 1 + .../utils/agent_factory.py | 1 + .../responses/browser-automation/main.py | 37 +- .../browser-automation/utils/constants.py | 7 + 20 files changed, 1267 insertions(+), 16 deletions(-) create mode 100644 samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/.dockerignore create mode 100644 samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/.env.example create mode 100644 samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/Dockerfile create mode 100644 samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/Program.cs create mode 100644 samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/agent.manifest.yaml create mode 100644 samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/agent.yaml create mode 100644 samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/browser-automation.csproj create mode 100644 samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/skills/form-filler.md create mode 100644 samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/skills/web-scraper.md create mode 100644 samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/BrowserSession.cs create mode 100644 samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/Constants.cs create mode 100644 samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/Redaction.cs create mode 100644 samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/Skills.cs create mode 100644 samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/ToolboxClient.cs diff --git a/samples/csharp/hosted-agents/agent-framework/browser-automation/prompts/base.md b/samples/csharp/hosted-agents/agent-framework/browser-automation/prompts/base.md index 569078267..e6ea03577 100644 --- a/samples/csharp/hosted-agents/agent-framework/browser-automation/prompts/base.md +++ b/samples/csharp/hosted-agents/agent-framework/browser-automation/prompts/base.md @@ -99,6 +99,7 @@ live browser. ## General browser work +- **NEVER ASK FOR CONFIRMATION.** Execute tasks fully without pausing. Do NOT say "If you want, I can continue" or "Should I proceed?" — JUST DO IT. Complete the entire task end-to-end. - Clarify only when the goal, target URL, or required data is ambiguous. - Prefer inspecting page state before interacting with elements. - Use screenshots when visual confirmation would help the user understand the diff --git a/samples/csharp/hosted-agents/agent-framework/browser-automation/utils/Middlewares.cs b/samples/csharp/hosted-agents/agent-framework/browser-automation/utils/Middlewares.cs index 6740491d2..df88efeab 100644 --- a/samples/csharp/hosted-agents/agent-framework/browser-automation/utils/Middlewares.cs +++ b/samples/csharp/hosted-agents/agent-framework/browser-automation/utils/Middlewares.cs @@ -171,7 +171,8 @@ public static async IAsyncEnumerable LiveViewUrlStreamingMi _logger.LogDebug("[streaming-middleware] Appending live_view_url"); yield return new AgentResponseUpdate( ChatRole.Assistant, - [new TextContent($"\n\n🔴 [Browser Live View]({liveViewUrl})")]); + [new TextContent($"\n\n🔴 [Browser Live View]({liveViewUrl})")]) + { FinishReason = ChatFinishReason.Stop }; } } } diff --git a/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/.dockerignore b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/.dockerignore new file mode 100644 index 000000000..6f155354d --- /dev/null +++ b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/.dockerignore @@ -0,0 +1,34 @@ +# Build artifacts +**/bin/ +**/obj/ + +# .NET user secrets +**/.secrets/ + +# Local settings +**/*.user +**/*.suo + +# IDE settings +.vs/ +.vscode/ +**/.idea/ + +# Test results +**/TestResults/ + +# Version control +.git/ +.gitignore + +# Docker files +.dockerignore + +# Docs +README.md + +# Local environment (never bake credentials into the image) +.env + +# NuGet +.nuget/ diff --git a/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/.env.example b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/.env.example new file mode 100644 index 000000000..635957bd8 --- /dev/null +++ b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/.env.example @@ -0,0 +1,14 @@ +# Required +# FOUNDRY_PROJECT_ENDPOINT=https://.services.ai.azure.com/api/projects/ +AZURE_AI_MODEL_DEPLOYMENT_NAME=gpt-4.1 + +# Foundry Toolbox name (hardcoded in agent.manifest.yaml; override only if using a different toolbox) +TOOLBOX_NAME=browser-automation-tools + +# Playwright workspace (used in agent.manifest.yaml for connection resource) +# PLAYWRIGHT_SERVICE_URL=wss://.api.playwright.microsoft.com/playwrightworkspaces//browsers +# PLAYWRIGHT_SERVICE_RESOURCE_ID=/subscriptions//resourceGroups//providers/Microsoft.LoadTestService/playwrightWorkspaces/ +# PLAYWRIGHT_SERVICE_ACCESS_TOKEN= + +# Optional +# BROWSER_TIMEOUT_SECONDS=180 diff --git a/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/Dockerfile b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/Dockerfile new file mode 100644 index 000000000..9d2f4a49a --- /dev/null +++ b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/Dockerfile @@ -0,0 +1,23 @@ +FROM mcr.microsoft.com/dotnet/aspnet:10.0 AS base +WORKDIR /app +EXPOSE 8088 +ENV ASPNETCORE_URLS=http://+:8088 + +# Install Node.js and @playwright/cli +RUN apt-get update && apt-get install -y curl && \ + curl -fsSL https://deb.nodesource.com/setup_22.x | bash - && \ + apt-get install -y nodejs && \ + npm install -g @playwright/cli@latest && \ + playwright-cli install --skills && \ + apt-get clean && rm -rf /var/lib/apt/lists/* + +FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build +WORKDIR /src +COPY . . +RUN dotnet restore +RUN dotnet publish -c Release -o /app/publish + +FROM base AS final +WORKDIR /app +COPY --from=build /app/publish . +ENTRYPOINT ["dotnet", "browser-automation.dll"] diff --git a/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/Program.cs b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/Program.cs new file mode 100644 index 000000000..7bbc313d7 --- /dev/null +++ b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/Program.cs @@ -0,0 +1,504 @@ +// Copyright (c) Microsoft. All rights reserved. + +/* + * Browser Automation — Bring Your Own Responses agent with Playwright via Toolbox (C#) + * + * Hosted agent that automates browser interactions using playwright-cli via + * Foundry Toolbox MCP. Supports multiple concurrent browser sessions with + * lazy initialization — sessions are only created when the model first needs + * to interact with a browser. + * + * Required environment variables: + * FOUNDRY_PROJECT_ENDPOINT — Foundry project endpoint (auto-injected) + * AZURE_AI_MODEL_DEPLOYMENT_NAME — Model deployment name + */ + +using System.Runtime.CompilerServices; +using System.Text.Json; +using Azure.AI.AgentServer.Responses; +using Azure.AI.AgentServer.Responses.Models; +using Azure.AI.Extensions.OpenAI; +using Azure.AI.Projects; +using Azure.Identity; +using Microsoft.Extensions.DependencyInjection; +using OpenAI.Responses; + +namespace BrowserAutomation; + +public class Program +{ + public static void Main(string[] args) + { + if (string.IsNullOrEmpty(Environment.GetEnvironmentVariable("APPLICATIONINSIGHTS_CONNECTION_STRING"))) + Console.Error.WriteLine( + "[WARNING] APPLICATIONINSIGHTS_CONNECTION_STRING not set — traces will not be sent " + + "to Application Insights. Set it to enable local telemetry. " + + "(This variable is auto-injected in hosted Foundry containers.)"); + + var foundryEndpoint = Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT") + ?? throw new InvalidOperationException("FOUNDRY_PROJECT_ENDPOINT environment variable is not set."); + var deployment = Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYMENT_NAME") + ?? throw new InvalidOperationException("AZURE_AI_MODEL_DEPLOYMENT_NAME environment variable is not set."); + + var credential = new DefaultAzureCredential(); + var projectClient = new AIProjectClient(new Uri(foundryEndpoint), credential); + var responsesClient = projectClient.ProjectOpenAIClient + .GetProjectResponsesClientForModel(deployment); + + // Toolbox MCP endpoint + var toolboxName = Environment.GetEnvironmentVariable("TOOLBOX_NAME"); + if (string.IsNullOrWhiteSpace(toolboxName)) + toolboxName = Constants.DefaultToolboxName; + var toolboxEndpoint = $"{foundryEndpoint.TrimEnd('/')}/toolboxes/{toolboxName}/mcp?api-version=v1"; + + ResponsesServer.Run(configure: builder => + { + builder.Services.AddSingleton(responsesClient); + builder.Services.AddSingleton(new ToolboxConfig(toolboxEndpoint, credential)); + }); + } +} + +/// Configuration for ToolboxClient creation per-request. +public record ToolboxConfig(string Endpoint, DefaultAzureCredential Credential); + +/// Browser session state. +public record SessionState(BrowserSession Browser, string? LiveViewUrl); + +// ────────────────────────────────────────────────────────────────── +// Handler +// ────────────────────────────────────────────────────────────────── + +public class BrowserAutomationHandler : ResponseHandler +{ + private readonly ProjectResponsesClient _responsesClient; + private readonly ToolboxConfig _toolboxConfig; + private readonly ILogger _logger; + + // Multi-session state — static so it persists across requests (matches Python module-level globals) + private static readonly Dictionary _sessions = new(); + private static readonly HashSet _usedSessions = new(); // sessions touched in current request + private static string? _lastSession; + private static ToolboxClient? _toolbox; + private readonly int _browserTimeout; + + public BrowserAutomationHandler( + ProjectResponsesClient responsesClient, + ToolboxConfig toolboxConfig, + ILogger logger) + { + _responsesClient = responsesClient; + _toolboxConfig = toolboxConfig; + _logger = logger; + _browserTimeout = int.TryParse(Environment.GetEnvironmentVariable("BROWSER_TIMEOUT_SECONDS"), out var t) ? t : 180; + } + + private ToolboxClient GetToolbox() + { + _toolbox ??= new ToolboxClient( + _toolboxConfig.Endpoint, + () => + { + var ctx = new Azure.Core.TokenRequestContext(new[] { Constants.AzureAiScope }); + return _toolboxConfig.Credential.GetToken(ctx, default).Token; + }, + _logger); + return _toolbox; + } + + public override IAsyncEnumerable CreateAsync( + CreateResponse request, + ResponseContext context, + CancellationToken cancellationToken) + { + return new TextResponse(context, request, + createTextStream: ct => ProcessAsync(request, context, ct)); + } + + private async IAsyncEnumerable ProcessAsync( + CreateResponse request, + ResponseContext context, + [EnumeratorCancellation] CancellationToken cancellationToken) + { + var userInput = await context.GetInputTextAsync(cancellationToken: cancellationToken) ?? ""; + if (string.IsNullOrWhiteSpace(userInput)) + { + yield return "No input provided."; + yield break; + } + + // Check for /verbose flag + var verboseMode = userInput.TrimStart().StartsWith("/verbose"); + if (verboseMode) + userInput = userInput.Replace("/verbose", "", StringComparison.OrdinalIgnoreCase).TrimStart(); + + _usedSessions.Clear(); + + // Build input items with history + var inputItems = new List(); + try + { + var history = await context.GetHistoryAsync(cancellationToken); + foreach (var item in history) + { + if (item is OutputItemMessage { Content: { } contents }) + { + foreach (var content in contents) + { + switch (content) + { + case MessageContentOutputTextContent { Text: { } assistantText }: + inputItems.Add(ResponseItem.CreateAssistantMessageItem(assistantText)); + break; + case MessageContentInputTextContent { Text: { } userText }: + inputItems.Add(ResponseItem.CreateUserMessageItem(userText)); + break; + } + } + } + } + } + catch (Exception ex) + { + _logger.LogWarning("get_history failed; continuing without: {Error}", ex.Message); + } + + if (_sessions.Count > 0) + { + inputItems.Add(ResponseItem.CreateUserMessageItem( + $"[Active sessions: [{string.Join(", ", _sessions.Keys)}], default: {_lastSession}]")); + } + inputItems.Add(ResponseItem.CreateUserMessageItem(userInput)); + + var systemPrompt = Constants.GetSystemPrompt(Skills.ListSkills()); + var tools = BuildTools(); + + var verboseText = new System.Text.StringBuilder(); + string? finalReply = null; + + // Agentic loop — runs until model produces a final response (no tool calls) + while (true) + { + if (cancellationToken.IsCancellationRequested) + { + yield return "⚠️ Cancelled.\n"; + yield break; + } + + var options = new CreateResponseOptions { Instructions = systemPrompt }; + foreach (var tool in tools) + options.Tools.Add(tool); + foreach (var item in inputItems) + options.InputItems.Add(item); + + var result = await _responsesClient.CreateResponseAsync(options, cancellationToken); + + var functionCalls = result.Value.OutputItems + .OfType() + .ToList(); + + if (functionCalls.Count == 0) + { + finalReply = result.Value.GetOutputText() ?? "(No response)"; + // Add output items to input for context + foreach (var item in result.Value.OutputItems) + inputItems.Add(item); + break; + } + + // Process tool calls + foreach (var fc in functionCalls) + { + var sessionsBefore = new HashSet(_sessions.Keys); + + var toolResult = await HandleToolCallAsync(fc.FunctionName, fc.FunctionArguments); + + // Detect new sessions (covers both explicit create_session and lazy creation via run_browser/run_parallel) + var newSessions = _sessions.Keys.Except(sessionsBefore); + foreach (var newSess in newSessions) + { + var url = _sessions[newSess].LiveViewUrl; + var sessionLog = !string.IsNullOrEmpty(url) + ? $"🌐 Created **{newSess}** → [Live View]({url})\n" + : $"🌐 Created **{newSess}** (session ready)\n"; + yield return sessionLog; + verboseText.Append(sessionLog); + } + + // Format verbose log + var log = FormatToolLog(fc.FunctionName, fc.FunctionArguments, toolResult); + if (log != null) + { + if (verboseMode) + { + yield return log.Value.Text; + verboseText.Append(log.Value.Text); + } + else + { + // Heartbeat: emit empty string to keep SSE alive + yield return ""; + } + } + + inputItems.Add(ResponseItem.CreateFunctionCallItem(fc.CallId, fc.FunctionName, fc.FunctionArguments)); + inputItems.Add(ResponseItem.CreateFunctionCallOutputItem(fc.CallId, toolResult)); + } + } + + if (verboseText.Length > 0) + yield return "\n---\n\n"; + + yield return finalReply; + + // Append active session live view links at the end for easy access + if (_sessions.Count > 0) + { + var links = _sessions.Select(kv => + { + var url = kv.Value.LiveViewUrl; + return !string.IsNullOrEmpty(url) + ? $"- **{kv.Key}**: [Live View]({url})" + : $"- **{kv.Key}**: (no live view)"; + }); + var footer = $"\n\n---\n**Active Sessions:**\n{string.Join("\n", links)}"; + if (_usedSessions.Count > 0) + footer += $"\n\n_Used Sessions in this response: {string.Join(", ", _usedSessions)}_"; + yield return footer + "\n"; + } + } + + // ── Tool execution ─────────────────────────────────────────────── + + private async Task HandleToolCallAsync(string functionName, BinaryData arguments) + { + try + { + var args = JsonSerializer.Deserialize(arguments); + + return functionName switch + { + "load_skill" => HandleLoadSkill(args), + "create_session" => await HandleCreateSessionAsync(args), + "end_session" => await HandleEndSessionAsync(args), + "run_browser" => await HandleRunBrowserAsync(args), + "run_parallel" => await HandleRunParallelAsync(args), + "list_sessions" => HandleListSessions(), + _ => JsonSerializer.Serialize(new { error = $"Unknown tool: {functionName}" }), + }; + } + catch (Exception ex) + { + _logger.LogError(ex, "Tool '{Name}' failed", functionName); + return JsonSerializer.Serialize(new { error = ex.Message }); + } + } + + private string HandleLoadSkill(JsonElement args) + { + var name = args.TryGetProperty("name", out var n) ? n.GetString() ?? "" : ""; + var content = Skills.LoadSkill(name); + if (content == null) + return JsonSerializer.Serialize(new { error = $"Skill '{name}' not found. Available: {string.Join(", ", Skills.ListSkills())}" }); + return JsonSerializer.Serialize(new { skill = name, instructions = content }); + } + + private async Task HandleCreateSessionAsync(JsonElement args) + { + var name = args.TryGetProperty("name", out var n) ? n.GetString() ?? $"session-{_sessions.Count + 1}" : $"session-{_sessions.Count + 1}"; + + if (_sessions.ContainsKey(name)) + return JsonSerializer.Serialize(new { status = "already_exists", session = name, live_view_url = _sessions[name].LiveViewUrl }); + + var toolbox = GetToolbox(); + var result = await toolbox.CallToolAsync("create_session"); + + var cdpUrl = result.TryGetProperty("cdp_url", out var cdp) ? cdp.GetString() : null; + var liveViewUrl = result.TryGetProperty("live_view_url", out var lv) ? lv.GetString() : null; + + if (string.IsNullOrEmpty(cdpUrl)) + return JsonSerializer.Serialize(new { error = "No CDP URL returned from Toolbox" }); + + var browser = new BrowserSession(name, _browserTimeout, _logger); + var (success, output) = await browser.ConnectAsync(cdpUrl); + if (!success) + return JsonSerializer.Serialize(new { error = $"Browser connect failed: {output}" }); + + _sessions[name] = new SessionState(browser, liveViewUrl); + _lastSession = name; + _logger.LogInformation("Session '{Name}' created (live_view: {HasUrl})", name, !string.IsNullOrEmpty(liveViewUrl)); + + return JsonSerializer.Serialize(new { status = "created", session = name, live_view_url = liveViewUrl }); + } + + private async Task HandleEndSessionAsync(JsonElement args) + { + var name = args.TryGetProperty("name", out var n) ? n.GetString() ?? "" : ""; + + if (name == "all") + { + var ended = _sessions.Keys.ToList(); + foreach (var sn in ended) + { + try { await _sessions[sn].Browser.CloseAsync(); } catch { } + } + _sessions.Clear(); + _lastSession = null; + return JsonSerializer.Serialize(new { status = "ended_all", sessions = ended }); + } + + if (!_sessions.ContainsKey(name)) + return JsonSerializer.Serialize(new { error = $"Session '{name}' not found. Available: [{string.Join(", ", _sessions.Keys)}]" }); + + try { await _sessions[name].Browser.CloseAsync(); } catch { } + _sessions.Remove(name); + if (_lastSession == name) + _lastSession = _sessions.Keys.FirstOrDefault(); + return JsonSerializer.Serialize(new { status = "ended", session = name, remaining = _sessions.Keys.ToList() }); + } + + private async Task HandleRunBrowserAsync(JsonElement args) + { + var sessName = args.TryGetProperty("session", out var s) ? s.GetString() : null; + sessName ??= _lastSession; + + // Lazy session creation + if (_sessions.Count == 0) + { + var createResult = await HandleCreateSessionAsync( + JsonDocument.Parse("""{"name":"default"}""").RootElement); + if (createResult.Contains("error")) + return createResult; + sessName = "default"; + } + else if (string.IsNullOrEmpty(sessName) || !_sessions.ContainsKey(sessName)) + { + return JsonSerializer.Serialize(new { error = $"Session '{sessName}' not found. Available: [{string.Join(", ", _sessions.Keys)}]" }); + } + + var browser = _sessions[sessName].Browser; + var command = args.TryGetProperty("command", out var c) ? c.GetString() ?? "" : ""; + var cmdArgs = args.TryGetProperty("args", out var a) && a.ValueKind == JsonValueKind.Array + ? a.EnumerateArray().Select(x => x.GetString() ?? "").ToArray() + : Array.Empty(); + + _lastSession = sessName; + _usedSessions.Add(sessName); + var (success, output) = await browser.RunAsync(command, cmdArgs); + return JsonSerializer.Serialize(new { success, output }); + } + + private async Task HandleRunParallelAsync(JsonElement args) + { + if (!args.TryGetProperty("tasks", out var tasksEl) || tasksEl.ValueKind != JsonValueKind.Array) + return JsonSerializer.Serialize(new { error = "No tasks provided" }); + + // Ensure default session + if (_sessions.Count == 0) + { + var createResult = await HandleCreateSessionAsync( + JsonDocument.Parse("""{"name":"default"}""").RootElement); + if (createResult.Contains("error")) + return createResult; + } + + var tasks = new List>(); + foreach (var task in tasksEl.EnumerateArray()) + { + tasks.Add(RunOneParallelTask(task)); + } + + var results = await Task.WhenAll(tasks); + return JsonSerializer.Serialize(new { parallel_results = results }); + } + + private async Task RunOneParallelTask(JsonElement task) + { + var sess = task.TryGetProperty("session", out var s) ? s.GetString() : null; + sess ??= _lastSession ?? "default"; + + if (!_sessions.ContainsKey(sess)) + return new { session = sess, error = $"Session '{sess}' not found" }; + + var browser = _sessions[sess].Browser; + var command = task.TryGetProperty("command", out var c) ? c.GetString() ?? "" : ""; + var cmdArgs = task.TryGetProperty("args", out var a) && a.ValueKind == JsonValueKind.Array + ? a.EnumerateArray().Select(x => x.GetString() ?? "").ToArray() + : Array.Empty(); + + _usedSessions.Add(sess); + var (success, output) = await browser.RunAsync(command, cmdArgs); + return new { session = sess, command, success, output }; + } + + private string HandleListSessions() + { + var info = new Dictionary(); + foreach (var (name, state) in _sessions) + { + info[name] = new { live_view_url = state.LiveViewUrl, connected = state.Browser.Connected }; + } + return JsonSerializer.Serialize(new { sessions = info, @default = _lastSession }); + } + + // ── Verbose logging ────────────────────────────────────────────── + + private (string Kind, string Text)? FormatToolLog(string name, BinaryData arguments, string resultText) + { + var args = JsonSerializer.Deserialize(arguments); + + return name switch + { + "end_session" => FormatEndSessionLog(args, resultText), + "run_browser" => FormatRunBrowserLog(args), + "run_parallel" => ("log", $"⚡ Running {(args.TryGetProperty("tasks", out var t) ? t.GetArrayLength() : 0)} tasks in parallel\n"), + "load_skill" => ("log", $"📖 Loading skill: {(args.TryGetProperty("name", out var n) ? n.GetString() : "?")}\n"), + "list_sessions" => ("log", "📋 Listed sessions\n"), + _ => null, + }; + } + + private (string Kind, string Text) FormatEndSessionLog(JsonElement args, string resultText) + { + var sessName = args.TryGetProperty("name", out var n) ? n.GetString() ?? "?" : "?"; + try + { + var result = JsonSerializer.Deserialize(resultText); + var status = result.TryGetProperty("status", out var s) ? s.GetString() : null; + if (status == "ended_all") return ("log", "🔴 Ended ALL sessions\n"); + if (status == "ended") return ("log", $"🔴 Ended **{sessName}**\n"); + } + catch { } + return ("log", $"🔴 End {sessName}\n"); + } + + private (string Kind, string Text) FormatRunBrowserLog(JsonElement args) + { + var sess = args.TryGetProperty("session", out var s) ? s.GetString() : _lastSession ?? "?"; + var cmd = args.TryGetProperty("command", out var c) ? c.GetString() ?? "" : ""; + var cmdArgs = args.TryGetProperty("args", out var a) && a.ValueKind == JsonValueKind.Array + ? a.EnumerateArray().Take(2).Select(x => Redaction.Redact(x.GetString() ?? "")).ToList() + : new List(); + return ("log", $"🔧 [{sess}] `{cmd} {string.Join(" ", cmdArgs)}`\n"); + } + + // ── Tool definitions ───────────────────────────────────────────── + + private static List BuildTools() + { + var defs = JsonSerializer.Deserialize(Constants.ToolDefinitions); + var tools = new List(); + foreach (var def in defs.EnumerateArray()) + { + var name = def.GetProperty("name").GetString()!; + var desc = def.GetProperty("description").GetString()!; + var parameters = def.GetProperty("parameters"); + tools.Add(ResponseTool.CreateFunctionTool( + functionName: name, + functionDescription: desc, + functionParameters: BinaryData.FromString(parameters.GetRawText()), + strictModeEnabled: false)); + } + return tools; + } +} diff --git a/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/agent.manifest.yaml b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/agent.manifest.yaml new file mode 100644 index 000000000..125be52c9 --- /dev/null +++ b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/agent.manifest.yaml @@ -0,0 +1,58 @@ +# yaml-language-server: $schema=https://raw.githubusercontent.com/microsoft/AgentSchema/refs/heads/main/schemas/v1.0/AgentManifest.yaml +name: browser-automation-csharp-byo +displayName: "Browser Automation Agent (C#, BYO Responses)" +description: > + A Foundry-hosted browser automation agent using the Responses protocol with + playwright-cli and Toolbox MCP. Supports multi-session browsing, web scraping, + and form filling. +metadata: + tags: + - AI Agent Hosting + - Responses Protocol + - Bring Your Own + - Browser Automation + - .NET +template: + name: browser-automation-csharp-byo + kind: hosted + protocols: + - protocol: responses + version: 1.0.0 + environment_variables: + # FOUNDRY_PROJECT_ENDPOINT and APPLICATIONINSIGHTS_CONNECTION_STRING + # are injected by the platform (hosted) and translated by azd (local) + # — do NOT declare them here. + - name: AZURE_AI_MODEL_DEPLOYMENT_NAME + value: "{{AZURE_AI_MODEL_DEPLOYMENT_NAME}}" + - name: TOOLBOX_NAME + value: "browser-automation-tools" +parameters: + PLAYWRIGHT_SERVICE_URL: + secret: false + description: Browser WebSocket endpoint for the Azure Playwright workspace. + PLAYWRIGHT_SERVICE_RESOURCE_ID: + secret: false + description: Azure resource ID of the Playwright workspace. + PLAYWRIGHT_SERVICE_ACCESS_TOKEN: + secret: true + description: Access token for the Azure Playwright workspace. +resources: + - kind: model + id: gpt-4.1 + name: AZURE_AI_MODEL_DEPLOYMENT_NAME + - kind: connection + name: browserautomation + authType: ApiKey + category: PlaywrightWorkspace + target: "{{PLAYWRIGHT_SERVICE_URL}}" + credentials: + key: "{{PLAYWRIGHT_SERVICE_ACCESS_TOKEN}}" + metadata: + resourceId: "{{PLAYWRIGHT_SERVICE_RESOURCE_ID}}" + - kind: toolbox + name: browser-automation-tools + tools: + - type: browser_automation_preview + browser_automation_preview: + connection: + project_connection_id: browserautomation diff --git a/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/agent.yaml b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/agent.yaml new file mode 100644 index 000000000..1e8cc7afe --- /dev/null +++ b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/agent.yaml @@ -0,0 +1,24 @@ +# yaml-language-server: $schema=https://raw.githubusercontent.com/microsoft/AgentSchema/refs/heads/main/schemas/v1.0/ContainerAgent.yaml + +kind: hosted +name: browser-automation-csharp-byo +description: | + A Foundry-hosted browser automation agent (C#, BYO Responses) for general browsing, web scraping, and form filling. +metadata: + tags: + - AI Agent Hosting + - Responses Protocol + - Bring Your Own + - Browser Automation + - .NET +protocols: + - protocol: responses + version: 1.0.0 +resources: + cpu: "0.25" + memory: 0.5Gi +environment_variables: + - name: AZURE_AI_MODEL_DEPLOYMENT_NAME + value: ${AZURE_AI_MODEL_DEPLOYMENT_NAME} + - name: TOOLBOX_NAME + value: ${TOOLBOX_NAME} diff --git a/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/browser-automation.csproj b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/browser-automation.csproj new file mode 100644 index 000000000..8c97fc933 --- /dev/null +++ b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/browser-automation.csproj @@ -0,0 +1,30 @@ + + + + net10.0 + enable + enable + + $(NoWarn);OPENAI001 + + + + + + + + + + + + + + + + + + + diff --git a/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/skills/form-filler.md b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/skills/form-filler.md new file mode 100644 index 000000000..f99144b8e --- /dev/null +++ b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/skills/form-filler.md @@ -0,0 +1,30 @@ +# Form Filler Skill + +You are filling out a web form. The browser is already connected. + +## Workflow + +1. **Snapshot** the page to see current form state. +2. **Identify** all visible form fields and their current values. +3. **Fill** each field using the appropriate command: + - Text inputs: `fill "value"` + - Dropdowns: `select "value"` + - Checkboxes: `check ` or `uncheck ` + - Radio buttons: `click ` +4. **Date pickers** — follow these steps carefully: + - Click the date input field to open the picker + - Snapshot to see the picker UI + - Navigate months if needed (click prev/next arrows) + - Click the specific day number + - Snapshot to confirm the date was set + - If the picker doesn't open, try `fill "YYYY-MM-DD"` directly +5. **After filling all fields**, ALWAYS click the Next/Continue/Submit button. +6. **Snapshot** after submission to verify success or see the next page. +7. **Repeat** for multi-page forms until fully complete. + +## Rules + +- Never skip required fields. +- If a field rejects input, try: click it first, use a different format, or clear and re-fill. +- Always snapshot between major actions to verify state. +- Do NOT stop mid-form. Complete the entire submission flow. diff --git a/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/skills/web-scraper.md b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/skills/web-scraper.md new file mode 100644 index 000000000..e363a2827 --- /dev/null +++ b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/skills/web-scraper.md @@ -0,0 +1,20 @@ +# Web Scraper Skill + +You are scraping data from a web page. The browser is already connected. + +## Workflow + +1. **Navigate** to the target URL with `goto`. +2. **Snapshot** to see the page structure. +3. **Extract** data using `eval` with JavaScript: + - `eval "document.querySelector('.price').textContent"` + - `eval "JSON.stringify([...document.querySelectorAll('tr')].map(r => r.textContent))"` +4. **Navigate** pagination if needed (click Next, snapshot, repeat). +5. **Report** the extracted data clearly to the user. + +## Tips + +- Use `snapshot` to understand page structure before writing selectors. +- For tables, extract row by row or use `eval` to get all at once. +- If content is behind clicks (tabs, accordions), click to reveal first. +- For infinite scroll pages, use `scroll down` then snapshot to load more. diff --git a/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/BrowserSession.cs b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/BrowserSession.cs new file mode 100644 index 000000000..30b890a9c --- /dev/null +++ b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/BrowserSession.cs @@ -0,0 +1,123 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Diagnostics; + +namespace BrowserAutomation; + +/// +/// Manages a playwright-cli session against a remote CDP browser. +/// +public class BrowserSession +{ + public string SessionId { get; } + public bool Connected { get; private set; } + + private readonly int _timeoutSeconds; + private readonly ILogger? _logger; + + public BrowserSession(string sessionId, int timeoutSeconds = 180, ILogger? logger = null) + { + SessionId = sessionId; + _timeoutSeconds = timeoutSeconds; + _logger = logger; + } + + /// Attach to a remote browser via CDP URL. + public async Task<(bool Success, string Output)> ConnectAsync(string cdpUrl) + { + var result = await RunCommandAsync($"attach --cdp={cdpUrl}"); + Connected = result.Success; + return result; + } + + /// Run a playwright-cli command in this session. + public async Task<(bool Success, string Output)> RunAsync(string command, string[] args) + { + if (!Connected) + return (false, "Browser not connected. Session may need to be recreated."); + + var fullCmd = command; + if (args.Length > 0) + { + var quoted = args.Select(a => a.Contains(' ') ? $"\"{a}\"" : a); + fullCmd += " " + string.Join(" ", quoted); + } + return await RunCommandAsync(fullCmd); + } + + /// Detach from the browser session. + public async Task CloseAsync() + { + if (Connected) + await RunCommandAsync("detach"); + Connected = false; + } + + private async Task<(bool Success, string Output)> RunCommandAsync(string command) + { + var cli = FindCli(); + var processArgs = $"-s={SessionId} {command}"; + _logger?.LogInformation("[pw-cli] {Cli} -s={SessionId} {Command}", cli, SessionId, Redaction.Redact(command)); + + ProcessStartInfo psi = new() + { + FileName = cli, + Arguments = processArgs, + RedirectStandardOutput = true, + RedirectStandardError = true, + UseShellExecute = false, + CreateNoWindow = true, + }; + + try + { + using var process = Process.Start(psi) + ?? throw new InvalidOperationException("Failed to start playwright-cli"); + + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(_timeoutSeconds)); + var stdoutTask = process.StandardOutput.ReadToEndAsync(cts.Token); + var stderrTask = process.StandardError.ReadToEndAsync(cts.Token); + + try + { + await process.WaitForExitAsync(cts.Token); + } + catch (OperationCanceledException) + { + process.Kill(entireProcessTree: true); + return (false, $"Command timed out after {_timeoutSeconds} seconds."); + } + + var stdout = Redaction.Redact(Truncate(await stdoutTask)); + var stderr = Redaction.Redact(Truncate(await stderrTask)); + var success = process.ExitCode == 0; + + var output = $"exit_code: {process.ExitCode}\nstdout:\n{(string.IsNullOrEmpty(stdout) ? "" : stdout)}"; + if (!string.IsNullOrEmpty(stderr)) + output += $"\n\nstderr:\n{stderr}"; + + return (success, output); + } + catch (Exception ex) + { + _logger?.LogError(ex, "[pw-cli] Failed to run command"); + return (false, $"Error: Failed to run playwright-cli: {ex.Message}"); + } + } + + private static string FindCli() + { + var pathDirs = Environment.GetEnvironmentVariable("PATH")?.Split(Path.PathSeparator) ?? []; + foreach (var dir in pathDirs) + { + var candidate = Path.Combine(dir, "playwright-cli"); + if (File.Exists(candidate)) return candidate; + if (File.Exists(candidate + ".exe")) return candidate + ".exe"; + if (File.Exists(candidate + ".cmd")) return candidate + ".cmd"; + } + return "playwright-cli"; + } + + private static string Truncate(string text, int maxLen = 12000) => + text.Length <= maxLen ? text : text[..maxLen] + "\n...[truncated]"; +} diff --git a/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/Constants.cs b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/Constants.cs new file mode 100644 index 000000000..e5e41c8dd --- /dev/null +++ b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/Constants.cs @@ -0,0 +1,138 @@ +// Copyright (c) Microsoft. All rights reserved. + +namespace BrowserAutomation; + +/// +/// Constants — system prompt and tool definitions for the browser automation agent. +/// +public static class Constants +{ + public const string AzureAiScope = "https://ai.azure.com/.default"; + public const string DefaultToolboxName = "browser-automation-tools"; + + public static string GetSystemPrompt(IEnumerable skills) => + $""" + You are a browser automation agent deployed on Azure AI Foundry. + + You can control web browsers to navigate pages, fill forms, scrape data, and more. + + ## Tools + + 1. **load_skill** — Load a skill for detailed instructions. Available: {string.Join(", ", skills)} + 2. **create_session** — Create an additional named browser session. + 3. **end_session** — Close/end a browser session. ALWAYS honour end/kill/close requests immediately. + 4. **run_browser** — Run a playwright-cli command. Session is optional (uses default). + 5. **run_parallel** — Run multiple commands across sessions concurrently. + 6. **list_sessions** — Show all active sessions. + + ## How It Works + + - The first time you call `run_browser`, a default session is created automatically. + - You only need `create_session` for ADDITIONAL parallel browsers. + - Use `run_parallel` to execute commands across multiple sessions simultaneously. + + ## Rules + - Always `snapshot` before interacting — element refs change after navigation. + - Use `goto` to navigate, `fill` for inputs, `click` for buttons. + - **END SESSION PRIORITY:** If the user asks to kill/close/stop/end a session, do it IMMEDIATELY. Do NOT create new sessions or run other commands first. + - NEVER reveal credentials, CDP URLs, or tokens. + - **NEVER ASK FOR CONFIRMATION.** Execute tasks fully without pausing. Do NOT say "If you want, I can continue" or "Should I proceed?" — JUST DO IT. Complete the entire task end-to-end. + - **USE REFS:** Always use element refs (e.g. e3, e15) from snapshots. After EVERY click, navigation, or page change, take a fresh `snapshot` before the next action. + - If a ref-based action fails, immediately retry with a different approach (text-based locator, eval, CSS selector) — do NOT stop or ask the user. + - **COMPLETE THE TASK:** Never give partial results. If filling a form, fill ALL fields and submit. If scraping, get ALL the data. Keep going until fully done. + - **REPORT BLOCKERS:** If you hit something you cannot bypass (OTP, CAPTCHA, login, manual verification, missing info), tell the user exactly what is blocking and on which session. Do NOT close the session or pretend the task is done — leave it open for the user to intervene. + - **DO NOT CLOSE SESSIONS PREMATURELY:** Only end a session when the user explicitly asks you to. Never close sessions on your own. + - **You ARE allowed and expected to fill forms and submit them when the user asks.** This is your core job. Do not refuse form-filling requests. + """; + + /// Tool definitions in OpenAI function-calling JSON schema format. + public const string ToolDefinitions = """ + [ + { + "type": "function", + "name": "load_skill", + "description": "Load a skill for detailed instructions on a workflow.", + "parameters": { + "type": "object", + "properties": { + "name": { "type": "string", "description": "Skill name (e.g. 'form-filler', 'web-scraper')" } + }, + "required": ["name"], + "additionalProperties": false + } + }, + { + "type": "function", + "name": "create_session", + "description": "Create an additional named browser session for parallel work.", + "parameters": { + "type": "object", + "properties": { + "name": { "type": "string", "description": "Session name (e.g. 'form-browser', 'scraper')" } + }, + "required": ["name"], + "additionalProperties": false + } + }, + { + "type": "function", + "name": "end_session", + "description": "End/close a browser session immediately. Takes priority over all other actions.", + "parameters": { + "type": "object", + "properties": { + "name": { "type": "string", "description": "Session name to end, or 'all' to end all." } + }, + "required": ["name"], + "additionalProperties": false + } + }, + { + "type": "function", + "name": "run_browser", + "description": "Run a playwright-cli command in a browser session.", + "parameters": { + "type": "object", + "properties": { + "session": { "type": "string", "description": "Session name. Optional — uses default if omitted." }, + "command": { "type": "string", "description": "Command: goto, snapshot, click, fill, type, press, keys, select, scroll, eval, screenshot, hover, dblclick, check, uncheck, wait, tab-list, tab-new, tab-close, go-back, go-forward, reload" }, + "args": { "type": "array", "items": { "type": "string" }, "description": "Command arguments." } + }, + "required": ["command"], + "additionalProperties": false + } + }, + { + "type": "function", + "name": "run_parallel", + "description": "Run multiple browser commands across sessions concurrently.", + "parameters": { + "type": "object", + "properties": { + "tasks": { + "type": "array", + "description": "List of tasks. Each has session (optional), command, and args.", + "items": { + "type": "object", + "properties": { + "session": { "type": "string" }, + "command": { "type": "string" }, + "args": { "type": "array", "items": { "type": "string" } } + }, + "required": ["command"] + } + } + }, + "required": ["tasks"], + "additionalProperties": false + } + }, + { + "type": "function", + "name": "list_sessions", + "description": "List all active browser sessions.", + "parameters": { "type": "object", "properties": {}, "additionalProperties": false } + } + ] + """; +} diff --git a/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/Redaction.cs b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/Redaction.cs new file mode 100644 index 000000000..c09022b19 --- /dev/null +++ b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/Redaction.cs @@ -0,0 +1,21 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Text.RegularExpressions; + +namespace BrowserAutomation; + +/// +/// Redaction utilities for sensitive values in logs and tool output. +/// +public static class Redaction +{ + private static readonly Regex TokenPattern = new(@"\beyJ[a-zA-Z0-9._-]{20,}\b", RegexOptions.Compiled); + private static readonly Regex AccessKeyPattern = new(@"(accessKey=)[^&\s""']+", RegexOptions.Compiled | RegexOptions.IgnoreCase); + + public static string Redact(string text) + { + text = TokenPattern.Replace(text, ""); + text = AccessKeyPattern.Replace(text, "$1"); + return text; + } +} diff --git a/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/Skills.cs b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/Skills.cs new file mode 100644 index 000000000..c9845be29 --- /dev/null +++ b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/Skills.cs @@ -0,0 +1,43 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Reflection; + +namespace BrowserAutomation; + +/// +/// Skills manager — loads embedded markdown skill files for guided workflows. +/// +public static class Skills +{ + private static readonly Assembly Assembly = typeof(Skills).Assembly; + + /// Load a skill markdown file by name. + public static string? LoadSkill(string name) + { + // Sanitize name + var safeName = new string(name.Where(c => char.IsLetterOrDigit(c) || c == '-' || c == '_').ToArray()); + var resourceName = Assembly.GetManifestResourceNames() + .FirstOrDefault(r => r.EndsWith($".{safeName}.md", StringComparison.OrdinalIgnoreCase)); + + if (resourceName == null) return null; + + using var stream = Assembly.GetManifestResourceStream(resourceName); + if (stream == null) return null; + using var reader = new StreamReader(stream); + return reader.ReadToEnd(); + } + + /// List available skill names. + public static List ListSkills() + { + return Assembly.GetManifestResourceNames() + .Where(r => r.EndsWith(".md", StringComparison.OrdinalIgnoreCase)) + .Select(r => + { + // Resource names are like "browser_automation.skills.form-filler.md" + var parts = r.Split('.'); + return parts.Length >= 2 ? parts[^2] : r; + }) + .ToList(); + } +} diff --git a/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/ToolboxClient.cs b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/ToolboxClient.cs new file mode 100644 index 000000000..d6cabd4ec --- /dev/null +++ b/samples/csharp/hosted-agents/bring-your-own/responses/browser-automation/utils/ToolboxClient.cs @@ -0,0 +1,171 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Net.Http.Headers; +using System.Text.Json; + +namespace BrowserAutomation; + +/// +/// Lightweight MCP client for Foundry Toolbox browser session management. +/// Mirrors the Python ToolboxClient — initialize, discover tools, call tools. +/// +public class ToolboxClient +{ + private const string ToolboxFeatures = "Toolboxes=V1Preview"; + + private readonly string _endpoint; + private readonly Func _getToken; + private readonly ILogger _logger; + private string? _sessionId; + private int _reqId; + private bool _initialized; + private readonly Dictionary _toolNames = new(); // suffix -> full name + + public ToolboxClient(string endpoint, Func tokenProvider, ILogger logger) + { + _endpoint = endpoint; + _getToken = tokenProvider; + _logger = logger; + } + + /// Send MCP initialize + notification, then discover tools. + public async Task InitializeAsync() + { + if (_initialized) return "already-initialized"; + + using var client = new HttpClient { Timeout = TimeSpan.FromSeconds(60) }; + + // Initialize + var initResp = await PostAsync(client, new + { + jsonrpc = "2.0", + id = NextId(), + method = "initialize", + @params = new + { + protocolVersion = "2024-11-05", + capabilities = new { }, + clientInfo = new { name = "browser-automation-agent", version = "1.0.0" }, + }, + }); + + // Capture session ID + if (initResp.Headers.TryGetValues("mcp-session-id", out var sidValues)) + { + var sid = sidValues.FirstOrDefault(); + if (!string.IsNullOrEmpty(sid) && sid != "None") + _sessionId = sid; + } + + var initData = await ReadJsonAsync(initResp); + + // Notification + await PostAsync(client, new + { + jsonrpc = "2.0", + method = "notifications/initialized", + }); + + // Discover tools + var listResp = await PostAsync(client, new + { + jsonrpc = "2.0", + id = NextId(), + method = "tools/list", + @params = new { }, + }); + + var listData = await ReadJsonAsync(listResp); + var tools = listData.GetProperty("result").GetProperty("tools"); + foreach (var tool in tools.EnumerateArray()) + { + var name = tool.GetProperty("name").GetString() ?? ""; + var parts = name.Split("___", 2); + var suffix = parts.Length == 2 ? parts[1] : name; + _toolNames[suffix] = name; + } + + _logger.LogInformation("Toolbox tools discovered: {Tools}", string.Join(", ", _toolNames.Values)); + _initialized = true; + + var serverName = initData.GetProperty("result") + .GetProperty("serverInfo") + .GetProperty("name") + .GetString() ?? "unknown"; + return serverName; + } + + /// Call a Toolbox tool and return the parsed JSON result. + public async Task CallToolAsync(string name, Dictionary? arguments = null) + { + await InitializeAsync(); + + // Resolve suffix to full name + if (!name.Contains("___") && _toolNames.TryGetValue(name, out var fullName)) + name = fullName; + + using var client = new HttpClient { Timeout = TimeSpan.FromSeconds(120) }; + var resp = await PostAsync(client, new + { + jsonrpc = "2.0", + id = NextId(), + method = "tools/call", + @params = new { name, arguments = arguments ?? new Dictionary() }, + }); + + var data = await ReadJsonAsync(resp); + + if (data.TryGetProperty("error", out var error)) + { + var msg = error.TryGetProperty("message", out var m) ? m.GetString() : "Unknown MCP error"; + throw new InvalidOperationException($"Toolbox error: {msg}"); + } + + var result = data.GetProperty("result"); + if (result.TryGetProperty("content", out var content)) + { + foreach (var item in content.EnumerateArray()) + { + if (item.TryGetProperty("type", out var type) && type.GetString() == "text" + && item.TryGetProperty("text", out var text)) + { + try + { + return JsonDocument.Parse(text.GetString()!).RootElement; + } + catch (JsonException) + { + return item; + } + } + } + } + + return result; + } + + private async Task PostAsync(HttpClient client, object body) + { + var json = JsonSerializer.Serialize(body); + using var request = new HttpRequestMessage(HttpMethod.Post, _endpoint) + { + Content = new StringContent(json, System.Text.Encoding.UTF8, "application/json"), + }; + request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", _getToken()); + request.Headers.TryAddWithoutValidation("Foundry-Features", ToolboxFeatures); + if (_sessionId != null) + request.Headers.TryAddWithoutValidation("mcp-session-id", _sessionId); + + var resp = await client.SendAsync(request); + resp.EnsureSuccessStatusCode(); + return resp; + } + + private static async Task ReadJsonAsync(HttpResponseMessage resp) + { + var body = await resp.Content.ReadAsStringAsync(); + return JsonDocument.Parse(body).RootElement; + } + + private int NextId() => ++_reqId; +} diff --git a/samples/python/hosted-agents/agent-framework/responses/14-browser-automation-agent/prompts/base.md b/samples/python/hosted-agents/agent-framework/responses/14-browser-automation-agent/prompts/base.md index fde4253de..54d7ab9a2 100644 --- a/samples/python/hosted-agents/agent-framework/responses/14-browser-automation-agent/prompts/base.md +++ b/samples/python/hosted-agents/agent-framework/responses/14-browser-automation-agent/prompts/base.md @@ -97,6 +97,7 @@ live browser. ## General browser work +- **NEVER ASK FOR CONFIRMATION.** Execute tasks fully without pausing. Do NOT say "If you want, I can continue" or "Should I proceed?" — JUST DO IT. Complete the entire task end-to-end. - Clarify only when the goal, target URL, or required data is ambiguous. - Prefer inspecting page state before interacting with elements. - Use screenshots when visual confirmation would help the user understand the diff --git a/samples/python/hosted-agents/agent-framework/responses/14-browser-automation-agent/utils/agent_factory.py b/samples/python/hosted-agents/agent-framework/responses/14-browser-automation-agent/utils/agent_factory.py index 671ac4ebb..b16855cf2 100644 --- a/samples/python/hosted-agents/agent-framework/responses/14-browser-automation-agent/utils/agent_factory.py +++ b/samples/python/hosted-agents/agent-framework/responses/14-browser-automation-agent/utils/agent_factory.py @@ -97,6 +97,7 @@ async def _inject_url_stream() -> AsyncIterable[ChatResponseUpdate]: yield ChatResponseUpdate( contents=[Content.from_text(text=f"\n\n🔴 [Browser Live View]({_live_view_url})\n")], role="assistant", + finish_reason="stop", ) context.result = ResponseStream(_inject_url_stream(), finalizer=ChatResponse.from_updates) diff --git a/samples/python/hosted-agents/bring-your-own/responses/browser-automation/main.py b/samples/python/hosted-agents/bring-your-own/responses/browser-automation/main.py index 6726b7089..71de6d9c5 100644 --- a/samples/python/hosted-agents/bring-your-own/responses/browser-automation/main.py +++ b/samples/python/hosted-agents/bring-your-own/responses/browser-automation/main.py @@ -62,6 +62,7 @@ _sessions: dict[str, dict] = {} # name -> {"browser": BrowserSession, "live_view_url": str} _last_session: str | None = None +_used_sessions: set[str] = set() # sessions touched in current request async def _create_session(name: str) -> dict: @@ -152,6 +153,7 @@ async def _handle_tool_call(call) -> str: command = args.get("command", "") cmd_args = args.get("args") or [] _last_session = sess_name + _used_sessions.add(sess_name) result = await browser.run(command, cmd_args) elif name == "run_parallel": @@ -169,6 +171,7 @@ async def _run_one(task: dict) -> dict: sess = task.get("session") or _last_session or "default" if sess not in _sessions: return {"session": sess, "error": f"Session '{sess}' not found"} + _used_sessions.add(sess) browser = _sessions[sess]["browser"] r = await browser.run(task.get("command", ""), task.get("args") or []) return {"session": sess, "command": task.get("command", ""), **r} @@ -242,7 +245,7 @@ async def _run_agent_loop_streaming(input_items: list[dict]): result_text = await _handle_tool_call(tc) - # Detect lazy session creation (e.g. run_browser auto-creates default) + # Detect new sessions (covers both explicit create_session and lazy creation via run_browser/run_parallel) new_sessions = set(_sessions.keys()) - sessions_before for new_sess in new_sessions: url = _sessions[new_sess].get("live_view_url", "") @@ -272,19 +275,7 @@ async def _run_agent_loop_streaming(input_items: list[dict]): def _format_tool_log(name: str, args: dict, result_text: str) -> tuple[str, str] | None: """Format a verbose log entry for a tool call. Returns (kind, text) or None.""" - if name == "create_session": - sess_name = args.get("name", "?") - try: - result = json.loads(result_text) - url = result.get("live_view_url") or "" - if url: - return ("session", f"🌐 Created **{sess_name}** → [Live View]({url})\n") - else: - return ("session", f"🌐 Created **{sess_name}** (session ready, live view pending)\n") - except (json.JSONDecodeError, TypeError): - pass - return ("session", f"🌐 Created **{sess_name}**\n") - elif name == "end_session": + if name == "end_session": sess_name = args.get("name", "?") try: result = json.loads(result_text) @@ -392,6 +383,7 @@ async def handler( history = [] input_items = _build_input(user_input, history) + _used_sessions.clear() logger.info("Processing request %s (sessions: %s)", context.response_id, list(_sessions.keys())) message_item = stream.add_output_item_message() @@ -431,11 +423,26 @@ async def handler( logger.exception("Agent loop failed: %s", exc) final_reply = f"Agent error: {exc}" - # Emit final reply (no session header — live view already shown in verbose logs) + # Emit final reply if verbose_text: yield text_content.emit_delta("\n---\n\n") yield text_content.emit_delta(final_reply) + + # Append active session live view links at the end for easy access + if _sessions: + links = [] + for name, s in _sessions.items(): + url = s.get("live_view_url", "") + if url: + links.append(f"- **{name}**: [Live View]({url})") + else: + links.append(f"- **{name}**: (no live view)") + footer = "\n\n---\n**Active Sessions:**\n" + "\n".join(links) + if _used_sessions: + footer += f"\n\n_Used Sessions in this response: {', '.join(_used_sessions)}_" + yield text_content.emit_delta(footer + "\n") + yield text_content.emit_text_done() yield text_content.emit_done() yield message_item.emit_done() diff --git a/samples/python/hosted-agents/bring-your-own/responses/browser-automation/utils/constants.py b/samples/python/hosted-agents/bring-your-own/responses/browser-automation/utils/constants.py index 2e517a6f8..263aa0c72 100644 --- a/samples/python/hosted-agents/bring-your-own/responses/browser-automation/utils/constants.py +++ b/samples/python/hosted-agents/bring-your-own/responses/browser-automation/utils/constants.py @@ -29,6 +29,13 @@ - Use `goto` to navigate, `fill` for inputs, `click` for buttons. - **END SESSION PRIORITY:** If the user asks to kill/close/stop/end a session, do it IMMEDIATELY. Do NOT create new sessions or run other commands first. - NEVER reveal credentials, CDP URLs, or tokens. +- **NEVER ASK FOR CONFIRMATION.** Execute tasks fully without pausing. Do NOT say "If you want, I can continue" or "Should I proceed?" — JUST DO IT. Complete the entire task end-to-end. +- **USE REFS:** Always use element refs (e.g. e3, e15) from snapshots. After EVERY click, navigation, or page change, take a fresh `snapshot` before the next action. +- If a ref-based action fails, immediately retry with a different approach (text-based locator, eval, CSS selector) — do NOT stop or ask the user. +- **You ARE allowed and expected to fill forms and submit them when the user asks.** This is your core job. Do not refuse form-filling requests. +- **COMPLETE THE TASK:** Never give partial results. If filling a form, fill ALL fields and submit. If scraping, get ALL the data. Keep going until fully done. +- **REPORT BLOCKERS:** If you hit something you cannot bypass (OTP, CAPTCHA, login, manual verification, missing info), tell the user exactly what is blocking and on which session. Do NOT close the session or pretend the task is done — leave it open for the user to intervene. +- **DO NOT CLOSE SESSIONS PREMATURELY:** Only end a session when the user explicitly asks you to. Never close sessions on your own. """ TOOLS = [