gen-ai: add run guardrail span and security finding by habibam · Pull Request #427 · open-telemetry/semantic-conventions-genai

habibam · 2026-08-04T03:40:10Z

Description

Defines semantic conventions for observing GenAI guardrails policy, security findings and other controls that inspect GenAI content or actions before, during, or after a GenAI operation.

Adds:

gen_ai.run_guardrail.client for out-of-process guardrail services.
gen_ai.run_guardrail.internal for in-process guardrail checks.
gen_ai.security.finding event for findings produced by a guardrail run.
gen_ai.security.guardrail.* (evaluating service/component),
gen_ai.security.verdict.* (result returned), gen_ai.security.action.type
(caller enforcement), gen_ai.security.policy.* /
gen_ai.security.policy.rule.id (policy/rule attribution),
gen_ai.security.risk.* (classification/scoring), and
gen_ai.security.external_finding_id (SIEM correlation).
OpenAI reference coverage for the client/internal guardrail spans and finding.

Successor to #262 by Nagkumar Arkalgud.

Motivation

User journey: security investigation and incident triage. When a GenAI application blocks, modifies, or escalates a request, responders need to answer from telemetry alone which guardrail ran, what it evaluated, what verdict it returned, what the caller actually enforced, which policy/rule fired, and how to correlate the finding with the external security system (SIEM). Also supports sampling decisions (verdict/action) and audit.

Separating verdict (the guardrail's judgment) from action (the caller's enforcement) lets telemetry represent cases where a guardrail returns one result but the caller enforces another (e.g. verdict deny, action escalate).

Prior art: Google's Model Armor emits these guardrail spans in production (public reference).
Content-safety services, DLP filters, prompt-injection detectors, and policy engines all expose equivalent concepts (guardrail identity, verdict, policy, finding) today in vendor-specific forms.

Prototype

The reference scenarios under reference/scenarios/openai/ are updated to emit both the run_guardrail client/internal spans and the gen_ai.security.finding event, with generated reports under reference/reports/.

Caveat / follow-up: the current scenario is hand-authored rather than produced by an auto-instrumented guardrail library. Per @lmolkova top level review on #262, a survey of instrumentable open-source server-side guardrail libraries and adding a real one to the reference scenarios is the main remaining work item and is tracked as an open question tobe addressed at the SIG meeting.

Checklist

Motivation section filled in above
Reference scenarios updated for affected libraries
Towncrier fragment added under changelog.d/

Reproduces open-telemetry#262 (original author Nagkumar Arkalgud), rebased onto current main. Co-authored-by: Nagkumar Arkalgud~ Signed-off-by: Habiba Mohamed <habibam314@gmail.com>

- Drop gen_ai.security.guardrail.id (redundant with .name; R424) - Drop gen_ai.security.guardrail.version (R440) - Drop gen_ai.security.guardrail.provider.name; reuse gen_ai.provider.name (R448) - Drop gen_ai.security.finding.evidence — carries customer content, GDPR/DLP (R658/R666) - Remove span-hierarchy requirement language (R666) - Regenerate docs, reports, and openai scenario data Signed-off-by: Habiba Mohamed <habibam314@gmail.com>

Signed-off-by: Habiba Mohamed <habibam314@gmail.com>

linux-foundation-easycla · 2026-08-04T03:40:18Z

The committers listed above are authorized under a signed CLA.

✅ login: habibam / name: Habiba Mohamed (40464bc, 9104e2c, ac80038, bc1f115)

opentelemetry-pr-dashboard · 2026-08-04T03:40:59Z

Pull request dashboard status

Waiting on the author · refreshed 2026-08-24 09:32 UTC

Resolve merge conflicts.

Respond to 1 review item (e.g. link a commit, explain why not, ask a follow-up):

Top-level threads: 1

Status above doesn't look right?

Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
Should this be with reviewers? Comment /dashboard route:reviewers to route it to them.
Anything wrong — including the routing? Report it with what you expected; it helps us improve the dashboard.

Copilot

Pull request overview

This PR introduces first-class GenAI security guardrail observability to the semantic conventions, adding two new run_guardrail span types (client + internal) and a gen_ai.security.finding event, plus the corresponding registry attributes and OpenAI reference coverage/reporting to demonstrate capturability.

Changes:

Add gen_ai.run_guardrail.{client,internal} span definitions and a gen_ai.security.finding event, with a new gen_ai.security.* attribute set in the registry.
Update the reference pipeline (model/spec loading, classification, report linking, and ordering) to recognize and report the new spans/events.
Extend the OpenAI reference scenario + generated reports/data to emit the new spans and event.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
reference/src/semconv_genai/semconv_model.py	Registers the new guardrail span specs and security finding event spec for reference evaluation.
reference/src/semconv_genai/report.py	Adds doc links for the new span/event types in generated reference reports.
reference/src/semconv_genai/data_files.py	Adds ordering for new span/event types in reference report generation.
reference/src/semconv_genai/classify.py	Disambiguates `run_guardrail` client vs internal span types during classification.
reference/scenarios/openai/scenario.py	Emits guardrail client/internal spans and `gen_ai.security.finding` event in the OpenAI scenario.
reference/scenarios/openai/data.json	Captures observed attribute coverage for the new span/event types in the OpenAI scenario.
reference/reports/run-guardrail-internal-span.md	New generated coverage report for internal guardrail spans.
reference/reports/run-guardrail-client-span.md	New generated coverage report for client guardrail spans.
reference/reports/gen-ai-security-finding-event.md	New generated coverage report for the security finding event.
reference/README.md	Lists the new guardrail span reports and the new event report.
model/gen-ai/spans.yaml	Adds shared attribute group + defines `gen_ai.run_guardrail.{client,internal}` spans.
model/gen-ai/registry.yaml	Adds `gen_ai.security.*` attributes and registers `run_guardrail` as an operation name.
model/gen-ai/gen-ai-security-content.json	Introduces a JSON schema placeholder for guardrail content value attributes.
model/gen-ai/events.yaml	Adds the `gen_ai.security.finding` event definition and its attribute set.
docs/registry/attributes/gen-ai.md	Regenerated attribute registry docs including the new `gen_ai.security.*` attributes.
docs/gen-ai/README.md	Adds the new “Security” documentation page to the GenAI docs index.
docs/gen-ai/openai.md	Adds `run_guardrail` to operation-name guidance tables for OpenAI docs.
docs/gen-ai/mcp.md	Adds `run_guardrail` to operation-name guidance tables for MCP docs.
docs/gen-ai/gen-ai-spans.md	Adds `run_guardrail` to operation-name guidance tables for span docs.
docs/gen-ai/gen-ai-security.md	New documentation page for guardrail spans + finding event (weaver-generated sections).
docs/gen-ai/gen-ai-metrics.md	Adds `run_guardrail` to operation-name guidance tables for metrics docs.
docs/gen-ai/gen-ai-events.md	Adds the `gen_ai.security.finding` event section and TOC entry.
docs/gen-ai/gen-ai-agent-spans.md	Adds `run_guardrail` to operation-name guidance tables for agent span docs.
docs/gen-ai/azure-ai-inference.md	Adds `run_guardrail` to operation-name guidance tables for Azure AI Inference docs.
docs/gen-ai/aws-bedrock.md	Adds `run_guardrail` to operation-name guidance tables for AWS Bedrock docs.
docs/gen-ai/anthropic.md	Adds `run_guardrail` to operation-name guidance tables for Anthropic docs.
changelog.d/NNN.component.md	Adds a Towncrier fragment describing the new security guardrail conventions.

+{
+    "description": "Represents content evaluated or produced by a security guardrail.",
+    "title": "SecurityGuardrailContent"
+}


@@ -0,0 +1 @@
+Add GenAI security guardrail span conventions and a security finding event.


opentelemetry-pr-dashboard · 2026-08-11T03:45:20Z

Hi @habibam — just a friendly reminder that this pull request is waiting on you. The dashboard status comment has the open items and is kept current.

Replying is enough to hand it off — answer, explain why no change is needed, or ask a follow-up. The dashboard routes it onward once nothing on the list is waiting on you.
To hand it back for any other reason, including the dashboard getting this wrong, comment /dashboard route:reviewers.

…l' framing Restructure the guardrail attributes into a layered namespace agreed at the Aug-7 GenAI SIG: a generic base tier (gen_ai.guardrail.component.name, .verdict.*, .action.*, .target.*) that any guardrail instrumentation can populate, and an opt-in security layer (gen_ai.guardrail.security.*: policy, content, risk, external_finding_id). Rename risk.finding -> risk.category since its values are risk categories (OWASP LLM Top-10), and rename the finding event to gen_ai.guardrail.security.finding. Stop describing the guardrail mechanism as a 'security guardrail' in the base tier: generic auto-instrumentation cannot know whether a guardrail is security-related, so the span, operation, component, and verdict now read as a plain 'guardrail'. 'security' is kept only for the genuine security layer (findings, policies, risk, external systems). Rename the changelog fragment to the successor PR number. Regenerates registry docs, embedded tables, and reference reports. Signed-off-by: Habiba Mohamed <habibam314@gmail.com>

AgentGymLeader · 2026-08-14T19:13:06Z

Saw the open item in your description — a real instrumented guardrail library in the reference scenarios, rather than hand-authored spans.

I have one running in #368. reference/scenarios/agent-governance/ drives the OpenAI Agents SDK with a plain-function input guardrail: one allowed run that executes a tool, and one tripwire run with zero model calls and no child execute span. The decision attributes are read off the SDK's own guardrail runtime objects, so it is genuine instrumentation rather than spans written by hand.

It currently sits on a draft PR of mine, but nothing in the scenario is specific to my proposal — it is just a guardrail library being instrumented for real. Happy to move it here, or leave it where it is and reference it from this PR. Whichever is less work for you.

One practical note while you are in here: EasyCLA is still failing on this PR, and reviewers generally will not start until that goes green.

…erlay Address review feedback to keep the guardrail attribute set minimal and make the security-specific fields separable from the base guardrail span. - Drop gen_ai.guardrail.security.policy.name and .policy.version. policy.id (with the optional policy.rule.id) is enough to correlate a verdict to the policy that produced it; the name/version pair added surface without a capture story. - Split the run_guardrail attributes into a base group (attributes.gen_ai.run_guardrail.common) and a separable security overlay (attributes.gen_ai.guardrail.security), so generic guardrail instrumentation can emit the base span without the security fields. Move gen_ai.conversation.id into the base group. Signed-off-by: Habiba Mohamed <habibam314@gmail.com>

Update the hand-authored openai security-guardrail scenario to drop the gen_ai.guardrail.security.policy.name / .policy.version attributes and regenerate its data.json to match the slimmed model. Signed-off-by: Habiba Mohamed <habibam314@gmail.com>

Add a runnable scenario that wraps the real openai-guardrails framework and emits the base gen_ai.run_guardrail.internal span from framework-owned runtime state, proving the base tier is capturable by generic instrumentation. It intentionally does not emit the security overlay or the finding event: the framework runtime exposes no policy identity, external finding id, or standardized risk class, which is the concrete capture gap that motivates keeping the security overlay separable. It emits only guardrail spans; the delegated chat inference span belongs to the openai library. Signed-off-by: Habiba Mohamed <habibam314@gmail.com>

Regenerated with 'uv run update-reports'. The base run_guardrail attributes now show coverage from both openai and openai-guardrails; the security overlay and finding event remain openai-only, matching the documented capture gap. Signed-off-by: Habiba Mohamed <habibam314@gmail.com>

Signed-off-by: Habiba Mohamed <habibam314@gmail.com>

Integrate the upstream conformance-tooling refactor with the GenAI security-guardrail conventions. semconv_model.py: upstream rewrite plus the run_guardrail spans and finding event; classify.py: accept deletion; openai/data.json: upstream keys plus guardrail entries; regenerate docs; add openai-guardrails/conformance.yaml. Signed-off-by: Habiba Mohamed <habibam314@gmail.com>

The gen-ai-events.md table-of-contents link for gen_ai.guardrail.security.finding still pointed at the pre-rename anchor (#event-gen_aisecurityfinding). Point it at the actual heading anchor (#event-gen_aiguardrailsecurityfinding) so the link checker passes. Signed-off-by: Habiba Mohamed <habibam314@gmail.com>

habibam added 3 commits August 3, 2026 15:32

gen-ai: add run guardrail span and security finding

bc1f115

Reproduces open-telemetry#262 (original author Nagkumar Arkalgud), rebased onto current main. Co-authored-by: Nagkumar Arkalgud~ Signed-off-by: Habiba Mohamed <habibam314@gmail.com>

changelog: rename fragment to PR number

9104e2c

Signed-off-by: Habiba Mohamed <habibam314@gmail.com>

habibam requested a review from a team as a code owner August 4, 2026 03:40

Copilot AI lite review requested due to automatic review settings August 4, 2026 03:40

github-project-automation Bot added this to GenAI Semantic Conventions and Instrumentation libraries Aug 4, 2026

Copilot started reviewing on behalf of habibam August 4, 2026 03:40 View session

opentelemetry-pr-dashboard Bot mentioned this pull request Aug 4, 2026

Pull Request Dashboard #204

Open

Copilot AI reviewed Aug 4, 2026

View reviewed changes

habibam mentioned this pull request Aug 4, 2026

gen-ai: add run guardrail span and security finding #262

Closed

MarkovianProtocol mentioned this pull request Aug 5, 2026

Question: is integrity of the telemetry itself in scope — canonical span hashing for externally verifiable evidence? #438

Closed

meshailabs mentioned this pull request Aug 6, 2026

Specify a machine-readable baseline for the Evidence Preservation list: map it to OpenTelemetry GenAI semantic conventions OpenSecureAIAlliance/RFCs#5

Open

surpradhan mentioned this pull request Aug 12, 2026

Project Proposal: Audit Logging SIG open-telemetry/community#2409

Open

AgentGymLeader mentioned this pull request Aug 14, 2026

Add an opaque governance record reference for decision operations #368

Draft

habibam added 9 commits August 17, 2026 19:45

reference: add openai-guardrails uv.lock

f9e9c83

Signed-off-by: Habiba Mohamed <habibam314@gmail.com>

fix CI tests

03f2f0d

thebenignhacker mentioned this pull request Aug 19, 2026

Proposal: execute_authorization, an operation for agent authorization/governance decisions #461

Closed

sjain700 approved these changes Aug 24, 2026

View reviewed changes

Navigation Menu

Uh oh!

gen-ai: add run guardrail span and security finding - #427

gen-ai: add run guardrail span and security finding#427
habibam wants to merge 13 commits into
open-telemetry:mainfrom
habibam:gen-ai-security-guardrails

habibam commented Aug 4, 2026 •

edited

Loading

Uh oh!

linux-foundation-easycla Bot commented Aug 4, 2026 •

edited

Loading

Uh oh!

opentelemetry-pr-dashboard Bot commented Aug 4, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

opentelemetry-pr-dashboard Bot commented Aug 11, 2026 •

edited

Loading

Uh oh!

AgentGymLeader commented Aug 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		@@ -0,0 +1 @@
		Add GenAI security guardrail span conventions and a security finding event.

Uh oh!

Conversation

habibam commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation

Prototype

Checklist

Uh oh!

linux-foundation-easycla Bot commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

opentelemetry-pr-dashboard Bot commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull request dashboard status

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

opentelemetry-pr-dashboard Bot commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AgentGymLeader commented Aug 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

habibam commented Aug 4, 2026 •

edited

Loading

linux-foundation-easycla Bot commented Aug 4, 2026 •

edited

Loading

opentelemetry-pr-dashboard Bot commented Aug 4, 2026 •

edited

Loading

opentelemetry-pr-dashboard Bot commented Aug 11, 2026 •

edited

Loading