Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 23 additions & 18 deletions uc-0a/agents.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,32 @@
# agents.md — UC-0A Complaint Classifier
# INSTRUCTIONS:
# 1. Open your AI tool
# 2. Paste the full contents of uc-0a/README.md
# 3. Use this prompt:
# "Read this UC README. Using the R.I.C.E framework, generate an
# agents.md YAML with four fields: role, intent, context, enforcement.
# Enforcement must include every rule listed under
# 'Enforcement Rules Your agents.md Must Include'.
# Output only valid YAML."
# 4. Paste the output below

role: >
[FILL IN]
A rule-based complaint classification agent for the City Operations team.
It operates on one complaint row at a time from a city CSV export and
produces the fields the Director's dashboard consumes. It has no access
to prior complaint history, reporter identity, or any system outside the
row it is given — only the `description` and other columns present in
that row.

intent: >
[FILL IN]
A correct output is one row where: category is always one of the ten
fixed schema values (never invented), priority is Urgent whenever a
severity keyword is present in the description (never missed), reason
names the specific word(s) from the description that drove the decision
(never blank, never generic), and flag is NEEDS_REVIEW whenever the
category cannot be determined with confidence — including when two
categories are equally supported by the text. Verifiable by re-reading
each row: the cited words must actually appear in that row's description.

context: >
[FILL IN]
The agent may use only the `description` field (and other row fields
such as `complaint_id`) supplied in the input CSV. It must not use
general knowledge about the city, the ward, or the reporter, and must
not infer facts that are not present as words in the description text.

enforcement:
- "[FILL IN: category enum rule]"
- "[FILL IN: severity keyword rule — list the keywords]"
- "[FILL IN: reason field rule]"
- "[FILL IN: ambiguity refusal rule]"
- "[FILL IN: no invented categories rule]"
- "Category must be exactly one value from: Pothole, Flooding, Streetlight, Waste, Noise, Road Damage, Heritage Damage, Heat Hazard, Drain Blockage, Other. No variations, synonyms, or invented sub-categories are permitted in the output."
- "Priority must be Urgent if the description contains any of these severity keywords (case-insensitive, substring match): injury, child, school, hospital, ambulance, fire, hazard, fell, collapse. This check is independent of category and cannot be skipped."
- "Every output row must include a non-empty reason field that names the specific keyword(s) matched in the description — never a generic phrase like 'looks urgent' or 'seems minor'."
- "If category cannot be determined confidently — zero category keywords match, OR two or more categories tie on keyword-match count — output category: Other and flag: NEEDS_REVIEW. Never pick one of the tied categories by arbitrary preference."
- "Never invent category names outside the allowed list. If any internal step would otherwise produce a value outside the enum, it must be forced to Other + NEEDS_REVIEW before the row is returned."
142 changes: 134 additions & 8 deletions uc-0a/classifier.py
Original file line number Diff line number Diff line change
@@ -1,26 +1,152 @@
"""
UC-0A — Complaint Classifier
classifier.py — Starter file
classifier.py

Build this using your AI coding tool:
1. Share agents.md, skills.md, and uc-0a/README.md
2. Ask the AI to implement this file
3. Run: python3 classifier.py --input ../data/city-test-files/test_pune.csv \
--output results_pune.csv
R.I.C.E-enforced complaint classifier. See agents.md for the full
role/intent/context/enforcement contract and skills.md for the skill
contracts this implementation follows.

Run:
python3 classifier.py --input ../data/city-test-files/test_pune.csv \
--output results_pune.csv
"""
import argparse
import csv

CATEGORIES = [
"Pothole", "Flooding", "Streetlight", "Waste", "Noise",
"Road Damage", "Heritage Damage", "Heat Hazard", "Drain Blockage", "Other",
]

# Enforcement rule 2 — must trigger Urgent regardless of category.
SEVERITY_KEYWORDS = [
"injury", "child", "school", "hospital", "ambulance",
"fire", "hazard", "fell", "collapse",
]

# Keyword sets per category. Substring match, case-insensitive.
# "Other" has no keyword set — it is the fallback when nothing else matches
# or when two categories tie (see classify_complaint).
CATEGORY_KEYWORDS = {
"Pothole": ["pothole"],
"Flooding": [
"flood", "flooded", "flooding", "waterlogging", "water-logging",
"knee-deep", "stranded", "inundat",
],
"Streetlight": [
"streetlight", "street light", "lights out", "light out",
"flickering", "sparking",
],
"Waste": [
"garbage", "waste", "dumped", "dead animal", "trash", "litter",
"overflowing",
],
"Noise": ["music", "noise", "loud", "horn", "honking"],
"Road Damage": [
"road surface", "cracked", "sinking", "manhole", "footpath",
"tiles broken", "upturned",
],
"Heritage Damage": ["heritage", "monument", "historic structure"],
"Heat Hazard": ["heatstroke", "heat wave", "extreme heat", "sunstroke"],
"Drain Blockage": [
"drain blocked", "drain choked", "clogged drain", "blocked drain",
"drain overflow",
],
}

MIN_DESCRIPTION_LENGTH = 8


def _matches(text_lower: str, keywords: list) -> list:
return [kw for kw in keywords if kw in text_lower]


def classify_complaint(row: dict) -> dict:
"""
Classify a single complaint row.
Returns dict with: complaint_id, category, priority, reason, flag
"""
raise NotImplementedError("Build this using your AI tool + agents.md")
complaint_id = (row.get("complaint_id") or "").strip()
description = (row.get("description") or "").strip()

if len(description) < MIN_DESCRIPTION_LENGTH:
return {
"complaint_id": complaint_id,
"category": "Other",
"priority": "Standard",
"reason": "Description too short to classify confidently.",
"flag": "NEEDS_REVIEW",
}

text_lower = description.lower()

severity_hits = _matches(text_lower, SEVERITY_KEYWORDS)
priority = "Urgent" if severity_hits else "Standard"

category_hits = {
cat: hits
for cat, hits in (
(cat, _matches(text_lower, kws)) for cat, kws in CATEGORY_KEYWORDS.items()
)
if hits
}

if not category_hits:
category = "Other"
flag = "NEEDS_REVIEW"
reason = "No category keywords matched the description."
else:
max_hits = max(len(hits) for hits in category_hits.values())
top_categories = sorted(
cat for cat, hits in category_hits.items() if len(hits) == max_hits
)
if len(top_categories) > 1:
all_matched = sorted({kw for hits in category_hits.values() for kw in hits})
category = "Other"
flag = "NEEDS_REVIEW"
reason = (
f"Ambiguous between {' and '.join(top_categories)} "
f"(matched keyword(s): {', '.join(all_matched)})."
)
else:
category = top_categories[0]
flag = ""
reason = f"Matched '{category}' on keyword(s): {', '.join(category_hits[category])}."

if category not in CATEGORIES:
# Enforcement rule 5 — never invent categories, even defensively.
category, flag = "Other", "NEEDS_REVIEW"

if severity_hits:
reason += f" Marked Urgent — severity keyword(s): {', '.join(severity_hits)}."

return {
"complaint_id": complaint_id,
"category": category,
"priority": priority,
"reason": reason,
"flag": flag,
}


def batch_classify(input_path: str, output_path: str):
"""Read input CSV, classify each row, write results CSV."""
raise NotImplementedError("Build this using your AI tool + agents.md")
fieldnames = ["complaint_id", "category", "priority", "reason", "flag"]
total, skipped = 0, 0
with open(input_path, newline="", encoding="utf-8") as infile, \
open(output_path, "w", newline="", encoding="utf-8") as outfile:
reader = csv.DictReader(infile)
writer = csv.DictWriter(outfile, fieldnames=fieldnames)
writer.writeheader()
for line_num, row in enumerate(reader, start=2): # header is line 1
total += 1
try:
writer.writerow(classify_complaint(row))
except Exception as e:
skipped += 1
print(f"[classifier] WARNING: skipped malformed row {line_num}: {e}")
print(f"[classifier] Processed {total} row(s), skipped {skipped} malformed row(s).")


if __name__ == "__main__":
parser = argparse.ArgumentParser(description="UC-0A Complaint Classifier")
Expand Down
16 changes: 16 additions & 0 deletions uc-0a/results_pune.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
complaint_id,category,priority,reason,flag
PM-202401,Pothole,Standard,Matched 'Pothole' on keyword(s): pothole.,
PM-202402,Pothole,Urgent,"Matched 'Pothole' on keyword(s): pothole. Marked Urgent — severity keyword(s): child, school.",
PM-202406,Flooding,Standard,"Matched 'Flooding' on keyword(s): flood, flooded, knee-deep, stranded.",
PM-202408,Flooding,Standard,"Matched 'Flooding' on keyword(s): flood, flooded.",
PM-202410,Streetlight,Standard,"Matched 'Streetlight' on keyword(s): streetlight, lights out.",
PM-202411,Streetlight,Urgent,"Matched 'Streetlight' on keyword(s): streetlight, flickering, sparking. Marked Urgent — severity keyword(s): hazard.",
PM-202413,Waste,Standard,"Matched 'Waste' on keyword(s): garbage, overflowing.",
PM-202418,Noise,Standard,Matched 'Noise' on keyword(s): music.,
PM-202419,Road Damage,Standard,"Matched 'Road Damage' on keyword(s): road surface, cracked, sinking.",
PM-202420,Road Damage,Urgent,Matched 'Road Damage' on keyword(s): manhole. Marked Urgent — severity keyword(s): injury.,
PM-202427,Flooding,Standard,Matched 'Flooding' on keyword(s): flood.,
PM-202428,Waste,Standard,Matched 'Waste' on keyword(s): dead animal.,
PM-202430,Other,Standard,"Ambiguous between Heritage Damage and Streetlight (matched keyword(s): heritage, lights out).",NEEDS_REVIEW
PM-202433,Waste,Standard,"Matched 'Waste' on keyword(s): waste, dumped.",
PM-202446,Road Damage,Urgent,"Matched 'Road Damage' on keyword(s): footpath, tiles broken, upturned. Marked Urgent — severity keyword(s): fell.",
33 changes: 24 additions & 9 deletions uc-0a/skills.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,30 @@
# skills.md — UC-0A Complaint Classifier
# INSTRUCTIONS: Same as agents.md — paste README into AI, ask for skills.md YAML

skills:
- name: classify_complaint
description: "[FILL IN]"
input: "[FILL IN]"
output: "[FILL IN]"
error_handling: "[FILL IN]"
description: >
Classifies a single complaint row into category, priority, reason,
and flag using fixed keyword rules against the CMC taxonomy and the
severity-keyword list. A tie between two or more categories on
keyword-match count is treated as ambiguous, not resolved by guessing.
input: "dict for one CSV row; must contain 'complaint_id' and 'description'"
output: "dict: {complaint_id, category, priority, reason, flag}"
error_handling: >
Description shorter than 8 characters -> category Other, flag
NEEDS_REVIEW, reason states it was too short to classify confidently.
Zero category keyword matches -> Other + NEEDS_REVIEW. Two or more
categories tied on keyword-match count -> Other + NEEDS_REVIEW
(ambiguous), never a confident single pick between the tied options.

- name: batch_classify
description: "[FILL IN]"
input: "[FILL IN]"
output: "[FILL IN]"
error_handling: "[FILL IN]"
description: >
Reads a city complaint CSV, classifies every row via
classify_complaint, and writes a results CSV with complaint_id,
category, priority, reason, flag columns.
input: "input_path: str (source CSV path), output_path: str (destination CSV path)"
output: "Writes output_path CSV; prints a summary line with row and skip counts."
error_handling: >
A row that raises an exception during classification (e.g. missing
required columns) is logged as a warning with its row number and
skipped; processing continues for all remaining rows so one malformed
row never aborts the batch.
44 changes: 21 additions & 23 deletions uc-mcp/agents.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,30 @@
# agents.md — UC-MCP MCP Server
# INSTRUCTIONS:
# 1. Open your AI tool
# 2. Paste the full contents of uc-mcp/README.md
# 3. Use this prompt:
# "Read this UC README. Using the R.I.C.E framework, generate an
# agents.md YAML with four fields: role, intent, context, enforcement.
# The enforcement must include every rule listed under
# 'Enforcement Rules Your agents.md Must Include'.
# Output only valid YAML."
# 4. Paste the output below, replacing this placeholder
# 5. Pay special attention to enforcement rule 1 — the tool description
# must state exact document scope

role: >
[FILL IN: Who is this agent? What layer of the stack does it operate at?
Hint: an MCP server that exposes policy retrieval as a tool]
A plain-HTTP MCP server that exposes the UC-RAG policy assistant as a
single discoverable tool, `query_policy_documents`, for any JSON-RPC
capable agent to call. It does not call the LLM directly and holds no
policy knowledge of its own — it only forwards questions to the RAG
server (rag_server.py, falling back to stub_rag.py) and returns its
result in MCP content format.

intent: >
[FILL IN: What does a correctly implemented MCP server produce?
Hint: JSON-RPC compliant responses, scoped tool description, correct refusals]
A correctly implemented server returns JSON-RPC 2.0 compliant responses
for both defined methods (tools/list, tools/call), exposes exactly one
tool whose description states its exact document scope and its refusal
behavior, and never leaves an agent guessing whether a call succeeded —
isError is set correctly and content is never empty. Verifiable against
the reference verification table in uc-mcp/README.md via test_client.py.

context: >
[FILL IN: What does this server have access to?
Hint: RAG server results only — no direct LLM calls, no outside knowledge]
The server has access only to the RAG server's query() result for the
current request and the llm_adapter's call_llm function passed through
to it. It has no direct LLM access of its own, no outside knowledge, and
no state carried between requests.

enforcement:
- "[FILL IN: Tool description scope rule]"
- "[FILL IN: Refusal documentation rule]"
- "[FILL IN: inputSchema required field rule]"
- "[FILL IN: isError on failure rule]"
- "[FILL IN: HTTP 200 for all JSON-RPC responses rule]"
- "Tool description must state the exact document scope: CMC HR Leave Policy, IT Acceptable Use Policy, and Finance Reimbursement Policy — named explicitly, not paraphrased as 'company policies' or similar."
- "Tool description must state what it cannot answer: questions outside these three documents return the refusal template via isError: true, so an agent reading the description knows in advance not to call it for out-of-scope questions like budget forecasts."
- "inputSchema must require 'question' as a non-empty string; tools/call must reject a missing or blank question with a JSON-RPC -32602 Invalid params error before ever calling the RAG server."
- "Error responses must use isError: true and a non-empty content array — a RAG refusal, an exception from the RAG server, and an unknown tool name are all still isError: true with the reason in content, never an empty content array."
- "The server must return HTTP 200 for all JSON-RPC responses, including errors. A malformed JSON body returns a JSON-RPC -32700 Parse error object at HTTP 200, not an HTTP 4xx/5xx status — actual transport failures (e.g. an unroutable path) are the only case for a non-200 status."
Loading