[Noida] Aditya — RAG-to-MCP Submission

[aditya2529]

RAG-to-MCP — Submission PR

Name: Aditya
City / Group: Noida
Date: 2026-06-28
AI tool(s) used: Claude (Claude Code), LLM provider swapped to Groq (Llama 3.3 70B)

---

Submission Checklist

• uc-0a/agents.md — present and updated
• uc-0a/skills.md — present and updated
• uc-0a/classifier.py — runs without crash
• uc-0a/results_[city].csv — output present (results_pune.csv)
• uc-rag/agents.md — present and updated
• uc-rag/skills.md — present and updated
• uc-rag/rag_server.py — not the stub, my implementation
• uc-mcp/agents.md — present and updated
• uc-mcp/skills.md — present and updated
• uc-mcp/mcp_server.py — passes all five test_client.py tests
• 3+ commits with meaningful messages, one per UC
• All sections below filled

---

UC-0A — Complaint Classifier

Which failure mode did you encounter first?

Severity blindness — injury/child/school complaints classified as Standard instead of Urgent.

Which enforcement rule fixed it? Quote from your agents.md:

"Priority must be Urgent if the description contains any of: injury, child, school, hospital, ambulance, fire, hazard, fell, collapse."

Your commit message for UC-0A:

[UC-0A] Implement complaint classifier: rule-based 10-category classifier with severity detection and ambiguity flags, RICE agents.md + skills.md, results_pune.csv

Verification checkpoints:

• All severity-signal rows (injury/child/school/hospital keywords) classified as Urgent
• No invented categories outside the defined taxonomy
• Justification column present and non-empty for every row

---

UC-RAG — RAG Server

Which failure mode did you encounter?

Wrong retrieval caused by chunk-boundary failure: whole-document (~400-token) chunks
retrieved the right document but at very low similarity (~0.34), so everything fell
below threshold and refused.

What chunking strategy did you use and why?

Clause-level, sentence-aware chunking: each numbered clause becomes one chunk, prefixed
with its section title for context. This keeps a whole clause (e.g. HR 5.2's dual-approver
obligation) intact — never split across chunks — while making chunks small enough that the
relevant clause scores high on cosine similarity. This produced 81 chunks across 3 docs.

Did your system correctly refuse "What is the flexible working culture?"?

Yes — returns the refusal template (top similarity ~0.28, below threshold).

Did your system retrieve the correct document for "Can I use my personal phone for work files?"?

Yes — retrieves the IT Acceptable Use policy (BYOD section 3.x), not HR leave, and does
not blend HR content into the answer.

Which enforcement rule in agents.md prevented answers outside retrieved context?

"The answer must use only information present in the retrieved chunks; never add context
from outside the retrieved set." (Plus the refusal rule when no chunk clears the threshold.)

Your commit message for UC-RAG:

[UC-RAG] Implement RAG server: clause-level sentence-aware chunking + ChromaDB cosine retrieval + threshold-gated grounded answers with citations and refusal, RICE agents.md + skills.md

Verification checkpoints:

• At least 3 test queries return grounded answers (cited from retrieved context)
• "What is the flexible working culture?" returns the refusal template
• "Can I use my personal phone for work files?" retrieves IT policy, not HR leave policy
• Chunking produces more than 1 chunk per document (81 chunks across 3 docs)

---

UC-MCP — MCP Server

Paste your tool description from mcp_server.py TOOL_DEFINITION:

"Answers questions about CMC (City Municipal Corporation) policies ONLY, drawn from three
documents: the HR Leave Policy, the IT Acceptable Use Policy, and the Finance Reimbursement
Policy. Returns answers grounded in retrieved document chunks with source citations. Any
question outside these three policies (e.g. budgets, forecasts, payroll, general advice)
returns a refusal — do not call this tool for those."

Does it state the document scope explicitly?

Yes — names all three policies and the out-of-scope refusal behaviour.

Run result: python3 test_client.py --run-all

tools/list → 1 tool, scope mentioned ✅ PASS
"Who approves leave without pay?" → isError False, HR answer ✅ PASS
"Can I use my personal phone for work files?" → isError False, IT-grounded ✅ PASS
"What is the budget forecast for 2025?" → isError True, refusal ✅ PASS
Unknown method → JSON-RPC error -32601 ✅ PASS
(All 5 reference tests pass.)

Did the budget forecast question return isError: true?

Yes.

In one sentence — why is the tool description the enforcement?

Because an agent decides whether to call the tool purely from its description, so stating
the exact scope (and what it refuses) is what stops the agent from calling it for
out-of-scope questions — the same role a RICE Enforcement rule plays for a prompt.

Your commit message for UC-MCP:

[UC-MCP] Implement MCP server: JSON-RPC tools/list + tools/call with scoped tool description, isError handling, -32601; swapped llm_adapter to Groq; RICE agents.md + skills.md

Verification checkpoints:

• Tool description explicitly states document scope (which policies are covered)
• Tool description states refusal behavior for out-of-scope queries
• python3 test_client.py --run-all executes without connection error
• Budget forecast question returns isError: true (out of scope)

---

CRAFT Reflection

Which step of the CRAFT loop was hardest across all three UCs?

Calibration — making the retrieval threshold meaningful. The README's nominal 0.6 was set
for an L2 similarity metric; in true cosine space the equivalent relevance cut-point for
all-MiniLM-L6-v2 is ~0.35. Measuring the actual in-scope vs out-of-scope score gap and
calibrating to it (rather than blindly using 0.6, which refused everything) was the key step.

What did you add to agents.md manually that the AI did not generate?

The cross-document non-blending rule for UC-RAG: "If the query spans two documents, retrieve
from each separately and never merge chunks from different documents into a single blended
claim" — this is what protects the personal-phone question from an IT+HR blend.

One specific task in your real work where you will use R.I.C.E in the next 7 days:

Building a policy/runbook Q&A assistant over internal documents at Iris Software, using a
scoped tool description and a retrieval threshold so it cites sources and refuses out-of-scope questions.

[github-actions]

Hi there, participant! Thanks for joining our RAG-to-MCP Workshop!

We're reviewing your PR for the 3 Use Cases (UC-0A, UC-RAG, UC-MCP). Once your submission is validated and merged, you'll be awarded your completion badge!

Next Steps:

• Make sure all 3 UCs are finished.
• Ensure your commit messages match the required format.
• Fill out every section of the PR template.
• Good luck!