Skip to content

Workshop Submission PR | UC-0A / UC-RAG / UC-MCP | RAG→MCP - #47

Open
SSukhvinderSingh wants to merge 4 commits into
nasscomAI:masterfrom
SSukhvinderSingh:participant/SSukhvinderSingh-Hyderabad
Open

Workshop Submission PR | UC-0A / UC-RAG / UC-MCP | RAG→MCP#47
SSukhvinderSingh wants to merge 4 commits into
nasscomAI:masterfrom
SSukhvinderSingh:participant/SSukhvinderSingh-Hyderabad

Conversation

@SSukhvinderSingh

Copy link
Copy Markdown

Name: S Sukhvinder Singh
City / Group: Hyderabad
Date: 2026-08-22


Checklist — Complete Before Opening This PR

✔️ agents.md committed for UC-0A, UC-RAG, UC-MCP
✔️ skills.md committed for UC-0A, UC-RAG, UC-MCP
✔️ results_hyderabad.csv present in uc-0a/
✔️ rag_server.py implemented in uc-rag/ (cosine space, 0.6 threshold, per-document answers, --json/--out)
✔️ mcp_server.py implemented in uc-mcp/ (scoped query_policy_documents + JSON-RPC)
✔️ 4 commits with meaningful messages following the formula
✔️ All sections below are filled in


UC-0A — Complaint Classifier

Which failure mode did you encounter first?

Taxonomy drift — the classifier accepted/created category variants outside the allowed enum instead of a fixed allowed list.

What enforcement rule fixed it? Quote the rule exactly as it appears in your agents.md:

"Category must be exactly one value from: Pothole, Flooding, Streetlight, Waste, Noise, Road Damage, Heritage Damage, Heat Hazard, Drain Blockage, Other. No variations and no invented names."

"Never invent category names outside the allowed list."

Your git commit message for UC-0A:

UC-0A Fix taxonomy drift: no fixed enum → restricted category to allowed list only


UC-RAG — RAG Server

Which failure mode did you encounter first?

ChromaDB defaulted to L2 distance, so the README's score = 1 - d/2 with the 0.6 threshold rejected every relevant chunk (relevant L2 distances ~1.2–1.4 exceed the 0.8 cutoff). The reference stub_rag.py failed identically at 0.6.

What enforcement rule fixed it? Quote the rule exactly as it appears in your agents.md:

"If no retrieved chunk scores above 0.6, output the refusal template and never answer from general knowledge. Score = 1 - d/2 where d is the ChromaDB cosine distance, so a chunk with raw cosine similarity s reports (1+s)/2; the 0.6 cutoff keeps chunks with raw cosine >= 0.2. (ChromaDB is configured with hnsw:space=cosine so the README's 0.6 threshold is honoured directly.)"

"If the passing chunks span more than one document, answer each document separately under its own labelled heading and never blend their claims into a single statement (retrieve_and_answer groups passing chunks by doc_name and emits one answer section per document)."

Your git commit message for UC-RAG:

UC-RAG Fix L2-distance-space: ChromaDB used L2 so the 0.6 threshold rejected all relevant chunks → switched collection to cosine space (kept THRESHOLD=0.6 + score=1-d/2), --json/--out output, and repointed LLM to gemini-2.5-flash


UC-MCP — MCP Server

Which failure mode did you encounter first?

Vague tool description — the tool didn't state its scope, so agents called query_policy_documents for out-of-scope questions (wasted calls / hallucinations). The handler and query_policy_documents were also raise NotImplementedError stubs.

What enforcement rule fixed it? Quote the rule exactly as it appears in your agents.md:

"Tool description must state the exact document scope: CMC HR Leave Policy, IT Acceptable Use Policy, and Finance Reimbursement Policy."

"Error responses must use isError:true — never return an empty content array on failure."

"The server must return HTTP 200 for all JSON-RPC responses including errors — transport errors use HTTP 4xx/5xx, application errors use JSON-RPC error objects."

Your git commit messages for UC-MCP:

UC-MCP Fix vague tool description: tool scope unstated so agents call it for out-of-scope questions -> implemented JSON-RPC (tools/list + tools/call) server with a scoped query_policy_documents tool (CMC HR/IT/Finance scope + refusal note), isError on refusal, HTTP 200 for all responses, and filled agents.md/skills.md

UC-MCP Fix rate-limited-model: gemini-2.5-flash deprecated + 429 free-tier -> switched LLM to gemma-4-26b-a4b-it via the Gemini API (same key)

…ejected all relevant chunks → switched collection to cosine space (kept THRESHOLD=0.6 + score=1-d/2), added per-document answers (rule nasscomAI#5), --json/--out output, and repointed LLM to gemini-2.5-flash
… it for out-of-scope questions -> implemented JSON-RPC (tools/list + tools/call) server with a scoped query_policy_documents tool (CMC HR/IT/Finance scope + refusal note), isError on refusal, HTTP 200 for all responses, and filled agents.md/skills.md
…-tier -> switched LLM to gemma-4-26b-a4b-it via the Gemini API (same key)
@github-actions

Copy link
Copy Markdown

Hi there, participant! Thanks for joining our RAG-to-MCP Workshop!

We're reviewing your PR for the 3 Use Cases (UC-0A, UC-RAG, UC-MCP). Once your submission is validated and merged, you'll be awarded your completion badge!

Next Steps:

  • Make sure all 3 UCs are finished.
  • Ensure your commit messages match the required format.
  • Fill out every section of the PR template.
  • Good luck!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant