Satara Rohan Dhumal - RAG-to-MCP Submission - #39
Open
rohan-dhumal wants to merge 3 commits into
Open
Conversation
…hild/school/hospital/ambulance/fire/hazard/fell/collapse triggers forcing Urgent priority
…nce → added per-document routing pass before chunk retrieval - Wrong retrieval: naive_query concatenates all 3 policies with no routing; "Can I use my personal phone for work files?" now resolves to IT policy chunk 3 (BYOD) only, never blending in HR or Finance chunks. - Chunk boundary: switched from token-count accumulation to section-aware chunking (splits on the documents' own "N. TITLE" headers), so clause 5.2 (Department Head + HR Director) stays whole in one chunk instead of risking a mid-clause cut, and each chunk keeps its section title for better embedding signal. - Threshold calibration: measured that all-MiniLM-L6-v2 cosine similarity for correct matches (0.37-0.78) never approaches the README's literal 0.6 for this corpus, while the true refusal case tops out at 0.21. Recalibrated threshold to 0.30 to preserve the refusal-when-irrelevant intent without refusing legitimate, correctly-grounded answers.
…Finance policy scope + explicit refusal note - Vague tool description: TOOL_DEFINITION description now names the exact three documents in scope and states out-of-scope questions return a refusal, so a calling agent can decide not to invoke the tool for questions like a budget forecast before ever calling it. - Implemented do_POST (tools/list, tools/call, -32700/-32601/-32602 JSON-RPC errors, always HTTP 200) and query_policy_documents (missing question / RAG refusal / exception -> isError: true). - Fixed a cold-start timing bug found while testing: the first tools/call triggered lazy loading of torch/sentence-transformers/chromadb inside the request handler, exceeding test_client.py's fixed 10s timeout and aborting the connection. Added a warmup query at server startup so the embedder and vector store are loaded before the server accepts requests. - Verified: all 5 test_client.py --run-all cases pass, including the budget-forecast question returning isError: true.
|
Hi there, participant! Thanks for joining our RAG-to-MCP Workshop! We're reviewing your PR for the 3 Use Cases (UC-0A, UC-RAG, UC-MCP). Once your submission is validated and merged, you'll be awarded your completion badge! Next Steps:
|
rohan-dhumal
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RAG-to-MCP — Submission PR
Name: Rohan Dhumal
City / Group: Satara
Date: 2026-07-22
AI tool(s) used: Claude Code (Sonnet 5)