Extension:ChatbotRagContent

This extension is used to notify an external RAG database of content changes, and supply an API for retrieval of adapted content for that database. This extension was written specifically for use in a joint chatbot with RAG project between Kol-Zchut and Webix, and as such, the data format is probably not universally useful.

Document update flow by Webix

Configuration options

Name	values	Role
$wgChatbotRagContentPingURL	URL	Pinged on every content update
$wgChatbotRagContentNamespaces	Array of namespaces	Which namespaces this extension should work in
$wgChatbotRagContentArticleTypeBlocklist	array of article types	Article types to be ignored
$wgChatbotRagContentTitleAllowlist	array of titles	Titles that override namespace and article type restrictions

$wgChatbotRagContentPingURL

The data will be sent as JSON to the specified URL, in the following format:

{
    "page_id": 3,
    "rev_id": 13500,
    "callback_url": "https://example.com/w/rest.php/cbragcontent/v0/page_id/"
}

API for content retrieval

The extension provides a MediaWiki REST API endpoint, in this form: https://example.com/w/rest.php/cbragcontent/v0/page_id/3

Scenarios handled

1. Pages updated
2. New pages created directly in an allowed namespace
3. Pages moved in/out of allowed namespaces

Magic Words

EXCLUDE_FROM_RAG

If this magic word is present anywhere in a page, the page will be excluded from RAG content and API responses. This is useful for hiding sensitive or irrelevant pages from the RAG system. To use, simply add __EXCLUDE_FROM_RAG__ anywhere in the page's wikitext. This also adds the page to the Pages excluded from RAG tracking category (localizable).

Example:

This page should not be included in RAG.
__EXCLUDE_FROM_RAG__