program: gssoc
📝 Description
This issue addresses a critical Denial of Service (DoS) vulnerability (Unconstrained Resource Consumption) in the backend's RAG scraper logic.
Currently, in �ackend/utils/ragUtilities.js, the scrapeWebpage and scrapeTitle functions use (await fetch(url)).text() to download the contents of arbitrary user-provided URLs.
Because there are no size limits or content-type validations, a malicious user can supply a URL pointing to an infinite data stream (e.g., /dev/urandom over HTTP) or a massive multi-gigabyte file (e.g., an ISO or video). Node.js will attempt to buffer the entire payload into memory as a single V8 string. This will rapidly exhaust the Node heap memory limit, resulting in an Out Of Memory (OOM) Panic, crashing the entire backend API and taking the application offline for all users.
The proposed fix introduces a safeFetchText helper that:
- Validates the Content-Type to ensure only text/html-based documents are processed.
- Streams the
esponse.body in chunks.
- Enforces a strict maximum payload size (e.g., 5MB). If the size is exceeded, the connection is aborted and an error is thrown, protecting the server's memory.
🔗 Related PR
N/A
🔄 Type of Change
🐛 Bug fix
🚨 Security Vulnerability (Critical severity)
🧪 How to Test
- Attempt to create a chat with a documentation URL pointing to a massive file (e.g., a 10GB test file).
- Verify that the backend safely rejects the URL with a "Payload too large" error instead of crashing the Node.js process.
📸 Screenshots (if applicable)
N/A
✅ Checklist
[x] I am contributing under GSSoC
[x] My code follows the project's existing style
[x] I have tested my changes
[x] I have linked the related issue above
program: gssoc
📝 Description
This issue addresses a critical Denial of Service (DoS) vulnerability (Unconstrained Resource Consumption) in the backend's RAG scraper logic.
Currently, in �ackend/utils/ragUtilities.js, the scrapeWebpage and scrapeTitle functions use (await fetch(url)).text() to download the contents of arbitrary user-provided URLs.
Because there are no size limits or content-type validations, a malicious user can supply a URL pointing to an infinite data stream (e.g., /dev/urandom over HTTP) or a massive multi-gigabyte file (e.g., an ISO or video). Node.js will attempt to buffer the entire payload into memory as a single V8 string. This will rapidly exhaust the Node heap memory limit, resulting in an Out Of Memory (OOM) Panic, crashing the entire backend API and taking the application offline for all users.
The proposed fix introduces a safeFetchText helper that:
esponse.body in chunks.
🔗 Related PR
N/A
🔄 Type of Change
🐛 Bug fix
🚨 Security Vulnerability (Critical severity)
🧪 How to Test
📸 Screenshots (if applicable)
N/A
✅ Checklist
[x] I am contributing under GSSoC
[x] My code follows the project's existing style
[x] I have tested my changes
[x] I have linked the related issue above