Bug Description
FileInterceptor (Multer) loads the entire uploaded ZIP into file.buffer in memory before passing it to MinIO. For large bots with vendored dependencies (400MB+), this can OOM the API pod.
Steps to Reproduce
- Upload a large custom bot ZIP (400MB+) via the API
- Observe memory usage of the API pod spike to hold the entire file in memory before MinIO receives it
Expected Behavior
The upload should be streamed directly to MinIO without buffering the full file in memory.
Actual Behavior
The entire ZIP is buffered in memory before being passed to MinIO, risking OOM errors for large uploads.
Component
API
Additional Context
Current flow
Client uploads ZIP
-> Multer buffers entire file in memory (file.buffer)
-> storageService.uploadBotFile(file.buffer, ...)
-> minioClient.putObject(bucket, path, buffer, buffer.length)
Suggested fix
Switch from Multer's memory storage to streaming:
- Use Multer disk storage or a custom storage engine that pipes to MinIO
- Or use
busboy / fastify-multipart to get a readable stream and pipe directly to minioClient.putObject(bucket, path, stream, size)
- The MinIO client already supports
Readable streams, so uploadBotFile just needs to accept a stream instead of a Buffer
Relevant files
api/src/custom-bot/custom-bot.controller.ts (lines 41-44) - FileInterceptor + file.buffer
api/src/custom-bot/storage.service.ts (lines 105-132) - uploadBotFile takes Buffer
Bug Description
FileInterceptor(Multer) loads the entire uploaded ZIP intofile.bufferin memory before passing it to MinIO. For large bots with vendored dependencies (400MB+), this can OOM the API pod.Steps to Reproduce
Expected Behavior
The upload should be streamed directly to MinIO without buffering the full file in memory.
Actual Behavior
The entire ZIP is buffered in memory before being passed to MinIO, risking OOM errors for large uploads.
Component
API
Additional Context
Current flow
Suggested fix
Switch from Multer's memory storage to streaming:
busboy/fastify-multipartto get a readable stream and pipe directly tominioClient.putObject(bucket, path, stream, size)Readablestreams, souploadBotFilejust needs to accept a stream instead of aBufferRelevant files
api/src/custom-bot/custom-bot.controller.ts(lines 41-44) - FileInterceptor + file.bufferapi/src/custom-bot/storage.service.ts(lines 105-132) - uploadBotFile takes Buffer