Skip to content

DevDocs Enhancement: Configurable Storage Path Implementation #33

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

tankcdr
Copy link

@tankcdr tankcdr commented Mar 19, 2025

DevDocs Enhancement: Configurable Storage Path Implementation

Overview

This PR improves the DevDocs application by implementing a flexible storage configuration feature that allows users to separate curated data from the source code. These changes enhance the application's flexibility and usability by making the storage location fully configurable through a single environment variable, working consistently across both local development and Docker deployments.

Changes Made

  • Environment Configuration: Added STORAGE_PATH environment variable to .env.template with documentation
  • API Layer: Modified app/api/storage/route.ts to use the configured storage path from environment variables
  • Startup Scripts: Updated start.sh to read the storage path from .env and pass it to the MCP server
  • Docker Integration: Updated docker-start.sh to respect the configured storage path and modified docker-compose.yml to use the environment variable for volume mounting
  • Frontend Storage Mount: Updated docker-compose.yml to add a storage volume mount to the frontend container using the STORAGE_PATH environment variable for consistency
  • MCP Server: Enhanced the MCP server code to check for environment variables
  • Documentation: Updated the README with comprehensive documentation on how to use the new feature

Technical Implementation

  • Implemented a flexible configuration system that works with both relative and absolute paths
  • Refactored storage access code to improve maintainability and configuration options
  • Added environment variable support to ensure consistent behavior across different deployment methods
  • Fixed potential inconsistencies between local and Docker deployments by standardizing the configuration approach
  • Maintained backward compatibility with existing setups to ensure smooth transitions

Testing Done

  • Verified functionality by testing with both relative and absolute storage paths
  • Tested in both local development and Docker environments
  • Ensured Docker integration works properly with configured storage paths
  • Validated MCP server communication with the new configuration system
  • Validated consolidated files were found and displayed correctly in the frontend ui

Usage Example

Users can now set the STORAGE_PATH environment variable in their .env file:

For storage inside the project (default)
STORAGE_PATH=storage/markdown

For storage outside the project
STORAGE_PATH=/path/to/your/storage

The application will respect this path in both local and Docker deployments, allowing users to store curated data wherever is most convenient for their workflow.

Additional Notes

  • This implementation allows teams to maintain separate data storage while working on the same codebase
  • The change is non-disruptive as it maintains the default behavior if no configuration is provided
  • Future work could include additional configuration options for other aspects of the application
  • This approach could be extended to other parts of the system that currently use hardcoded paths
  • Fixed an inconsistency in path handling between the main storage API and the download API
  • Ensured all components use the same configurable storage path mechanism

@tankcdr tankcdr marked this pull request as draft March 19, 2025 16:04
tankcdr added 3 commits March 19, 2025 12:09
This commit fixes an inconsistency in how storage paths are handled between
different parts of the application. Previously, the download route was using
a hardcoded storage path while the main API route was using the configurable
path from environment variables.

The change ensures the download route uses the same storage path resolution
logic, allowing files to be properly accessed when using custom storage
locations. This resolves the issue where consolidated files weren't appearing
in the frontend after setting a custom STORAGE_PATH.
The frontend container was trying to access a storage path that doesn't exist
within the container's filesystem, causing ENOENT errors when listing files.
This change:

- Adds a volume mount to map host storage to the frontend container
- Sets consistent STORAGE_PATH environment variable
- Ensures all containers (frontend, backend, MCP) share storage access
The frontend container was attempting to access files in a directory path
that existed on the host but not inside the container, resulting in
"ENOENT: no such file or directory" errors when scanning the storage directory.

Changes:
- Add volume mount to frontend container mapping host storage to container path
- Set consistent STORAGE_PATH environment variable across all containers
- Fix syntax errors and improve error handling in storage/route.ts
- Clean up temporary debug statements after diagnosis

This ensures all containers (frontend, backend, MCP) share access to the same
storage directory, allowing files created by one container to be accessible
by others, which fixes file listing and retrieval in the UI.
@tankcdr tankcdr changed the title added tested functionality (on macos) to allow configuration of stora… DevDocs Enhancement: Configurable Storage Path Implementation Mar 19, 2025
@tankcdr tankcdr marked this pull request as ready for review March 19, 2025 17:17
@Shubham-Khichi Shubham-Khichi self-requested a review March 21, 2025 21:49
@Shubham-Khichi Shubham-Khichi self-assigned this Mar 21, 2025
@tankcdr
Copy link
Author

tankcdr commented Apr 8, 2025

Hey @Shubham-Khichi is there anything you want mw to do to advance this?

@Shubham-Khichi
Copy link
Contributor

Hi @tankcdr sorry for delayed reply I was finding some solution to other issues, but I had your commit in mind.

There seems to be conflicts on this branch with main. Can you resolve this on your side if it allows you? Else I can do it from my side.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants