feat: Implement decentralized storage enhancements with IPFS, Arweave, and CLI support

[vishwamartur]

🚀 Decentralized Storage Enhancements

This PR implements comprehensive decentralized storage capabilities for JuliaOS as requested in issue #112.

✅ Features Implemented

Storage Providers

• IPFS Storage Provider - Full implementation with HTTP API and CLI support
• Arweave Storage Provider - Complete implementation with direct API and Bundlr integration
• Enhanced Local Storage - Improved with metadata support and provider factory

CLI Commands

• juliaos storage list-providers - Show available storage providers
• juliaos storage current-provider - Display current provider
• juliaos storage switch <provider> - Switch storage providers with configuration
• juliaos storage info - Show detailed provider information
• juliaos storage upload <file> - Upload files with metadata
• juliaos storage download <key> - Download files by storage key
• juliaos storage list [prefix] - List stored files with filtering
• juliaos storage delete <key> - Delete files with confirmation
• juliaos storage exists <key> - Check file existence

Agent Tools

• tool_file_upload - Agent file upload capability
• tool_file_download - Agent file download capability
• tool_storage_manage - Storage management operations

HTTP API Endpoints

• /api/v1/storage/providers - Provider management
• /api/v1/storage/files - File operations
• /api/v1/storage/stats - Storage statistics

🔧 Key Components

New Files Added:

• julia/src/storage/ipfs_storage.jl - IPFS integration
• julia/src/storage/arweave_storage.jl - Arweave integration
• julia/src/api/StorageHandlers.jl - HTTP API handlers
• julia/apps/cli.jl - Comprehensive CLI implementation
• julia/bin/juliaos - Cross-platform CLI wrapper
• backend/src/agents/tools/tool_file_*.jl - Agent storage tools
• julia/test/storage_test.jl - Comprehensive test suite
• julia/test/cli_test.jl - CLI functionality tests
• julia/examples/storage_demo.jl - Working demonstration
• docs/storage-enhancements.md - Complete documentation
• docs/cli-storage-commands.md - CLI usage guide

Modified Files:

• julia/src/storage/Storage.jl - Enhanced with provider factory
• julia/src/api/Routes.jl - Added storage routes
• julia/src/api/API.jl - Included storage handlers
• julia/config/config.toml - Added storage provider configurations
• backend/src/agents/tools/Tools.jl - Registered new tools
• README.md - Updated with storage features

🎯 Usage Examples

CLI Usage:

# Switch to IPFS
juliaos storage switch ipfs --config-json '{"api_url": "http://localhost:5001"}'

# Upload a file
juliaos storage upload data.json --key "dataset_v1" --metadata '{"type": "dataset"}'

# Download a file
juliaos storage download "dataset_v1" --output downloaded.json

Agent Integration:

# Agent blueprint with storage tools
blueprint = AgentBlueprint(
    tools=[
        ToolBlueprint("file_upload", Dict("max_file_size" => 5*1024*1024)),
        ToolBlueprint("file_download", Dict("include_metadata" => true)),
        ToolBlueprint("storage_manage", Dict("allow_provider_switch" => true))
    ]
)

Programmatic Usage:

using JuliaOS.JuliaOSFramework.Storage

# Switch to IPFS
Storage.switch_provider(:ipfs, config=Dict("api_url" => "http://localhost:5001"))

# Save data with metadata
Storage.save_default("my_key", data; metadata=Dict("type" => "analysis"))

# Load data
result = Storage.load_default("my_key")

🧪 Testing

• ✅ Comprehensive unit tests for all storage providers
• ✅ Integration tests for agent file operations
• ✅ CLI functionality tests
• ✅ Error handling and edge case coverage
• ✅ Cross-platform compatibility tests

📚 Documentation

• ✅ Complete API documentation
• ✅ CLI usage guide with examples
• ✅ Configuration reference
• ✅ Troubleshooting guide
• ✅ Integration examples

🔗 Integration

This implementation seamlessly integrates with:

• ✅ Existing JuliaOS agent system
• ✅ HTTP API infrastructure
• ✅ Configuration management
• ✅ Error handling and logging
• ✅ Cross-platform deployment

🎉 Benefits

1. Decentralized Storage: Agents can now store data on IPFS and Arweave
2. Scalability: Multiple storage backends for different use cases
3. CLI Access: Complete command-line interface for storage operations
4. Agent Integration: Built-in tools for agent file operations
5. Runtime Switching: Dynamic provider switching without restart
6. Metadata Support: Rich metadata handling across all providers
7. Error Resilience: Comprehensive error handling and fallbacks

🔄 Migration Path

Existing local storage continues to work unchanged. New features are additive and backward-compatible.

---

Closes #112

Testing Instructions:

1. Run julia --project=julia julia/test/storage_test.jl for storage tests
2. Run julia --project=julia julia/test/cli_test.jl for CLI tests
3. Try julia --project=julia julia/examples/storage_demo.jl for demo
4. Test CLI: julia --project=julia julia/apps/cli.jl storage list-providers

---

https://docs.juliaos.com/technical/concepts/storage