Skip to content

Commit 15a1cb7

Browse files
endomorphosisendomorphosis
committed
Add comprehensive testing and validation scripts for IPFS Datasets MCP Tools
- Implemented a simple test runner (test_mcp_runner.py) to execute MCP tool tests without pytest. - Created a setup verification script (test_mcp_setup.py) to check MCP server configuration and imports. - Developed a comprehensive test suite (test_mcp_tools_comprehensive.py) to validate all MCP tools functionality. - Added a multiple tools testing script (test_multiple_tools.py) to verify functionality across various tool categories. - Introduced a validation script (validate_tools.py) to ensure all MCP tools can be imported and are functional. - Created a configuration verification script (verify_mcp_config.py) to check MCP configurations and server status.
1 parent a40784e commit 15a1cb7

File tree

124 files changed

+12066
-1269
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

124 files changed

+12066
-1269
lines changed

‎.vscode/launch.json

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
{
2+
"version": "0.2.0",
3+
"configurations": [
4+
{
5+
"name": "Debug MCP Tools Test",
6+
"type": "debugpy",
7+
"request": "launch",
8+
"program": "${workspaceFolder}/comprehensive_mcp_test.py",
9+
"console": "integratedTerminal",
10+
"python": "${workspaceFolder}/.venv/bin/python",
11+
"cwd": "${workspaceFolder}",
12+
"env": {
13+
"PYTHONPATH": "${workspaceFolder}"
14+
}
15+
},
16+
{
17+
"name": "Debug MCP Server",
18+
"type": "debugpy",
19+
"request": "launch",
20+
"module": "ipfs_datasets_py.mcp_server",
21+
"console": "integratedTerminal",
22+
"python": "${workspaceFolder}/.venv/bin/python",
23+
"cwd": "${workspaceFolder}",
24+
"env": {
25+
"PYTHONPATH": "${workspaceFolder}"
26+
}
27+
},
28+
{
29+
"name": "Debug Individual MCP Tool",
30+
"type": "debugpy",
31+
"request": "launch",
32+
"program": "${workspaceFolder}/debug_tool.py",
33+
"console": "integratedTerminal",
34+
"python": "${workspaceFolder}/.venv/bin/python",
35+
"cwd": "${workspaceFolder}",
36+
"env": {
37+
"PYTHONPATH": "${workspaceFolder}"
38+
}
39+
}
40+
]
41+
}

‎.vscode/mcp_config.json

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"mcpServers": {
3+
"ipfs-datasets": {
4+
"command": "python",
5+
"args": ["-m", "ipfs_datasets_py.mcp_server"],
6+
"cwd": "/home/barberb/ipfs_datasets_py",
7+
"env": {
8+
"PYTHONPATH": "/home/barberb/ipfs_datasets_py",
9+
"IPFS_DATASETS_CONFIG": "/home/barberb/ipfs_datasets_py/config/mcp_config.yaml"
10+
}
11+
}
12+
}
13+
}

‎.vscode/settings.json

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
{
2+
"python.defaultInterpreterPath": "./.venv/bin/python",
3+
"python.terminal.activateEnvironment": true,
4+
"python.terminal.activateEnvInCurrentTerminal": true,
5+
"mcp.servers": {
6+
"ipfs-datasets": {
7+
"command": "python",
8+
"args": ["-m", "ipfs_datasets_py.mcp_server", "--host", "127.0.0.1", "--port", "8000"],
9+
"cwd": "/home/barberb/ipfs_datasets_py",
10+
"env": {
11+
"PYTHONPATH": "/home/barberb/ipfs_datasets_py",
12+
"IPFS_DATASETS_CONFIG": "/home/barberb/ipfs_datasets_py/config/mcp_config.yaml"
13+
}
14+
}
15+
},
16+
"copilot-mcp.servers": {
17+
"ipfs-datasets": {
18+
"command": "python",
19+
"args": ["-m", "ipfs_datasets_py.mcp_server", "--host", "127.0.0.1", "--port", "8000"],
20+
"cwd": "/home/barberb/ipfs_datasets_py",
21+
"env": {
22+
"PYTHONPATH": "/home/barberb/ipfs_datasets_py",
23+
"IPFS_DATASETS_CONFIG": "/home/barberb/ipfs_datasets_py/config/mcp_config.yaml"
24+
}
25+
}
26+
}
27+
}

‎.vscode/tasks.json

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
{
2+
"version": "2.0.0",
3+
"tasks": [
4+
{
5+
"label": "Run MCP Tools Test",
6+
"type": "shell",
7+
"command": "${workspaceFolder}/.venv/bin/python",
8+
"args": [
9+
"comprehensive_mcp_test.py"
10+
],
11+
"group": "test",
12+
"isBackground": false,
13+
"problemMatcher": [],
14+
"options": {
15+
"cwd": "${workspaceFolder}"
16+
}
17+
},
18+
{
19+
"label": "Install Dependencies",
20+
"type": "shell",
21+
"command": "${workspaceFolder}/.venv/bin/pip",
22+
"args": [
23+
"install",
24+
"-r",
25+
"requirements.txt"
26+
],
27+
"group": "build",
28+
"isBackground": false,
29+
"problemMatcher": [],
30+
"options": {
31+
"cwd": "${workspaceFolder}"
32+
}
33+
},
34+
{
35+
"label": "Test Individual MCP Tool",
36+
"type": "shell",
37+
"command": "${workspaceFolder}/.venv/bin/python",
38+
"args": [
39+
"-c",
40+
"import asyncio; from ipfs_datasets_py.mcp_server.tools.${input:toolCategory}.${input:toolName} import ${input:toolName}; result = asyncio.run(${input:toolName}('test')); print('Result:', result.get('status', 'unknown'))"
41+
],
42+
"group": "test",
43+
"isBackground": false,
44+
"problemMatcher": [],
45+
"options": {
46+
"cwd": "${workspaceFolder}"
47+
}
48+
},
49+
{
50+
"label": "Start MCP Server",
51+
"type": "shell",
52+
"command": "${workspaceFolder}/.venv/bin/python",
53+
"args": [
54+
"-m",
55+
"ipfs_datasets_py.mcp_server",
56+
"--stdio"
57+
],
58+
"group": "build",
59+
"isBackground": true,
60+
"problemMatcher": [],
61+
"options": {
62+
"cwd": "${workspaceFolder}"
63+
}
64+
},
65+
{
66+
"label": "Test Dataset Tools",
67+
"type": "shell",
68+
"command": "${workspaceFolder}/.venv/bin/python",
69+
"args": [
70+
"-c",
71+
"import asyncio; from ipfs_datasets_py.mcp_server.tools.dataset_tools.load_dataset import load_dataset; result = asyncio.run(load_dataset('test_dataset')); print('Load Dataset Result:', result.get('status'))"
72+
],
73+
"group": "test",
74+
"isBackground": false,
75+
"problemMatcher": [],
76+
"options": {
77+
"cwd": "${workspaceFolder}"
78+
}
79+
},
80+
{
81+
"label": "Test IPFS Tools",
82+
"type": "shell",
83+
"command": "${workspaceFolder}/.venv/bin/python",
84+
"args": [
85+
"-c",
86+
"import asyncio; from ipfs_datasets_py.mcp_server.tools.ipfs_tools.get_from_ipfs import get_from_ipfs; result = asyncio.run(get_from_ipfs('QmTest123')); print('IPFS Get Result:', result.get('status'))"
87+
],
88+
"group": "test",
89+
"isBackground": false,
90+
"problemMatcher": [],
91+
"options": {
92+
"cwd": "${workspaceFolder}"
93+
}
94+
},
95+
{
96+
"label": "Test Vector Tools",
97+
"type": "shell",
98+
"command": "${workspaceFolder}/.venv/bin/python",
99+
"args": [
100+
"-c",
101+
"import asyncio; from ipfs_datasets_py.mcp_server.tools.vector_tools.create_vector_index import create_vector_index; result = asyncio.run(create_vector_index('test_index', [{'text': 'test', 'metadata': {}}])); print('Vector Index Result:', result.get('status'))"
102+
],
103+
"group": "test",
104+
"isBackground": false,
105+
"problemMatcher": [],
106+
"options": {
107+
"cwd": "${workspaceFolder}"
108+
}
109+
},
110+
{
111+
"label": "Test Audit Tools",
112+
"type": "shell",
113+
"command": "${workspaceFolder}/.venv/bin/python",
114+
"args": [
115+
"-c",
116+
"import asyncio; from ipfs_datasets_py.mcp_server.tools.audit_tools.generate_audit_report import generate_audit_report; result = asyncio.run(generate_audit_report('comprehensive')); print('Audit Report Result:', result.get('status'))"
117+
],
118+
"group": "test",
119+
"isBackground": false,
120+
"problemMatcher": [],
121+
"options": {
122+
"cwd": "${workspaceFolder}"
123+
}
124+
}
125+
],
126+
"inputs": [
127+
{
128+
"id": "toolCategory",
129+
"description": "Tool category",
130+
"default": "dataset_tools",
131+
"type": "pickString",
132+
"options": [
133+
"dataset_tools",
134+
"ipfs_tools",
135+
"vector_tools",
136+
"graph_tools",
137+
"audit_tools",
138+
"provenance_tools",
139+
"security_tools"
140+
]
141+
},
142+
{
143+
"id": "toolName",
144+
"description": "Tool name",
145+
"default": "load_dataset",
146+
"type": "promptString"
147+
}
148+
]
149+
}

‎=6.30.1

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Requirement already satisfied: protobuf in ./.venv/lib/python3.12/site-packages (5.29.4)

‎FINAL_TEST_REPORT.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# IPFS Datasets MCP Tools - Final Test Report
22

3-
Generated: 2025-05-24T20:25:05.051106
3+
Generated: 2025-05-24T22:57:44.451715
44

55
## Executive Summary
66

‎MCP_CONFIGURATION_SUMMARY.md

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
# VS Code MCP Configuration Summary
2+
3+
## ✅ CONFIGURATION COMPLETED
4+
5+
Your IPFS Datasets MCP server has been successfully configured for VS Code Copilot integration!
6+
7+
### 🎯 What Was Configured
8+
9+
#### 1. VS Code Extensions
10+
- ✅ **Copilot MCP Extension Installed** (`automatalabs.copilot-mcp`)
11+
- This extension enables MCP server integration with GitHub Copilot Chat
12+
- Allows VS Code Copilot to use external MCP tools
13+
14+
#### 2. MCP Server Module
15+
- ✅ **Created `__main__.py`** for module execution
16+
- Enables running server with: `python -m ipfs_datasets_py.mcp_server`
17+
- Supports command-line arguments (--host, --port, --debug)
18+
- Handles fallback to simple server if needed
19+
20+
#### 3. VS Code Settings Configuration
21+
- ✅ **Updated `.vscode/settings.json`** with MCP server settings:
22+
```json
23+
{
24+
"mcp.servers": {
25+
"ipfs-datasets": {
26+
"command": "python",
27+
"args": ["-m", "ipfs_datasets_py.mcp_server", "--host", "127.0.0.1", "--port", "8000"],
28+
"cwd": "/home/barberb/ipfs_datasets_py",
29+
"env": {
30+
"PYTHONPATH": "/home/barberb/ipfs_datasets_py",
31+
"IPFS_DATASETS_CONFIG": "/home/barberb/ipfs_datasets_py/config/mcp_config.yaml"
32+
}
33+
}
34+
},
35+
"copilot-mcp.servers": {
36+
"ipfs-datasets": { /* same configuration */ }
37+
}
38+
}
39+
```
40+
41+
#### 4. Startup Scripts
42+
- ✅ **Created `start_mcp_server.sh`** executable script
43+
- Sets up environment variables
44+
- Activates virtual environment
45+
- Starts MCP server with proper configuration
46+
47+
#### 5. Test & Verification
48+
- ✅ **Created `test_mcp_setup.py`** verification script
49+
- ✅ **Updated `VSCODE_MCP_GUIDE.md`** with complete instructions
50+
51+
### 🚀 How to Use
52+
53+
#### Step 1: Start the MCP Server
54+
```bash
55+
# Option A: Use the startup script
56+
./start_mcp_server.sh
57+
58+
# Option B: Direct command
59+
python -m ipfs_datasets_py.mcp_server --host 127.0.0.1 --port 8000 --debug
60+
```
61+
62+
#### Step 2: Use in VS Code Copilot
63+
1. **Open VS Code** in this workspace
64+
2. **Open Copilot Chat** (Ctrl+Shift+P → "GitHub Copilot: Open Chat")
65+
3. **Start using MCP tools** with natural language:
66+
67+
**Example prompts:**
68+
- "What IPFS dataset tools are available?"
69+
- "Load a dataset from `/path/to/data.json`"
70+
- "Upload this content to IPFS and pin it"
71+
- "Create a vector index for similarity search"
72+
- "Generate an audit report for recent operations"
73+
74+
### 🛠️ Available Tools (21 total)
75+
76+
The MCP server exposes all IPFS dataset tools:
77+
78+
#### 📊 Dataset Management
79+
- `load_dataset` - Load datasets from various sources
80+
- `save_dataset` - Save datasets to storage
81+
- `process_dataset` - Apply operations to datasets
82+
- `convert_dataset_format` - Convert between formats
83+
84+
#### 🗂️ IPFS Operations
85+
- `get_from_ipfs` - Retrieve content from IPFS
86+
- `pin_to_ipfs` - Pin content to IPFS
87+
88+
#### 🔍 Search & Analysis
89+
- `create_vector_index` - Create vector search index
90+
- `search_vector_index` - Search vector index
91+
- `query_knowledge_graph` - Query knowledge graphs
92+
93+
#### 📋 Audit & Reporting
94+
- `record_audit_event` - Record audit events
95+
- `generate_audit_report` - Generate audit reports
96+
- `record_provenance` - Record data provenance
97+
98+
#### 🔒 Security & Access
99+
- `check_access_permission` - Check access permissions
100+
101+
#### 🌐 Web Archive Processing
102+
- `extract_text_from_warc` - Extract text from WARC files
103+
- `extract_metadata_from_warc` - Extract metadata
104+
- `extract_links_from_warc` - Extract links
105+
- `index_warc` - Index WARC files
106+
107+
*Plus additional specialized tools...*
108+
109+
### ✅ Verification Checklist
110+
111+
Before using, verify:
112+
- [ ] VS Code Copilot MCP extension is installed and enabled
113+
- [ ] MCP server can start without errors: `python -m ipfs_datasets_py.mcp_server --help`
114+
- [ ] Virtual environment is activated (if using one)
115+
- [ ] Port 8000 is available
116+
- [ ] VS Code is opened in this workspace directory
117+
118+
### 🔧 Troubleshooting
119+
120+
#### If tools don't appear in Copilot:
121+
1. **Restart VS Code** after configuration changes
122+
2. **Verify MCP server is running** on port 8000
123+
3. **Check VS Code Developer Console** (Help → Toggle Developer Tools)
124+
4. **Ensure extensions are enabled** in Extensions panel
125+
126+
#### If server won't start:
127+
1. **Check Python environment**: `which python`
128+
2. **Verify dependencies**: `pip list | grep mcp`
129+
3. **Test module**: `python -c "import ipfs_datasets_py.mcp_server"`
130+
4. **Check port availability**: `netstat -tulpn | grep 8000`
131+
132+
### 🎉 Ready to Use!
133+
134+
Your IPFS Datasets MCP server is now configured and ready for VS Code Copilot integration.
135+
136+
**Next steps:**
137+
1. Start the MCP server: `./start_mcp_server.sh`
138+
2. Open VS Code Copilot Chat
139+
3. Try: "What IPFS dataset tools are available?"
140+
141+
The 21 MCP tools should now be accessible through natural language in VS Code Copilot! 🚀

0 commit comments

Comments
 (0)