diff --git a/docs/Developer Tools Plugin.md b/docs/Developer Tools Plugin.md new file mode 100644 index 0000000..7d6bfc8 --- /dev/null +++ b/docs/Developer Tools Plugin.md @@ -0,0 +1,717 @@ +# Developer Tools Plugin + +## Overview + +The Developer Tools plugin extends Frappe Assistant Core from a data-layer assistant into a development assistant. It provides generic filesystem operations that allow the AI to create Frappe apps, write code files (Script Reports, custom modules, etc.), and read existing code — all through MCP tools. + +This plugin bridges the gap where users need complex Script Reports, custom server-side logic, or other code artifacts that currently require a developer to write manually. + +### Key Design Principles + +- **Generic tools, not per-doctype tools** — The tools operate on files and directories, not on specific Frappe DocTypes. The AI decides what content to write based on the user's request. +- **Custom apps only** — Write operations are restricted to custom apps. Standard apps (frappe, erpnext, hrms, etc.) are protected from modification. +- **Python APIs, not bench CLI** — All operations use Frappe's Python APIs internally (e.g., `frappe.installer.install_app()`), making the plugin compatible with both self-hosted installations and Frappe Cloud. +- **System Manager only** — All tools require the System Manager role due to the sensitive nature of filesystem operations. +- **Inline syntax validation** — `write_file` automatically validates Python and JSON files after writing and reports errors immediately so the AI can self-correct. + +--- + +## Plugin Architecture + +``` +plugins/developer_tools/ +├── __init__.py +├── plugin.py # DeveloperToolsPlugin(BasePlugin) +└── tools/ + ├── __init__.py # Shared security helpers + ├── ensure_app.py # EnsureApp — create custom Frappe apps + ├── write_file.py # WriteFile — write files to custom apps (with syntax validation) + ├── read_file.py # ReadFile — read files from any app + ├── list_app_files.py # ListAppFiles — browse app directories + └── describe_app.py # DescribeApp — full app structure tree for LLM context +``` + +### Tools Summary + +| Tool | Purpose | Read/Write | Protected App Check | +| :---- | :---- | :---- | :---- | +| `ensure_app` | Create a new Frappe app non-interactively | Write | Yes — cannot overwrite standard apps | +| `write_file` | Write content to a file in a custom app (validates syntax) | Write | Yes — blocks writes to standard apps | +| `read_file` | Read a file from any Frappe app | Read | No — reading standard apps is allowed | +| `list_app_files` | List files/directories in any app | Read | No — listing standard apps is allowed | +| `describe_app` | Get complete app structure tree with context | Read | No — can describe any app | + +--- + +## Deployment Strategies: Self-Hosted vs Frappe Cloud + +### The Problem with Fra ppe Cloud + +Frappe Cloud uses Docker containers. When you upgrade or redeploy a bench: + +- A **new container** is built from the `apps.json` manifest +- Only apps listed in `apps.json` (with git repos) survive +- Any app created locally on the filesystem **gets wiped** + +This means `ensure_app` creating a local-only app works perfectly for self-hosted but is **ephemeral on Frappe Cloud**. + +### Two-Strategy Approach + +The Developer Tools plugin uses a **dual strategy** based on the deployment environment: + +#### Strategy 1: Filesystem (Self-Hosted) — Full Capability + +On self-hosted installations, the tools write directly to the filesystem: + +``` +ensure_app → write_file → bench migrate → Standard Script Report on disk +``` + +**Advantages:** + +- Full Python capabilities (unrestricted imports, any library) +- Better performance (compiled once, cached) +- Version-controllable via git +- Full Frappe module system integration + +**Use when:** Self-hosted, development environments, or when the user has a git repo backing the custom app. + +#### Strategy 2: Database (Frappe Cloud) — Persistent but Restricted + +On Frappe Cloud (or when persistence across rebuilds is required), the AI should use existing FAC tools to create **database-stored artifacts** instead: + +| Artifact | Filesystem Tool | Database Alternative | Survives Rebuild? | +| :---- | :---- | :---- | :---- | +| Script Report | `write_file` (4 files) | `create_document` (Report DocType, `is_standard="No"`) | DB: Yes, FS: No | +| Server Script | `write_file` (.py) | `create_document` (Server Script DocType) | DB: Yes, FS: No | +| Client Script | `write_file` (.js) | `create_document` (Client Script DocType) | DB: Yes, FS: No | +| Print Format | `write_file` (.html) | `create_document` (Print Format, `custom_format=1`) | DB: Yes, FS: No | +| Web Form scripts | `write_file` | `create_document` (Web Form, `client_script` field) | DB: Yes, FS: No | + +**Database-stored code limitations:** + +- Executed via `safe_exec()` with RestrictedPython — no arbitrary imports +- Only Frappe-whitelisted utilities available (frappe.db, frappe.utils, json, etc.) +- Slower execution (compiled on each run, not cached) +- No access to external Python libraries + +**Use when:** Frappe Cloud, production environments where persistence matters more than capability. + +### How the AI Should Decide + +The AI should ask or detect the environment: + +``` +User: "Create a Script Report for customer outstanding" + +AI decision tree: +├─ Is this self-hosted / development? +│ └─ YES → Use filesystem tools (ensure_app + write_file) +│ Full Python, better performance, git-trackable +│ +└─ Is this Frappe Cloud / needs to survive rebuilds? + └─ YES → Does the report need external libraries or complex imports? + ├─ NO → Use create_document (Report DocType, is_standard="No") + │ Stored in DB, persists across rebuilds + └─ YES → Use filesystem tools BUT warn the user: + "This report uses libraries not available in safe_exec. + It will work now but won't survive a Frappe Cloud rebuild. + Consider backing it up to a git repo." +``` + +### Future: Git-Backed Custom App + +The ideal long-term solution for Frappe Cloud is for `ensure_app` to: + +1. Create the app locally +2. Initialize a git repository +3. Push to a configured remote (GitHub/GitLab) +4. Add the repo URL to the bench's `apps.json` + +This way the custom app survives rebuilds because Frappe Cloud pulls it from the git repo. This is not in the initial scope but is the natural evolution. + +--- + +## Security Model + +### Path Sandboxing + +All file paths accepted by the tools are **relative to the bench `apps/` directory**. The tool resolves the full path internally and validates it: + +``` +User provides: "fac_custom_code/fac_custom_code/my_module/report/my_report/my_report.py" +Tool resolves: "/home/user/frappe-bench/apps/fac_custom_code/fac_custom_code/my_module/report/my_report/my_report.py" +``` + +**Security checks applied to every path:** + +1. **Symlink resolution** — Uses `os.path.realpath()` (not `os.path.abspath()`) to resolve symlinks. This prevents symlink-escape attacks where a symlink inside `apps/` points to `/etc/` or another sensitive directory. +2. **Boundary check** — The resolved path must start with the `apps/` directory. Any path traversal (e.g., `../../etc/passwd`) is rejected. +3. **Null byte rejection** — Paths containing null bytes are rejected to prevent null byte injection. +4. **Depth limit** — Paths with more than 10 directory levels are rejected as a sanity check. + +### Protected App Blocklist + +Write operations (`write_file`, `ensure_app`) check the target app name against a blocklist of standard Frappe ecosystem apps: + +```py +PROTECTED_APPS = {"frappe", "erpnext", "hrms", "payments", "india_compliance", "lending", "education"} +``` + +Read operations (`read_file`, `list_app_files`, `describe_app`) do **not** enforce this blocklist. The AI needs to read standard app code to understand patterns and generate correct code. + +### Permission Model + +All Developer Tools require the **System Manager** role. This is enforced via an explicit role check at the start of each tool's `execute()` method, rather than through the `requires_permission` DocType mechanism (since there is no single DocType to gate on). + +| Tool | Required Role | Rationale | +| :---- | :---- | :---- | +| `ensure_app` | System Manager | Creates apps, modifies apps.txt, runs install\_app | +| `write_file` | System Manager | Writes to filesystem | +| `read_file` | System Manager | Reads source code from filesystem | +| `list_app_files` | System Manager | Exposes filesystem structure | +| `describe_app` | System Manager | Exposes full app structure | + +--- + +## Tool Details + +### 1\. `ensure_app` — Create a Custom Frappe App + +Creates a new Frappe app non-interactively if it doesn't already exist, and installs it on the current site. This is the first tool the AI should call before writing any code files. + +#### How It Works + +The standard `bench new-app` command is interactive — it prompts for app title, publisher, email, etc. The `ensure_app` tool bypasses this by: + +1. **Scaffolding the app directory structure programmatically** — It reuses the template strings from `frappe.utils.boilerplate` (`hooks_template`, `init_template`, `pyproject_template`, `patches_template`) but fills in the values from the tool's input parameters instead of interactive prompts. + +2. **Registering the app with Frappe** — After creating the files: + + - Adds the app name to `sites/apps.txt` (how Bench discovers apps) + - Calls `frappe.installer.install_app(app_name)` which: + - Creates Module Def documents in the database + - Syncs DocTypes from JSON files + - Marks all patches as complete + - Runs any `after_install` hooks + +#### Directory Structure Created + +``` +apps/{app_name}/ +├── {app_name}/ +│ ├── __init__.py # __version__ = "0.0.1" +│ ├── hooks.py # App metadata and configuration +│ ├── modules.txt # Module list (one entry: app title) +│ ├── patches.txt # Migration patches (empty sections) +│ ├── {app_title_scrubbed}/ # Default module directory +│ │ └── __init__.py +│ ├── templates/ +│ │ ├── __init__.py +│ │ └── pages/ +│ │ └── __init__.py +│ ├── templates/includes/ +│ ├── config/ +│ │ └── __init__.py +│ ├── public/ +│ │ ├── css/ +│ │ ├── js/ +│ │ └── .gitkeep +│ └── www/ +└── pyproject.toml # Python package metadata +``` + +#### Input Parameters + +| Parameter | Type | Default | Description | +| :---- | :---- | :---- | :---- | +| `app_name` | string | `"fac_custom_code"` | Snake\_case app name. Must match `^[a-z][a-z0-9_]*$` | +| `app_title` | string | Title-cased app\_name | Human-readable title | +| `app_description` | string | `"Custom code generated by Frappe Assistant"` | App description | + +#### Return Value + +```json +{ + "success": true, + "app_name": "fac_custom_code", + "app_path": "/home/user/frappe-bench/apps/fac_custom_code", + "already_existed": false, + "message": "App 'fac_custom_code' created and installed on site" +} +``` + +If the app already exists and is valid: + +```json +{ + "success": true, + "app_name": "fac_custom_code", + "already_existed": true, + "message": "App 'fac_custom_code' already exists" +} +``` + +#### Why a Default App Name? + +The default app name `fac_custom_code` provides a single, predictable location for all AI-generated code. This simplifies: + +- **Discovery** — Users know where to find AI-generated code +- **Maintenance** — One app to manage, update, or remove +- **Backup** — One directory to back up for custom code + +--- + +### 2\. `write_file` — Write Code to the Filesystem + +The core tool of the plugin. Writes content to a file within a custom Frappe app's directory structure. Creates intermediate directories as needed. **Automatically validates syntax for Python and JSON files.** + +#### How It Works + +1. Validates the path is within `apps/` and not targeting a protected app +2. Creates any missing parent directories (e.g., for `report/my_report/my_report.py`, creates the `report/my_report/` directories) +3. Writes the content to the file +4. Sets appropriate file permissions (0o644 for files, 0o755 for directories) +5. **Runs syntax validation** based on file extension (see below) + +#### Inline Syntax Validation + +After writing the file, `write_file` automatically validates its syntax based on the file extension. The file is **always written** (so the AI can inspect and fix it), but the validation result is included in the response. + +| Extension | Validation Method | What It Catches | +| :---- | :---- | :---- | +| `.py` | `ast.parse(content)` | Syntax errors, indentation errors, invalid Python | +| `.json` | `json.loads(content)` | Malformed JSON, trailing commas, missing brackets | +| `.js` | *None (deferred)* | JS errors surface in the browser console | +| Other | *None* | No validation for HTML, CSS, txt, etc. | + +**Why `ast.parse()` and not `py_compile`?** + +- `ast.parse()` is pure parsing — it catches syntax errors without needing to write a `.pyc` file +- It works on content strings directly (no temp file needed) +- It's part of Python's stdlib with zero overhead + +**Example response with validation error:** + +```json +{ + "success": true, + "file_path": "fac_custom_code/fac_custom_code/reports/report/my_report/my_report.py", + "bytes_written": 1234, + "created_new": true, + "validation": { + "valid": false, + "language": "python", + "error": "SyntaxError: unexpected indent (line 42, col 8)", + "line": 42, + "col": 8 + } +} +``` + +**Example response when validation passes:** + +```json +{ + "success": true, + "file_path": "...", + "bytes_written": 1234, + "created_new": true, + "validation": { + "valid": true, + "language": "python" + } +} +``` + +The AI can then use `read_file` to see the exact file content and `write_file` again with corrected code. This creates a self-correcting loop without human intervention. + +#### Why No Separate Update Tool? + +Updating an existing file uses the same `read_file` → `write_file(overwrite=true)` flow: + +1. AI calls `read_file` to get current content +2. AI modifies the content in its context +3. AI calls `write_file` with `overwrite=true` to write the full file back + +This is the same pattern used by every AI coding assistant (Cursor, Claude Code, GitHub Copilot). A separate "update" or "patch" tool adds complexity without meaningful benefit. If large-file partial edits become a bottleneck later, a `patch_file` tool (old\_string/new\_string replacement) can be added. + +#### Input Parameters + +| Parameter | Type | Default | Description | +| :---- | :---- | :---- | :---- | +| `file_path` | string | *required* | Path relative to `apps/` directory | +| `content` | string | *required* | File content to write (max 1MB) | +| `overwrite` | boolean | `true` | Whether to overwrite existing files | + +#### Return Value + +```json +{ + "success": true, + "file_path": "fac_custom_code/fac_custom_code/custom_reports/report/my_report/my_report.py", + "bytes_written": 1234, + "created_new": true, + "validation": { + "valid": true, + "language": "python" + } +} +``` + +#### Example: Creating a Script Report + +A Script Report requires 4 files. The AI would call `write_file` four times: + +**1\. Report JSON metadata:** + +``` +file_path: "fac_custom_code/fac_custom_code/custom_reports/report/sales_summary/sales_summary.json" +content: { + "doctype": "Report", + "name": "Sales Summary", + "report_name": "Sales Summary", + "ref_doctype": "Sales Invoice", + "report_type": "Script Report", + "is_standard": "Yes", + "module": "Custom Reports", + "roles": [{"role": "Accounts User"}] +} +``` + +**2\. Python controller:** + +``` +file_path: "fac_custom_code/fac_custom_code/custom_reports/report/sales_summary/sales_summary.py" +content: + import frappe + + def execute(filters=None): + columns = [ + {"label": "Customer", "fieldname": "customer", "fieldtype": "Link", "options": "Customer", "width": 200}, + {"label": "Total", "fieldname": "total", "fieldtype": "Currency", "width": 150} + ] + data = frappe.db.sql(""" + SELECT customer, SUM(grand_total) as total + FROM `tabSales Invoice` + WHERE docstatus = 1 + GROUP BY customer + ORDER BY total DESC + """, as_dict=True) + return columns, data +``` + +**3\. JavaScript filters:** + +``` +file_path: "fac_custom_code/fac_custom_code/custom_reports/report/sales_summary/sales_summary.js" +content: + frappe.query_reports["Sales Summary"] = { + filters: [ + { + fieldname: "from_date", + label: __("From Date"), + fieldtype: "Date", + default: frappe.datetime.add_months(frappe.datetime.get_today(), -1) + }, + { + fieldname: "to_date", + label: __("To Date"), + fieldtype: "Date", + default: frappe.datetime.get_today() + } + ] + }; +``` + +**4\. Python init file:** + +``` +file_path: "fac_custom_code/fac_custom_code/custom_reports/report/sales_summary/__init__.py" +content: "" (empty) +``` + +After writing these files, the user (or a future automation) would run `bench migrate` or clear cache to make the report visible in Frappe. + +--- + +### 3\. `read_file` — Read Code from the Filesystem + +Reads file contents from any Frappe app's directory. Essential for the AI to inspect existing code patterns before generating new code. + +#### How It Works + +1. Validates the path is within `apps/` (but does NOT check the protected app blocklist — reading is safe) +2. Verifies the target is a regular file (not a directory, device, or socket) +3. Checks file size (max 5MB) +4. Reads and returns content, truncated to `max_lines` if the file is large + +#### Input Parameters + +| Parameter | Type | Default | Description | +| :---- | :---- | :---- | :---- | +| `file_path` | string | *required* | Path relative to `apps/` directory | +| `max_lines` | integer | `500` | Maximum number of lines to return | + +#### Return Value + +```json +{ + "success": true, + "file_path": "erpnext/erpnext/stock/report/stock_analytics/stock_analytics.py", + "content": "import frappe\n...", + "lines": 150, + "truncated": false, + "size_bytes": 4521 +} +``` + +#### Typical Use Case + +The AI reads an existing Script Report from erpnext to understand the pattern, then generates a similar report for the user's custom requirement: + +``` +Step 1: read_file("erpnext/erpnext/stock/report/stock_analytics/stock_analytics.py") +Step 2: read_file("erpnext/erpnext/stock/report/stock_analytics/stock_analytics.js") +Step 3: Understand the pattern +Step 4: write_file(...) to create a new report following the same pattern +``` + +--- + +### 4\. `list_app_files` — Browse App Directory Structure + +Lists files and directories within a Frappe app. Helps the AI understand the existing structure before creating new files. + +#### How It Works + +1. Validates the path is within `apps/` +2. Lists directory contents using `os.listdir()` or `os.walk()` (for recursive listing) +3. Filters out noise directories: `__pycache__`, `.git`, `node_modules`, `.egg-info` +4. Optionally filters by glob pattern (e.g., `*.py`) + +#### Input Parameters + +| Parameter | Type | Default | Description | +| :---- | :---- | :---- | :---- | +| `path` | string | `""` | Directory path relative to `apps/`. Empty \= list all apps | +| `pattern` | string | *none* | Glob pattern filter (e.g., `"*.py"`, `"*.json"`) | +| `recursive` | boolean | `false` | List recursively (max 5 levels deep) | +| `max_results` | integer | `200` | Maximum entries to return | + +#### Return Value + +```json +{ + "success": true, + "path": "fac_custom_code/fac_custom_code", + "entries": [ + {"name": "__init__.py", "type": "file", "size": 42}, + {"name": "hooks.py", "type": "file", "size": 1024}, + {"name": "custom_reports", "type": "dir", "size": 0}, + {"name": "public", "type": "dir", "size": 0} + ], + "total": 4, + "truncated": false +} +``` + +--- + +### 5\. `describe_app` — Full App Structure Tree for LLM Context + +Provides the AI with a complete, structured view of a Frappe app's directory tree — similar to the Unix `tree` command but optimized for LLM consumption. This is the tool the AI should call first when it needs to understand an app's layout before making changes. + +#### Why This Tool Exists + +`list_app_files` is good for browsing one directory at a time, but the AI often needs a **full picture** of an app's structure in a single call — which modules exist, where reports live, what DocTypes are defined, etc. Without this, the AI would need multiple sequential `list_app_files` calls, wasting tool calls and context. + +`describe_app` gives the AI everything it needs in one shot to: + +- Know where to place new files (correct module, correct subdirectory) +- Avoid creating duplicate modules or reports +- Understand the app's conventions (naming patterns, module organization) +- Provide informed suggestions ("You already have a `reports` module — should I add the new report there?") + +#### How It Works + +1. Validates the app exists in `apps/` +2. Walks the app's directory tree (up to configurable depth) +3. Filters out noise: `__pycache__`, `.git`, `node_modules`, `.egg-info`, `.pyc` files +4. Generates a hierarchical tree structure +5. Optionally annotates files with metadata (size, type classification) +6. Parses `modules.txt` to identify registered modules +7. Detects Frappe artifacts (DocTypes, Reports, Pages) from directory conventions + +#### Input Parameters + +| Parameter | Type | Default | Description | +| :---- | :---- | :---- | :---- | +| `app_name` | string | *required* | App name (e.g., `"fac_custom_code"`, `"erpnext"`) | +| `max_depth` | integer | `4` | Maximum directory depth to traverse | +| `include_metadata` | boolean | `true` | Include file sizes and artifact type annotations | + +#### Return Value + +```json +{ + "success": true, + "app_name": "fac_custom_code", + "app_title": "Fac Custom Code", + "modules": ["Fac Custom Code"], + "tree": "fac_custom_code/\n├── __init__.py\n├── hooks.py\n├── modules.txt\n├── patches.txt\n├── fac_custom_code/\n│ ├── __init__.py\n│ └── report/\n│ └── sales_summary/\n│ ├── __init__.py\n│ ├── sales_summary.json [Report]\n│ ├── sales_summary.py [Script Report Controller]\n│ └── sales_summary.js [Report Filters]\n├── public/\n│ └── .gitkeep\n└── templates/\n └── pages/", + "summary": { + "modules": 1, + "doctypes": 0, + "reports": 1, + "pages": 0, + "total_files": 12 + } +} +``` + +#### Artifact Detection + +The tool recognizes Frappe directory conventions and annotates them: + +| Directory Pattern | Detected As | Annotation | +| :---- | :---- | :---- | +| `*/doctype/*/` | DocType | `[DocType]` on .json, `[Controller]` on .py | +| `*/report/*/` | Report | `[Report]` on .json, `[Script Report Controller]` on .py | +| `*/page/*/` | Page | `[Page]` on .json | +| `*/print_format/*/` | Print Format | `[Print Format]` on .html | +| `*/workspace/` | Workspace | `[Workspace]` on .json | + +#### Example Output (Tree Format) + +``` +fac_custom_code/ +├── __init__.py (27 B) +├── hooks.py (1.2 KB) +├── modules.txt (16 B) +├── patches.txt (89 B) +├── fac_custom_code/ ── Module: "Fac Custom Code" +│ ├── __init__.py +│ ├── doctype/ +│ │ └── custom_setting/ +│ │ ├── custom_setting.json [DocType] +│ │ ├── custom_setting.py [Controller] +│ │ └── __init__.py +│ └── report/ +│ ├── sales_summary/ +│ │ ├── sales_summary.json [Report] +│ │ ├── sales_summary.py [Script Report Controller] +│ │ ├── sales_summary.js [Report Filters] +│ │ └── __init__.py +│ └── customer_aging/ +│ ├── customer_aging.json [Report] +│ ├── customer_aging.py [Script Report Controller] +│ ├── customer_aging.js [Report Filters] +│ └── __init__.py +├── config/ +│ └── __init__.py +├── public/ +│ ├── css/ +│ ├── js/ +│ └── .gitkeep +└── templates/ + ├── __init__.py + └── pages/ + └── __init__.py +``` + +#### Comparison: `describe_app` vs `list_app_files` + +| Aspect | `list_app_files` | `describe_app` | +| :---- | :---- | :---- | +| Scope | Single directory | Full app tree | +| Output | Flat list of entries | Hierarchical tree \+ annotations | +| Context | Browsing, searching | Understanding full app structure | +| Use case | "What's in this folder?" | "Show me the whole app layout" | +| When to use | Known path, looking for specific files | Starting a new task, need orientation | + +--- + +## End-to-End Workflow + +Here's the complete flow when a user asks the AI to create a Script Report: + +``` +User: "Create a Script Report that shows top customers by outstanding amount" + +AI workflow: +│ +├─ 1. ensure_app(app_name="fac_custom_code") +│ → App created (or already exists) +│ +├─ 2. describe_app(app_name="fac_custom_code") +│ → Full tree: see modules, existing reports, understand structure +│ +├─ 3. read_file("erpnext/erpnext/accounts/report/accounts_receivable/accounts_receivable.py") +│ → Study a similar existing report for patterns +│ +├─ 4. write_file(".../__init__.py", content="") +│ → validation: {valid: true} +│ +├─ 5. write_file(".../top_customers_outstanding.json", content="{...}") +│ → validation: {valid: true, language: "json"} +│ +├─ 6. write_file(".../top_customers_outstanding.py", content="def execute(...)...") +│ → validation: {valid: true, language: "python"} +│ (If syntax error: AI reads error, fixes code, writes again) +│ +├─ 7. write_file(".../top_customers_outstanding.js", content="frappe.query_reports[...]...") +│ → No JS validation (errors will surface in browser) +│ +└─ 8. AI tells user: "Report created. Run 'bench migrate' or clear cache to see it." +``` + +--- + +## Frappe Cloud Compatibility + +### Runtime Compatibility + +The Developer Tools plugin is designed to run on Frappe Cloud: + +- **No `bench` CLI calls** — All operations use Frappe's Python APIs (`frappe.installer.install_app()`, `frappe.clear_cache()`, etc.) +- **No subprocess execution** — File operations use Python's built-in `os` and `io` modules +- **Standard file permissions** — Files are created with 0o644 (readable by the web server) + +### Persistence Limitations + +On Frappe Cloud, filesystem-created apps and files **do not survive bench upgrades/redeployments** (new Docker container \= clean slate). See the [Deployment Strategies](#deployment-strategies-self-hosted-vs-frappe-cloud) section above for the dual approach. + +For Frappe Cloud users who need persistent, complex reports (requiring full Python imports), the recommended path is: + +1. Use Developer Tools to create and test locally +2. Push the custom app to a git repository +3. Add the git repo URL to the Frappe Cloud bench configuration +4. The app will then survive redeployments + +--- + +## Future Considerations + +### Potential Additional Tools + +These tools are not in the initial scope but could be added later: + +- **`delete_file`** — Remove a file from a custom app (with safety confirmations) +- **`create_module`** — Create a new Frappe module within an app (module directory \+ Module Def) +- **`patch_file`** — Surgical old\_string/new\_string replacements for editing large files without rewriting them entirely +- **`sync_app`** — Call `frappe.modules.utils.sync_for()` directly to sync DocTypes without a full migrate + +### Beyond Script Reports + +The same generic tools support creating: + +- **Print Formats** — Custom print format HTML/CSS/Jinja files +- **Workspace pages** — Custom workspace JSON definitions +- **Custom Python modules** — Utility functions, API endpoints +- **Fixtures** — Data fixtures for seeding/testing +- **Patches** — Database migration patches +- **Page files** — Custom Frappe pages with HTML/JS/CSS +- **DocTypes** — Custom DocType JSON definitions with controllers + +The tools are intentionally generic so that any file-based Frappe artifact can be created without needing a dedicated tool. diff --git a/docs/skills/bench_tools.md b/docs/skills/bench_tools.md new file mode 100644 index 0000000..6f82f46 --- /dev/null +++ b/docs/skills/bench_tools.md @@ -0,0 +1,278 @@ +--- +name: bench-tools +description: > + Use this skill whenever the user wants to manage Frappe apps on the bench or export + fixtures. Trigger on phrases like: "create an app", "install app", "uninstall app", + "remove app from bench", "list apps", "list sites", "export fixtures", "export custom fields", + "export property setters", "export server scripts", "export client scripts", + "what apps do I have", "what sites do I have". This skill documents the exact workflow + for bench_help and bench_execute tools - always load it before any bench operation. +--- + +# Frappe Bench Tools Skill + +## Overview + +Two MCP tools handle all bench and fixture operations: + +- bench_help - call this FIRST. Returns all available actions and required parameters. +- bench_execute - executes the actual operation with the given action and parameters. + +**Always call `bench_help` before `bench_execute`** if you are unsure what parameters are needed. + +--- + +## Tools and Actions + +### bench_help +No parameters needed. Returns the full list of available actions. + +```python +bench_help() +# Returns all actions with descriptions and required params +``` + +### bench_execute +Single tool that handles all operations via the `action` parameter. + +| Action | Description | Required Params | +|---|---|---| +| `list_apps` | Lists ALL bench apps AND site-installed apps separately | none | +| `list_sites` | Lists all available sites on this bench | none | +| `create_app` | Creates a new custom Frappe app on bench | `app_name` | +| `install_app` | Installs a bench app onto the current site | `app_name` | +| `uninstall_app` | Removes app from site only - keeps files on bench | `app_name` | +| `remove_app` | Permanently deletes app from bench entirely | `app_name` | +| `export_fixtures` | Exports DocType customizations as fixture JSON to an app | `app_name`, `doctype`, `filters` | + +--- + +## Critical Rules - Read Before Every Operation + +### Rule 1 - list_apps returns TWO separate lists +4 +```json +{ + "bench_apps": ["app1", "app2", "app3"], - apps that EXIST on disk + "site_apps": ["app1"] - apps INSTALLED on current site +} +``` + +An app can exist on bench but NOT be installed on site. Always show both lists clearly to the user. + +### Rule 2 - export_fixtures requires ALL THREE parameters + +Tool **fails immediately** if any are missing: +- `app_name` - which app to write fixtures into +- `doctype` - which DocType to export (must be from allowed list below) +- `filters` - which specific records to export (must be specific - never export everything) + +**Never call export_fixtures without all three.** Ask user for missing params before calling. + +### Rule 3 - Allowed DocTypes for export_fixtures only + +| DocType | Filter Fields | +|---|---| +| `Custom Field` | `dt` | +| `Property Setter` | `doc_type`, `field_name` | +| `Client Script` | `dt`, `module` | +| `Server Script` | `name`, `module` | +| `Role` | `name` | +| `Workflow` | `document_type` | +| `Print Format` | `doc_type`, `module` | +| `Notification` | `document_type`, `module` | + +If user asks to export any other DocType - inform them it is not supported. + +### Rule 4 - remove_app handles FULL cleanup automatically + +`remove_app` does everything in order: +1. Uninstalls from site (`frappe.installer.remove_app`) +2. Removes from DB installed_apps +3. pip uninstalls the package +4. Removes from `sites/apps.txt` +5. Deletes the app directory + +**No need to call `uninstall_app` before `remove_app`** - it handles everything. Always show confirmation to user before proceeding. + +### Rule 5 - create_app does NOT install on site + +`create_app` only scaffolds the app directory on bench. It does NOT install it on the site. +After `create_app`, user must explicitly call `install_app` if they want it active on site. + +### Rule 6 - Protected apps cannot be touched + +These apps are protected and will be blocked: +`frappe`, `erpnext`, `hrms`, `payments`, `frappe_assistant_core` + +--- + +## Exact Filter Formats by DocType + +Always use the most specific filters possible. The more specific the filter, the fewer unnecessary records get exported. + +### Custom Field +```json +{"dt": "Sales Invoice"} +{"dt": "Purchase Order"} +``` + +### Property Setter +```json +{"doc_type": "Sales Invoice"} +{"doc_type": "Sales Invoice", "field_name": "max_discount"} +``` + +### Client Script +```json +{"dt": "Sales Invoice"} +{"module": "Accounts"} +``` + +### Server Script +```json +{"name": ["in", ["get_reqDate_MR_SPQ", "set_default_time"]]} +{"module": "Accounts"} +``` + +### Role +```json +{"name": ["in", ["Accounts Manager", "Stock Manager"]]} +``` + +### Workflow +```json +{"document_type": "Purchase Order"} +``` + +### Print Format +```json +{"doc_type": "Sales Invoice"} +{"doc_type": "Sales Invoice", "module": "Accounts"} +``` + +### Notification +```json +{"document_type": "Sales Invoice"} +{"module": "Accounts"} +``` + +--- + +## Step-by-Step Workflows + +### Check what apps exist on bench vs site +```python +bench_execute(action="list_apps") +# bench_apps = apps that exist on disk +# site_apps = apps installed and active on current site +``` + +### Check available sites +```python +bench_execute(action="list_sites") +# Returns all sites that have a site_config.json +``` + +### Create a new app (bench only, not installed) +```python +bench_execute(action="create_app", app_name="my_custom_app") +# App is created on bench but NOT installed on site +``` + +### Create app and install on site (two steps) +```python +bench_execute(action="create_app", app_name="my_custom_app") +bench_execute(action="install_app", app_name="my_custom_app") +``` + +### Install an existing bench app onto site +```python +bench_execute(action="install_app", app_name="my_custom_app") +``` + +### Uninstall app from site (keep files on bench) +```python +bench_execute(action="uninstall_app", app_name="my_custom_app") +# App files still exist on bench - can reinstall anytime +``` + +### Remove app completely from bench +```python +bench_execute(action="remove_app", app_name="my_custom_app") +# Uninstalls from site + pip uninstall + deletes directory - PERMANENT +``` + +### Export Custom Field fixtures +```python +bench_execute( + action="export_fixtures", + app_name="my_custom_app", + doctype="Custom Field", + filters={"dt": "Sales Invoice"} +) +``` + +### Export Server Script fixtures by name +```python +bench_execute( + action="export_fixtures", + app_name="my_custom_app", + doctype="Server Script", + filters={"name": ["in", ["script_one", "script_two"]]} +) +``` + +### Create app then export fixtures (single user request) +```python +# Step 1 +bench_execute(action="create_app", app_name="my_custom_app") +# Step 2 +bench_execute( + action="export_fixtures", + app_name="my_custom_app", + doctype="Custom Field", + filters={"dt": "Sales Invoice"} +) +``` + +--- + +## Interaction Rules + +1. **"What apps do I have"** - call `list_apps`, show bench_apps and site_apps separately with clear labels +2. **"What sites do I have"** - call `list_sites` +3. **"Create app" without name** - ask for app_name before calling tool +4. **"Install app" without specifying which** - call `list_apps` first, show bench_apps that are NOT in site_apps, ask user to pick +5. **"Export fixtures" without details** - ask for app_name, doctype, and filters before calling - do NOT call tool without all three +6. **"Remove app" without specifying which** - call `list_apps` first, show list, ask which one, then show confirmation +7. **"Remove app"** - always show confirmation warning before proceeding - removal is permanent and cannot be undone +8. **Multi-step request** (e.g. "create app and export fixtures to it") - execute steps sequentially without asking for confirmation between steps + +--- + +## Common Errors and Fixes + +### "MISSING REQUIRED PARAMETER: doctype" +**Cause:** Called export_fixtures without specifying DocType. +**Fix:** Always specify doctype. Example: `doctype="Custom Field"` + +### "MISSING REQUIRED PARAMETER: filters" +**Cause:** Called export_fixtures without filters. +**Fix:** Always provide specific filters. Example: `filters={"dt": "Sales Invoice"}` + +### "Cannot create or overwrite protected app" +**Cause:** Tried to create/remove a protected system app. +**Fix:** Only custom apps can be managed. Protected: frappe, erpnext, hrms, etc. + +### "App is not installed on the current site" +**Cause:** Called uninstall_app on an app not currently installed on site. +**Fix:** Check site_apps from list_apps before uninstalling. + +### "No module named 'app_name'" +**Cause:** App was deleted from disk but still registered in DB installed_apps (ghost app). +**Fix:** This is automatically cleaned when list_apps is called next time. + +### "App already exists" +**Cause:** create_app called with a name that already exists on bench. +**Fix:** Tool returns `already_existed: true` and does nothing - safe to continue. diff --git a/frappe_assistant_core/plugins/developer_tools/__init__.py b/frappe_assistant_core/plugins/developer_tools/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/frappe_assistant_core/plugins/developer_tools/plugin.py b/frappe_assistant_core/plugins/developer_tools/plugin.py new file mode 100644 index 0000000..96bb202 --- /dev/null +++ b/frappe_assistant_core/plugins/developer_tools/plugin.py @@ -0,0 +1,63 @@ +# Copyright (C) 2025 Promantia +# Developer Tools Plugin + +from typing import Any, Dict, List, Optional, Tuple + +import frappe +from frappe import _ + +from frappe_assistant_core.plugins.base_plugin import BasePlugin + + +class DeveloperToolsPlugin(BasePlugin): + """ + Plugin providing filesystem tools for creating + Frappe apps and writing code files. + + Tools: + - ensure_app : create a custom Frappe app + - write_file : write code files to a custom app + - read_file : read files from any app + - list_app_files: browse app directory structure + - describe_app : get full app structure tree + """ + + def get_info(self) -> Dict[str, Any]: + return { + "name": "developer_tools", + "display_name": "Developer Tools", + "description": ( + "Filesystem tools for creating Frappe apps, " + "writing Script Reports, and reading existing code." + ), + "version": "1.0.0", + "author": "Promantia", + "dependencies": [], + "requires_restart": False, + } + + def get_tools(self) -> List[str]: + return [ + "bench_help", + "bench_execute", + "get_logs", + "write_file", + "read_file", + "list_app_files", + "describe_app", + ] + + def validate_environment(self) -> Tuple[bool, Optional[str]]: + """ + Validate that the bench path is resolvable + and the apps/ directory exists. + """ + try: + bench_path = frappe.utils.get_bench_path() + import os + apps_path = os.path.join(bench_path, "apps") + if not os.path.exists(apps_path): + return False, f"apps/ directory not found at {apps_path}" + return True, None + except Exception as e: + return False, f"Cannot resolve bench path: {str(e)}" diff --git a/frappe_assistant_core/plugins/developer_tools/tools/__init__.py b/frappe_assistant_core/plugins/developer_tools/tools/__init__.py new file mode 100644 index 0000000..c5a1a85 --- /dev/null +++ b/frappe_assistant_core/plugins/developer_tools/tools/__init__.py @@ -0,0 +1,83 @@ +# Copyright (C) 2025 Promantia +# Developer Tools Plugin — Shared security helpers + +import os +import frappe + +PROTECTED_APPS = { + "frappe", + "frappe_assistant_core", + "erpnext", + "hrms", + "payments", + "india_compliance", + "lending", + "education", +} + + +def assert_system_manager(): + """ + Raises frappe.PermissionError if current user + does not have System Manager role. + Call this as the first line in every tool's execute(). + """ + user = frappe.session.user + + if user == "Guest": + frappe.throw( + "Guest users are not allowed to use Developer Tools.", + frappe.PermissionError + ) + + if "System Manager" not in frappe.get_roles(user): + frappe.throw( + f"User {user} does not have System Manager role. " + f"System Manager is required for all Developer Tools.", + frappe.PermissionError + ) + + +def resolve_and_validate_path(relative_path): + """ + Resolves a relative path against the bench apps/ directory. + Validates: + - No null bytes + - Max 10 directory levels deep + - No path traversal (../../) + - No symlink escapes outside apps/ + Returns the resolved absolute path if valid. + Raises frappe.ValidationError if invalid. + """ + # Null byte check + if "\x00" in relative_path: + frappe.throw( + "Invalid path: null bytes are not allowed.", + frappe.ValidationError + ) + + # Depth check + parts = [p for p in relative_path.replace("\\", "/").split("/") if p] + if len(parts) > 10: + frappe.throw( + f"Invalid path: too many directory levels ({len(parts)}). Max is 10.", + frappe.ValidationError + ) + + # Build full path + bench_path = frappe.utils.get_bench_path() + apps_path = os.path.join(bench_path, "apps") + full_path = os.path.join(apps_path, relative_path) + + # Resolve symlinks and traversal + real_path = os.path.realpath(full_path) + + # Boundary check + if not real_path.startswith(apps_path + os.sep) and real_path != apps_path: + frappe.throw( + f"Invalid path: resolves outside the apps/ directory. " + f"Path traversal and symlink escapes are not allowed.", + frappe.ValidationError + ) + + return real_path diff --git a/frappe_assistant_core/plugins/developer_tools/tools/bench_execute.py b/frappe_assistant_core/plugins/developer_tools/tools/bench_execute.py new file mode 100644 index 0000000..6a7f0f7 --- /dev/null +++ b/frappe_assistant_core/plugins/developer_tools/tools/bench_execute.py @@ -0,0 +1,485 @@ +# Copyright (C) 2025 Promantia +# Developer Tools Plugin — bench_execute tool + +import ast +import json +import os +import re +import shutil +import sys +from typing import Any, Dict + +import frappe +import frappe.installer +from frappe import _ + +from frappe_assistant_core.core.base_tool import BaseTool +from frappe_assistant_core.plugins.developer_tools.tools import ( + PROTECTED_APPS, + assert_system_manager, +) + +_APP_NAME_RE = re.compile(r"^[a-z][a-z0-9_]*$") + +ALLOWED_FIXTURE_DOCTYPES = { + "Custom Field", + "Client Script", + "Server Script", + "Property Setter", + "Role", + "Workflow", + "Print Format", + "Notification", +} + + +def _convert_filters_to_list(filters: dict) -> list: + result = [] + for field, value in filters.items(): + if isinstance(value, list) and len(value) == 2 and isinstance(value[0], str): + result.append([field, value[0], value[1]]) + else: + result.append([field, "=", value]) + return result + + +class BenchExecute(BaseTool): + """ + Executes bench operations: list apps, create/install/uninstall/remove apps, + and export fixtures. Always call bench_help first to know available actions and parameters. + """ + + def __init__(self): + super().__init__() + self.name = "bench_execute" + self.description = ( + "Executes bench operations. Always call bench_help first to know available actions " + "and parameters. Do NOT ask user for bench paths or site names." + ) + self.category = "Developer Tools" + self.source_app = "frappe_assistant_core" + + self.inputSchema = { + "type": "object", + "properties": { + "action": { + "type": "string", + "description": "Operation to perform.", + "enum": ["list_apps", "list_sites", "create_app", "install_app", "uninstall_app", "remove_app", "export_fixtures"], + }, + "app_name": { + "type": "string", + "description": "Snake-case app name (required for all actions except list_apps).", + }, + "doctype": { + "type": "string", + "description": "DocType to export as a fixture (only for export_fixtures).", + }, + "filters": { + "type": "object", + "description": "Filters to specify which records to export (only for export_fixtures).", + }, + }, + "required": ["action"], + } + + def execute(self, arguments: Dict[str, Any]) -> Dict[str, Any]: + assert_system_manager() + + action = arguments.get("action") + + if action == "list_apps": + bench_path = frappe.utils.get_bench_path() + apps_path = os.path.join(bench_path, "apps") + + all_entries = os.listdir(apps_path) + bench_apps = [ + entry for entry in all_entries + if not entry.startswith(".") + and entry not in PROTECTED_APPS + and os.path.isdir(os.path.join(apps_path, entry)) + ] + + all_installed = frappe.get_installed_apps() + site_apps = [ + app for app in all_installed + if app not in PROTECTED_APPS + ] + + ghosts = [ + app for app in all_installed + if app not in PROTECTED_APPS + and not os.path.isdir(os.path.join(apps_path, app)) + ] + if ghosts: + clean_list = [app for app in all_installed if app not in ghosts] + frappe.db.set_global("installed_apps", frappe.as_json(clean_list)) + frappe.db.commit() + site_apps = [app for app in site_apps if app not in ghosts] + + return { + "success": True, + "bench_apps": bench_apps, + "bench_apps_count": len(bench_apps), + "site_apps": site_apps, + "site_apps_count": len(site_apps), + "message": "bench_apps are all apps on disk. site_apps are apps installed on current site.", + } + + elif action == "list_sites": + bench_path = frappe.utils.get_bench_path() + sites_path = os.path.join(bench_path, "sites") + available_sites = [] + for item in os.listdir(sites_path): + site_config = os.path.join(sites_path, item, "site_config.json") + if os.path.isfile(site_config): + available_sites.append(item) + return { + "success": True, + "sites": available_sites, + "count": len(available_sites), + "message": "These are the available sites on this bench.", + } + + elif action == "create_app": + app_name = arguments.get("app_name") + if not app_name: + frappe.throw("app_name is required for create_app.", frappe.ValidationError) + + if not _APP_NAME_RE.match(app_name): + frappe.throw( + _( + "Invalid app_name '{0}'. Must match ^[a-z][a-z0-9_]*$ " + "(lowercase letters, digits, underscores; must start with a letter)." + ).format(app_name), + frappe.ValidationError, + ) + + if app_name in PROTECTED_APPS: + frappe.throw( + _("Cannot create or overwrite protected app '{0}'.").format(app_name), + frappe.PermissionError, + ) + + bench_path = frappe.utils.get_bench_path() + apps_path = os.path.join(bench_path, "apps") + app_path = os.path.join(apps_path, app_name) + + if os.path.isdir(app_path): + return { + "success": True, + "already_existed": True, + "installed": False, + "apps_txt_updated": False, + "app_name": app_name, + "message": f"App '{app_name}' already exists at {app_path}.", + } + + app_title = arguments.get("app_title") or app_name.replace("_", " ").title() + app_description = arguments.get("app_description") or "" + + hooks = frappe._dict( + app_name=app_name, + app_title=app_title, + app_description=app_description, + app_publisher="Promantia", + app_email="dev@promantia.com", + app_license="mit", + create_github_workflow=False, + ) + + from frappe.utils.boilerplate import _create_app_boilerplate + _create_app_boilerplate(apps_path, hooks, no_git=True) + + outer_init = os.path.join(apps_path, app_name, "__init__.py") + if not os.path.exists(outer_init): + open(outer_init, "w").close() + + pyproject = os.path.join(apps_path, app_name, "pyproject.toml") + if not os.path.exists(pyproject): + with open(pyproject, "w") as f: + f.write(f"""[project] +name = "{app_name}" +version = "0.0.1" + +[build-system] +requires = ["flit_core >=3.2,<4"] +build-backend = "flit_core.buildapi" +""") + + from pip._internal.cli.main import main as pip_main + pip_main(["install", "--quiet", "-e", os.path.join(apps_path, app_name)]) + + if app_path not in sys.path: + sys.path.insert(0, app_path) + + apps_txt_path = os.path.join(bench_path, "sites", "apps.txt") + apps_txt_updated = False + if os.path.exists(apps_txt_path): + with open(apps_txt_path) as f: + existing = [line.strip() for line in f if line.strip()] + else: + existing = [] + + if app_name not in existing: + existing.append(app_name) + with open(apps_txt_path, "w") as f: + f.write("\n".join(existing) + "\n") + apps_txt_updated = True + + return { + "success": True, + "already_existed": False, + "installed": False, + "apps_txt_updated": apps_txt_updated, + "app_name": app_name, + "app_title": app_title, + "message": f"App '{app_name}' created successfully. Use install_app to install it on the site.", + } + + elif action == "install_app": + app_name = arguments.get("app_name") + if not app_name: + frappe.throw(_("app_name is required for install_app."), frappe.ValidationError) + + frappe.installer.install_app(app_name) + + return { + "success": True, + "app_name": app_name, + "message": f"App '{app_name}' installed on current site successfully.", + } + + elif action == "uninstall_app": + app_name = arguments.get("app_name") + if not app_name: + frappe.throw(_("app_name is required for uninstall_app."), frappe.ValidationError) + + installed = frappe.get_installed_apps() + if app_name not in installed: + return { + "success": False, + "app_name": app_name, + "message": f"App '{app_name}' is not installed on the current site.", + } + + frappe.installer.remove_app(app_name, yes=True, no_backup=True) + + installed = frappe.get_installed_apps() + if app_name in installed: + new_list = [app for app in installed if app != app_name] + frappe.db.set_global("installed_apps", frappe.as_json(new_list)) + frappe.db.commit() + + return { + "success": True, + "app_name": app_name, + "message": f"App '{app_name}' uninstalled from current site successfully.", + } + + elif action == "remove_app": + app_name = arguments.get("app_name") + if not app_name: + frappe.throw(_("app_name is required for remove_app."), frappe.ValidationError) + + if app_name in PROTECTED_APPS: + frappe.throw( + _("Cannot remove protected app '{0}'.").format(app_name), + frappe.PermissionError, + ) + + bench_path = frappe.utils.get_bench_path() + app_path = os.path.join(bench_path, "apps", app_name) + + if not os.path.isdir(app_path): + frappe.throw( + f"App '{app_name}' directory does not exist at {app_path}.", + frappe.ValidationError, + ) + + # Uninstall from site first if installed + installed_apps = frappe.get_installed_apps() + if app_name in installed_apps: + frappe.installer.remove_app(app_name, yes=True, no_backup=True) + else: + # Still clean from DB just in case + new_list = [] + for app in installed_apps: + if app != app_name: + new_list.append(app) + frappe.db.set_global("installed_apps", frappe.as_json(new_list)) + frappe.db.commit() + + shutil.rmtree(app_path) + + # Remove from DB installed apps + installed = frappe.get_installed_apps() + if app_name in installed: + new_list = [app for app in installed if app != app_name] + frappe.db.set_global("installed_apps", frappe.as_json(new_list)) + frappe.db.commit() + + # Pip uninstall the package + from pip._internal.cli.main import main as pip_main + try: + pip_main(["uninstall", "-y", app_name]) + except Exception: + pass # ignore if not pip installed + + apps_txt_path = os.path.join(bench_path, "sites", "apps.txt") + apps_txt_updated = False + if os.path.exists(apps_txt_path): + with open(apps_txt_path) as f: + existing = [line.strip() for line in f if line.strip()] + if app_name in existing: + existing.remove(app_name) + with open(apps_txt_path, "w") as f: + f.write("\n".join(existing) + "\n") + apps_txt_updated = True + + return { + "success": True, + "app_name": app_name, + "apps_txt_updated": apps_txt_updated, + "message": f"App '{app_name}' directory removed from bench successfully.", + } + + elif action == "export_fixtures": + if not arguments.get("app_name"): + frappe.throw("app_name is required for export_fixtures.", frappe.ValidationError) + if not arguments.get("doctype"): + frappe.throw("MISSING REQUIRED PARAMETER: doctype. Cannot proceed without doctype. User must specify the DocType to export. Example: 'Custom Field', 'Property Setter', 'Client Script'", frappe.ValidationError) + if not arguments.get("filters"): + frappe.throw("MISSING REQUIRED PARAMETER: filters. Cannot proceed without filters. User must specify exact filters. Example: {\"dt\": \"Sales Invoice\"} or {\"module\": \"HR\"}", frappe.ValidationError) + + app_name = arguments.get("app_name") + doctype = arguments.get("doctype") + filters = arguments.get("filters") or {} + + if app_name in PROTECTED_APPS: + frappe.throw( + _("Cannot export fixtures to protected app '{0}'.").format(app_name), + frappe.PermissionError, + ) + + bench_path = frappe.utils.get_bench_path() + apps_path = os.path.join(bench_path, "apps") + app_path = os.path.join(apps_path, app_name) + + if not os.path.isdir(app_path): + frappe.throw( + f"App '{app_name}' does not exist on this bench. Use create_app to create it first.", + frappe.ValidationError, + ) + + if doctype not in ALLOWED_FIXTURE_DOCTYPES: + frappe.throw( + _("DocType '{0}' is not allowed as a fixture. Allowed: {1}").format( + doctype, ", ".join(sorted(ALLOWED_FIXTURE_DOCTYPES)) + ), + frappe.ValidationError, + ) + + records = frappe.get_all(doctype, filters=filters, fields=["*"]) + + if not records: + return { + "success": False, + "message": f"No {doctype} records found with given filters.", + } + + full_records = [] + for record in records: + doc = frappe.get_doc(doctype, record["name"]) + full_records.append(doc.as_dict()) + + doctype_snake = frappe.scrub(doctype) + fixture_dir = os.path.join(app_path, app_name, "fixtures") + fixture_file = os.path.join(fixture_dir, f"{doctype_snake}.json") + os.makedirs(fixture_dir, exist_ok=True) + + if os.path.exists(fixture_file): + with open(fixture_file, "r") as f: + existing_data = json.load(f) + existing_names = {r["name"] for r in existing_data} + new_records = [r for r in full_records if r["name"] not in existing_names] + existing_data.extend(new_records) + final_records = existing_data + else: + final_records = full_records + + with open(fixture_file, "w") as f: + json.dump(final_records, f, indent=2, default=str) + + hooks_file = os.path.join(app_path, app_name, "hooks.py") + hooks_updated = False + + if os.path.exists(hooks_file): + with open(hooks_file, "r") as f: + content = f.read() + + hooks_filters = _convert_filters_to_list(filters) + new_entry = {"dt": doctype, "filters": hooks_filters} + tree = ast.parse(content) + fixtures_found = False + + for node in ast.walk(tree): + if isinstance(node, ast.Assign): + for target in node.targets: + if isinstance(target, ast.Name) and target.id == "fixtures": + fixtures_found = True + try: + existing_entries = ast.literal_eval(node.value) + existing_entry_index = None + for i, e in enumerate(existing_entries): + if isinstance(e, dict) and (e.get("dt") == doctype or e.get("doctype") == doctype): + existing_entry_index = i + break + + if existing_entry_index is None: + existing_entries.append(new_entry) + hooks_updated = True + elif existing_entries[existing_entry_index] != new_entry: + existing_entries[existing_entry_index] = new_entry + hooks_updated = True + + if hooks_updated: + new_fixtures_str = f"fixtures = {json.dumps(existing_entries, indent=4)}" + lines = content.split("\n") + start_line = node.lineno - 1 + end_line = node.end_lineno + new_lines = ( + lines[:start_line] + + new_fixtures_str.split("\n") + + lines[end_line:] + ) + content = "\n".join(new_lines) + except Exception: + pass + + if not fixtures_found: + new_fixtures_str = f"\nfixtures = {json.dumps([new_entry], indent=4)}\n" + content += new_fixtures_str + hooks_updated = True + + with open(hooks_file, "w") as f: + f.write(content) + + return { + "success": True, + "app_name": app_name, + "doctype": doctype, + "records_exported": len(full_records), + "fixture_file": os.path.join(app_name, app_name, "fixtures", f"{doctype_snake}.json"), + "hooks_updated": hooks_updated, + "message": f"{len(full_records)} {doctype} records exported to {app_name}", + } + + else: + frappe.throw( + f"Unknown action '{action}'. Call bench_help to see available operations.", + frappe.ValidationError, + ) + + +bench_execute = BenchExecute diff --git a/frappe_assistant_core/plugins/developer_tools/tools/bench_help.py b/frappe_assistant_core/plugins/developer_tools/tools/bench_help.py new file mode 100644 index 0000000..8dfcc86 --- /dev/null +++ b/frappe_assistant_core/plugins/developer_tools/tools/bench_help.py @@ -0,0 +1,95 @@ +# Copyright (C) 2025 Promantia +# Developer Tools Plugin — bench_help tool + +from typing import Any, Dict + +from frappe_assistant_core.core.base_tool import BaseTool + + +class BenchHelp(BaseTool): + """ + Returns all available bench operations with their required parameters. + Always call this before bench_execute so you know what actions and parameters are available. + """ + + def __init__(self): + super().__init__() + self.name = "bench_help" + self.description = ( + "Call this FIRST before any bench operation. Returns all available bench operations " + "with their required parameters. Always call bench_help before bench_execute so you " + "know what actions and parameters are available.\n\n" + "INTERACTION RULES:\n" + "1. After showing available operations, always ask user which operation they want as a question with each operation as a separate clickable option.\n" + "2. When user asks what operations are available, call this tool then present each action as a separate option in a question format.\n" + "3. When user wants to remove, uninstall or install an app without specifying which app, first call bench_execute with list_apps, then present each app as a separate clickable option in a question." + ) + self.category = "Developer Tools" + self.source_app = "frappe_assistant_core" + + self.inputSchema = { + "type": "object", + "properties": {}, + } + + def execute(self, arguments: Dict[str, Any]) -> Dict[str, Any]: + return { + "success": True, + "actions": [ + { + "action": "list_apps", + "description": "List all apps on bench and apps installed on site separately", + "params": [], + }, + { + "action": "list_sites", + "description": "List all available sites on this bench", + "params": [], + }, + { + "action": "create_app", + "description": "Create a new custom Frappe app on bench", + "params": ["app_name", "app_title (optional)", "app_description (optional)"], + }, + { + "action": "install_app", + "description": "Install an existing bench app onto the current site", + "params": ["app_name"], + }, + { + "action": "uninstall_app", + "description": "Uninstall app from current site only, keeps files on bench", + "params": ["app_name"], + }, + { + "action": "remove_app", + "description": "Permanently remove app from bench — uninstalls from site, pip uninstalls package, removes directory", + "params": ["app_name"], + }, + { + "action": "export_fixtures", + "description": ( + "Export DocType customizations as fixtures to a custom app. REQUIRED params: app_name, doctype, filters. Tool will FAIL if any are missing.\n\n" + "ALLOWED DOCTYPES for export:\n" + "- Custom Field → filter by: dt (DocType name)\n" + "- Property Setter → filter by: doc_type, field_name\n" + "- Client Script → filter by: dt, module\n" + "- Server Script → filter by: name, module\n" + "- Role → filter by: name\n" + "- Workflow → filter by: document_type\n" + "- Print Format → filter by: doc_type, module\n" + "- Notification → filter by: document_type, module\n\n" + "Always provide very specific filters. Examples:\n" + "- Custom Field: {\"dt\": \"Sales Invoice\"}\n" + "- Property Setter: {\"doc_type\": \"Sales Invoice\", \"field_name\": \"max_discount\"}\n" + "- Server Script: {\"name\": [\"in\", [\"script1\", \"script2\"]]}\n" + "- Role: {\"name\": [\"in\", [\"Accounts Manager\", \"Stock Manager\"]]}\n\n" + "The more specific the filters, the fewer and more precise the records exported." + ), + "params": ["app_name", "doctype", "filters"], + }, + ], + } + + +bench_help = BenchHelp diff --git a/frappe_assistant_core/plugins/developer_tools/tools/describe_app.py b/frappe_assistant_core/plugins/developer_tools/tools/describe_app.py new file mode 100644 index 0000000..8db2c7f --- /dev/null +++ b/frappe_assistant_core/plugins/developer_tools/tools/describe_app.py @@ -0,0 +1 @@ +# describe_app — stub (implementation pending) diff --git a/frappe_assistant_core/plugins/developer_tools/tools/ensure_app.py b/frappe_assistant_core/plugins/developer_tools/tools/ensure_app.py new file mode 100644 index 0000000..3d27e18 --- /dev/null +++ b/frappe_assistant_core/plugins/developer_tools/tools/ensure_app.py @@ -0,0 +1,172 @@ +# Copyright (C) 2025 Promantia +# Developer Tools Plugin — ensure_app tool + +import os +import re +import sys +from typing import Any, Dict + +import frappe +import frappe.installer +from frappe import _ + +from frappe_assistant_core.core.base_tool import BaseTool +from frappe_assistant_core.plugins.developer_tools.tools import PROTECTED_APPS, assert_system_manager + +_APP_NAME_RE = re.compile(r"^[a-z][a-z0-9_]*$") + + +class EnsureApp(BaseTool): + """ + Ensures a custom Frappe app exists in the bench. + + Idempotent: if the app directory already exists, returns immediately with + already_existed=True. Otherwise scaffolds the app, registers it in + sites/apps.txt, and installs it on the current site. + """ + + def __init__(self): + super().__init__() + self.name = "ensure_app" + self.description = ( + "Creates and installs a new Frappe app. Automatically handles bench path detection " + "and site installation — do NOT ask the user for bench paths, site names, or " + "installation preferences. Just call this tool with app_name only. " + "Idempotent — safe to call when the app may already exist." + ) + self.category = "Developer Tools" + self.source_app = "frappe_assistant_core" + + self.inputSchema = { + "type": "object", + "properties": { + "app_name": { + "type": "string", + "description": ( + "Snake-case app name (lowercase letters, digits, underscores; " + "must start with a letter). Defaults to 'fac_custom_code'." + ), + "default": "fac_custom_code", + }, + "app_title": { + "type": "string", + "description": "Human-readable title. Defaults to title-cased app_name.", + }, + "app_description": { + "type": "string", + "description": "Short description of the app.", + }, + }, + "required": [], + } + + def execute(self, arguments: Dict[str, Any]) -> Dict[str, Any]: + assert_system_manager() + + app_name = arguments.get("app_name") or "fac_custom_code" + + if not _APP_NAME_RE.match(app_name): + frappe.throw( + _( + "Invalid app_name '{0}'. Must match ^[a-z][a-z0-9_]*$ " + "(lowercase letters, digits, underscores; must start with a letter)." + ).format(app_name), + frappe.ValidationError, + ) + + if app_name in PROTECTED_APPS: + frappe.throw( + _("Cannot create or overwrite protected app '{0}'.").format(app_name), + frappe.PermissionError, + ) + + bench_path = frappe.utils.get_bench_path() + apps_path = os.path.join(bench_path, "apps") + app_path = os.path.join(apps_path, app_name) + + # Idempotent check + if os.path.isdir(app_path): + return { + "success": True, + "already_existed": True, + "installed": False, + "apps_txt_updated": False, + "app_name": app_name, + "app_title": arguments.get("app_title") or app_name.replace("_", " ").title(), + "message": f"App '{app_name}' already exists at {app_path}.", + } + + app_title = arguments.get("app_title") or app_name.replace("_", " ").title() + app_description = arguments.get("app_description") or "" + + hooks = frappe._dict( + app_name=app_name, + app_title=app_title, + app_description=app_description, + app_publisher="Promantia", + app_email="dev@promantia.com", + app_license="mit", + create_github_workflow=False, + ) + + from frappe.utils.boilerplate import _create_app_boilerplate + _create_app_boilerplate(apps_path, hooks, no_git=True) + + outer_init = os.path.join(apps_path, app_name, "__init__.py") + if not os.path.exists(outer_init): + open(outer_init, "w").close() + + # Create pyproject.toml so the package is pip-installable + pyproject = os.path.join(apps_path, app_name, "pyproject.toml") + if not os.path.exists(pyproject): + with open(pyproject, "w") as f: + f.write(f"""[project] +name = "{app_name}" +version = "0.0.1" + +[build-system] +requires = ["flit_core >=3.2,<4"] +build-backend = "flit_core.buildapi" +""") + + # Register package using pip Python API — no subprocess, no bench CLI + from pip._internal.cli.main import main as pip_main + pip_main(["install", "--quiet", "-e", os.path.join(apps_path, app_name)]) + + # Also add to sys.path for current process + if app_path not in sys.path: + sys.path.insert(0, app_path) + + # Register in sites/apps.txt + apps_txt_path = os.path.join(bench_path, "sites", "apps.txt") + apps_txt_updated = False + if os.path.exists(apps_txt_path): + with open(apps_txt_path) as f: + existing = [line.strip() for line in f if line.strip()] + else: + existing = [] + + if app_name not in existing: + existing.append(app_name) + with open(apps_txt_path, "w") as f: + f.write("\n".join(existing) + "\n") + apps_txt_updated = True + + frappe.db.delete("Module Def", {"app_name": app_name}) + frappe.db.commit() + + # Install using Frappe API only — no bench CLI or subprocess + frappe.installer.install_app(app_name, set_as_patched=True) + + return { + "success": True, + "already_existed": False, + "installed": True, + "apps_txt_updated": apps_txt_updated, + "app_name": app_name, + "app_title": app_title, + "message": f"App '{app_name}' created and installed successfully.", + } + + +ensure_app = EnsureApp diff --git a/frappe_assistant_core/plugins/developer_tools/tools/list_app_files.py b/frappe_assistant_core/plugins/developer_tools/tools/list_app_files.py new file mode 100644 index 0000000..2149e0e --- /dev/null +++ b/frappe_assistant_core/plugins/developer_tools/tools/list_app_files.py @@ -0,0 +1 @@ +# list_app_files — stub (implementation pending) diff --git a/frappe_assistant_core/plugins/developer_tools/tools/read_file.py b/frappe_assistant_core/plugins/developer_tools/tools/read_file.py new file mode 100644 index 0000000..a68cd79 --- /dev/null +++ b/frappe_assistant_core/plugins/developer_tools/tools/read_file.py @@ -0,0 +1 @@ +# read_file — stub (implementation pending) diff --git a/frappe_assistant_core/plugins/developer_tools/tools/write_file.py b/frappe_assistant_core/plugins/developer_tools/tools/write_file.py new file mode 100644 index 0000000..e4cfe23 --- /dev/null +++ b/frappe_assistant_core/plugins/developer_tools/tools/write_file.py @@ -0,0 +1 @@ +# write_file — stub (implementation pending)