Skip to content

Conversation

@jhaaaa
Copy link
Collaborator

@jhaaaa jhaaaa commented Oct 28, 2025

Split LLM ingestion bundles into llms-chat-apps.txt and llms-agents.txt and update generator in generate_llms_full.py to build three targeted files from docs/pages

Introduce a configurable generate_llms_txt in generate_llms_full.py that assembles content from selected docs/pages subdirectories and generates llms-chat-apps.txt, llms-agents.txt, and llms-full.txt; update the workflow to ignore llms-*.txt and add an npm script to run the generator; revise chat-apps intro docs to reference the new files.

📍Where to Start

Start with the __main__ script block and the generate_llms_txt function in generate_llms_full.py.


📊 Macroscope summarized 2abc30b. 1 file reviewed, 3 issues evaluated, 3 issues filtered, 0 comments posted

🗂️ Filtered Issues

llms/generate_llms_full.py — 0 comments posted, 3 evaluated, 3 filtered
  • line 24: os.makedirs(output_dir) is called outside the surrounding try block and without exist_ok=True. This can raise a FileExistsError or PermissionError due to a race between the existence check and directory creation, or if a file already exists at output_dir. Because it is outside the try, the exception will terminate the script without the intended error handling/logging. Move the creation inside the try or use os.makedirs(output_dir, exist_ok=True) and catch exceptions. [ Already posted ]
  • line 78: Error detection for content uses isinstance(content, str) and content.startswith("Error:"). This will falsely classify a legitimate markdown file whose content begins with "Error:" as unreadable and skip it. Use a structured return (e.g., (ok, content)), exceptions, or a sentinel type/value to distinguish read errors from valid file content. [ Already posted ]
  • line 91: The function logs print(f"Successfully processed {len(processed_files)} Markdown files into {output_file}") even when processed_files is empty. The calling script treats an empty result as a failure and exits with error. This creates contradictory success/failure messaging for the same operation, confusing operators and automated log parsers. Adjust logging to report zero-file cases as a warning or failure consistently with the caller. [ Low confidence ]

@jhaaaa jhaaaa requested a review from a team as a code owner October 28, 2025 22:33
@vercel
Copy link

vercel bot commented Oct 28, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
docs-xmtp-org Ready Ready Preview Comment Nov 13, 2025 1:47am

@jhaaaa jhaaaa changed the title Custom 404 page Split llms-full.txt into llms-chat-apps.txt and llms-agents.txt Nov 13, 2025
@jhaaaa jhaaaa merged commit 6370198 into main Nov 13, 2025
4 checks passed
@jhaaaa jhaaaa deleted the add-custom-404 branch November 13, 2025 01:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants