Skip to content

docs: add macOS/Apple Silicon setup guide#319

Open
karishmahegde wants to merge 4 commits intojenkinsci:mainfrom
karishmahegde:main
Open

docs: add macOS/Apple Silicon setup guide#319
karishmahegde wants to merge 4 commits intojenkinsci:mainfrom
karishmahegde:main

Conversation

@karishmahegde
Copy link
Copy Markdown

Summary

Adds a dedicated setup guide for macOS/Apple Silicon (M1/M2/M3) contributors
at docs/macos-setup.md.

The existing documentation covers Linux and Windows but has no macOS-specific
guide. During local setup on Apple M2, I identified several issues that are not
documented anywhere:

  • llama-cpp-python and python-multipart must be installed manually — they
    are not listed in requirements-cpu.txt
  • The default gpu_layers: 0 in config.yml causes CPU-only inference,
    resulting in response times of 5+ minutes. Setting it to 32 enables Apple
    Metal GPU acceleration, reducing response time to under 30 seconds
  • IS_CPU_REQ=1 must be used with make api to avoid numpy version conflicts
    in requirements.txt

Testing Done

Tested end-to-end on macOS Apple Silicon (M2, Metal GPU) with Python 3.12:

  • Completed full setup following the guide from scratch
  • Ran the data pipeline successfully — embedded 16,135 documents into FAISS index
  • Started backend with GPU acceleration confirmed via Metal logs
  • Sent test messages and received correct responses via the frontend

Changes

  • Added docs/macos-setup.md with step-by-step Apple Silicon setup instructions
  • Includes common troubleshooting section based on issues encountered during setup

@karishmahegde karishmahegde requested a review from a team as a code owner March 28, 2026 23:25
@berviantoleo
Copy link
Copy Markdown
Contributor

Just for your information, we're planning to use Docker #75 for setting up the chatbot. So, "hopefully" we don't need to maintain much documentation based on each operating system and/or specific system. But we may need to consider some part that might be a missed feature while hosting it in Docker.

Don't need to worry, I'll keep this for our consideration.

@berviantoleo berviantoleo added the documentation Improvements or additions to documentation label Mar 29, 2026
@karishmahegde
Copy link
Copy Markdown
Author

Hi @berviantoleo
Thank you for the context! That makes sense, a Docker-based setup would
significantly simplify the contributor experience across all platforms.

I've updated my proposal to reflect this, pivoting the setup documentation
point toward contributing to the Docker configuration with attention to
GPU acceleration across different hardware setups.

I'd be happy to help with the Docker setup effort and identify any
Apple Silicon specific considerations that might need to be addressed!

Copy link
Copy Markdown
Contributor

@berviantoleo berviantoleo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add "link" in the main setup docs.

@karishmahegde
Copy link
Copy Markdown
Author

Hi @berviantoleo, I've added the link to the macOS/Apple Silicon setup guide in the main README, docs/setup.md, and docs/README.md.

Please let me know if any further changes are needed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants