docs: add macOS/Apple Silicon setup guide#319
docs: add macOS/Apple Silicon setup guide#319karishmahegde wants to merge 4 commits intojenkinsci:mainfrom
Conversation
|
Just for your information, we're planning to use Docker #75 for setting up the chatbot. So, "hopefully" we don't need to maintain much documentation based on each operating system and/or specific system. But we may need to consider some part that might be a missed feature while hosting it in Docker. Don't need to worry, I'll keep this for our consideration. |
|
Hi @berviantoleo I've updated my proposal to reflect this, pivoting the setup documentation I'd be happy to help with the Docker setup effort and identify any |
berviantoleo
left a comment
There was a problem hiding this comment.
Please add "link" in the main setup docs.
|
Hi @berviantoleo, I've added the link to the macOS/Apple Silicon setup guide in the main README, docs/setup.md, and docs/README.md. Please let me know if any further changes are needed! |
Summary
Adds a dedicated setup guide for macOS/Apple Silicon (M1/M2/M3) contributors
at
docs/macos-setup.md.The existing documentation covers Linux and Windows but has no macOS-specific
guide. During local setup on Apple M2, I identified several issues that are not
documented anywhere:
llama-cpp-pythonandpython-multipartmust be installed manually — theyare not listed in
requirements-cpu.txtgpu_layers: 0inconfig.ymlcauses CPU-only inference,resulting in response times of 5+ minutes. Setting it to
32enables AppleMetal GPU acceleration, reducing response time to under 30 seconds
IS_CPU_REQ=1must be used withmake apito avoid numpy version conflictsin
requirements.txtTesting Done
Tested end-to-end on macOS Apple Silicon (M2, Metal GPU) with Python 3.12:
Changes
docs/macos-setup.mdwith step-by-step Apple Silicon setup instructions