New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

6B-chat在A100上推理 #33

Open

sunyclj opened this issue Jul 2, 2024 · 1 comment

sunyclj commented Jul 2, 2024

1、输入汉字长度843字，显存占用25.4G，输出平均1.35字/s；
2、输入汉字长度1770字，显存不足；
请问以上情况是正常的嘛？

otaku-47 commented Jul 2, 2024

+1，遇到了同样的问题

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment