Summary
After a request was sent to the local AI the absolute amount of tokens it took should be showed to the user so it's more easy to estimate how big a context window has to be.
Problem / Motivation
It's important oft users to know how hard the context window is used.
Proposed Solution
Output of context size in tokens after the message is received. The whole size not only the text. Including tools and old chats all that got send to the model.
Summary
After a request was sent to the local AI the absolute amount of tokens it took should be showed to the user so it's more easy to estimate how big a context window has to be.
Problem / Motivation
It's important oft users to know how hard the context window is used.
Proposed Solution
Output of context size in tokens after the message is received. The whole size not only the text. Including tools and old chats all that got send to the model.