Artificial Intelligence in Elexis

[rgwch]

Here's a proof of concept for the use of locally running LLM's to save some time when reading incoming documents.
It's just an extension of the global-Inbox feature:

https://elexis.ch/ungrad/features/inbox_de/#ki-zusammenfassung

It uses a proxy which should run on the same machine at the moment and hands the document over to a LLM run by node-llama-cpp.

Then, it creates a new Konsultation/Encounter and writes the summary there.

There are several enhancements possible: We can ask the AI to retrieve the medication from hospital letters and insert them in the "Fixmedikation" after having it checked by the user. We can bill automatically an "Aktenstudium" when inserting the document. And so on.

Some thoughts on this topic are also here (I was using Ollama there instead of node-llama-cpp, but it's about the same principle)

Of course we need a somewhat better computer to use that. I'm using a Ryzen 7, 48GB RAM, and an NVidia 5060 Ti with 16GB. It is necessary to install Nvidia's CUDA toolkit to allow use of the graphic card for other jobs than just display. Driver and Cuda install was a bit tricky at this time on Ubuntu Linux, but finally it worked.

[col-panic]

@rgwch thank you for your input on this topic. We thought about something like this for quite a while, yet there still are, and maybe for some time being, always will be obstacles. I'm not primarily talking about the hardware, we could host an instance that satisfy privacy requirements - but about the content the AI generates.

I would be really interested in an answer to the folllwing questions:

• You, as a physician - would you really rely on the summary generated by the AI?
• What if the summary is wrong?
• You would bill an "Aktenstudium" while you did not really do it - don't you think this could lead to legal implications?
• Could such wrong data lead to wrong decisions done by you potentially endangering the patient?
• Do you see areas in the software which are an absolute no-go for AI?

🤔

[rgwch]

Thanks for your thoughs, @col-panic Answers below.

• You, as a physician - would you really rely on the summary generated by the AI? What if the summary is wrong?

Well, I always have to verify and assess incoming documents (does it match what I know of the patient? does it match what I expected). Letters from Hospitals are not always correct, since today they are written with the help of computer systems and in parts automatically compiled from earlier informations that might be outdated and so on. So, the AI does not "think" or "decide" for me, it just writes the summary for me. Usually, I'll always have a quick look at the letter anyway. The bigger time saving comes later: When I want to know what was two years ago, it's much faster to scan the Summaries in the Encounter than to read the letters again.

• You would bill an "Aktenstudium" while you did not really do it - don't you think this could lead to legal implications?

It is an "Aktenstudium", since I read and assess the summary (and probably/typically have also a look to the "procedere" Part of the document) (Note: "Aktenstudium" is worth CHF 3.20 currently, and I usually forget to bill it, when I study some documents).

• Could such wrong data lead to wrong decisions done by you potentially endangering the patient?

I don't think so. It is the same class as wrong lab values or wrong parts in hospital reports: It is always matched against the complete picture. And if action is needed, we normally do further checks.

• Do you see areas in the software which are an absolute no-go for AI?

Personally I think, that in a few decades, family doctors will be replaced by AIs.
Of course, that is not what we are talking here about. For now the no-go is everything with immediate consequences. So it may not write prescriptions, but it may warn if there ist a reason, not to make a prescription. It may not answer a request, but it may summarize it.

Short: AI ist an aid like lab values, specialist checks or ECG are: Never the only reason to make a decision, but a help to come faster to better decisions.

[huthomas]

Looking at the code I see the context size is set to 16384. I am not sure that the resulting summary will be complete, if the text extracted from the pdf is longer than 16384 tokens.

My comment is based on https://medium.com/google-cloud/langchain-chain-types-large-document-summarization-using-langchain-and-google-cloud-vertex-ai-1650801899f6 where different approaches to summarize texts larger than the context size are discussed.

[rgwch]

Valid point, thanks. My incoming documents are usually from 2 to 5 pages. And I'm in a very early stage of test, currently it's just a proof of concept, until now it worked sufficiently for most docs (and failed miserably for some, but rather depending on the structure than on the length). Actually my choice of parameters was not a thing of deep consideration...
I think I'll switch to Ollama anyway, which is also a wrapper around Llama-CPP, but is somewhat easier to handle than node-llama-cpp (installs as a service, opens a REST-Server automatically, frees resources automatically, no need to install NodeJS). But the concept remains the same.