Skip to content

Commit 4f31654

Browse files
bigcat88bigcat88
authored
persistent storage + Transformers AI (#123)
An example of a Nextcloud Talk bot that uses a very small language model, and the cache with this model is in the application's persistent storage and persists between application updates. --------- Signed-off-by: Alexander Piskun <[email protected]>
1 parent 85f2ab5 commit 4f31654

File tree

13 files changed

+279
-0
lines changed

13 files changed

+279
-0
lines changed

‎.run/TalkBotAI.run.xml

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
<component name="ProjectRunConfigurationManager">
2+
<configuration default="false" name="TalkBotAI" type="PythonConfigurationType" factoryName="Python">
3+
<module name="nc_py_api" />
4+
<option name="INTERPRETER_OPTIONS" value="" />
5+
<option name="PARENT_ENVS" value="true" />
6+
<envs>
7+
<env name="APP_ID" value="talk_bot_ai" />
8+
<env name="APP_PORT" value="9034" />
9+
<env name="APP_SECRET" value="12345" />
10+
<env name="APP_VERSION" value="1.0.0" />
11+
<env name="NEXTCLOUD_URL" value="http://nextcloud.local" />
12+
<env name="PYTHONUNBUFFERED" value="1" />
13+
</envs>
14+
<option name="SDK_HOME" value="" />
15+
<option name="WORKING_DIRECTORY" value="$PROJECT_DIR$" />
16+
<option name="IS_MODULE_SDK" value="true" />
17+
<option name="ADD_CONTENT_ROOTS" value="true" />
18+
<option name="ADD_SOURCE_ROOTS" value="true" />
19+
<EXTENSION ID="PythonCoverageRunConfigurationExtension" runner="coverage.py" />
20+
<option name="SCRIPT_NAME" value="$PROJECT_DIR$/examples/as_app/talk_bot_AI/src/main.py" />
21+
<option name="PARAMETERS" value="" />
22+
<option name="SHOW_COMMAND_LINE" value="false" />
23+
<option name="EMULATE_TERMINAL" value="false" />
24+
<option name="MODULE_MODE" value="false" />
25+
<option name="REDIRECT_INPUT" value="false" />
26+
<option name="INPUT_FILE" value="" />
27+
<method v="2" />
28+
</configuration>
29+
</component>

‎CHANGELOG.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,13 @@
22

33
All notable changes to this project will be documented in this file.
44

5+
## [0.2.1 - 2023-09-14]
6+
7+
### Added
8+
9+
- NextcloudApp: `ex_app.persistent_storage` function that returns path for the Application persistent storage.
10+
- NextcloudApp: `from nc_py_api.ex_app import persist_transformers_cache` - automatic use of persistent app directory for the AI models caching.
11+
512
## [0.2.0 - 2023-09-13]
613

714
### Added

‎examples/as_app/talk_bot_AI/Dockerfile

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
FROM python:3.11-bookworm
2+
3+
COPY requirements.txt /
4+
ADD /src/ /app/
5+
6+
RUN \
7+
python3 -m pip install -r requirements.txt && rm -rf ~/.cache && rm requirements.txt
8+
9+
WORKDIR /app
10+
ENTRYPOINT ["python3", "main.py"]

‎examples/as_app/talk_bot_AI/HOW_TO_INSTALL.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
How To Install
2+
==============
3+
4+
Currently, while AppAPI hasn't been published on the App Store, and App Store support hasn't been added yet,
5+
installation is a little bit tricky.
6+
7+
Steps to Install:
8+
9+
1. [Install AppAPI](https://cloud-py-api.github.io/app_api/Installation.html)
10+
2. Create a deployment daemon according to the [instructions](https://cloud-py-api.github.io/app_api/CreationOfDeployDaemon.html#create-deploy-daemon) of the AppPI
11+
3. php occ app_api:app:deploy talk_bot_ai "daemon_deploy_name" \
12+
--info-xml https://raw.githubusercontent.com/cloud-py-api/nc_py_api/main/examples/as_app/talk_bot_ai/appinfo/info.xml
13+
14+
to deploy a docker image with Bot to docker.
15+
16+
4. php occ app_api:app:register talk_bot_ai "daemon_deploy_name" -e --force-scopes \
17+
--info-xml https://raw.githubusercontent.com/cloud-py-api/nc_py_api/main/examples/as_app/talk_bot_ai/appinfo/info.xml
18+
19+
to call its **enable** handler and accept all required API scopes by default.
20+
21+
22+
In a few months
23+
===============
24+
25+
1. Install AppAPI from App Store
26+
2. Configure Deploy Daemon with GUI provided by AppAPI
27+
3. Go to External Applications page in Nextcloud UI
28+
4. Find this bot in a list and press "Install" and "Enable" buttons, like with usual Applications.

‎examples/as_app/talk_bot_AI/Makefile

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
.DEFAULT_GOAL := help
2+
3+
.PHONY: help
4+
help:
5+
@echo "Welcome to TalkBotAI example. Please use \`make <target>\` where <target> is one of"
6+
@echo " "
7+
@echo " Next commands are only for dev environment with nextcloud-docker-dev!"
8+
@echo " They should run from the host you are developing on(with activated venv) and not in the container with Nextcloud!"
9+
@echo " "
10+
@echo " build-push build image and upload to ghcr.io"
11+
@echo " "
12+
@echo " deploy deploy example to registered 'docker_dev'"
13+
@echo " "
14+
@echo " run28 install TalkBotAI for Nextcloud 28"
15+
@echo " run27 install TalkBotAI for Nextcloud 27"
16+
@echo " "
17+
@echo " For development of this example use PyCharm run configurations. Development is always set for last Nextcloud."
18+
@echo " First run 'TalkBotAI' and then 'make manual_register', after that you can use/debug/develop it and easy test."
19+
@echo " "
20+
@echo " manual_register perform registration of running 'TalkBotAI' into the 'manual_install' deploy daemon."
21+
22+
.PHONY: build-push
23+
build-push:
24+
docker login ghcr.io
25+
docker buildx build --push --platform linux/arm64/v8,linux/amd64 --tag ghcr.io/cloud-py-api/talk_bot_ai:latest .
26+
27+
.PHONY: deploy
28+
deploy:
29+
docker exec master-nextcloud-1 sudo -u www-data php occ app_api:app:deploy talk_bot_ai docker_dev \
30+
--info-xml https://raw.githubusercontent.com/cloud-py-api/nc_py_api/main/examples/as_app/talk_bot_ai/appinfo/info.xml
31+
32+
.PHONY: run28
33+
run28:
34+
docker exec master-nextcloud-1 sudo -u www-data php occ app_api:app:unregister talk_bot_ai --silent || true
35+
docker exec master-nextcloud-1 sudo -u www-data php occ app_api:app:register talk_bot_ai docker_dev -e --force-scopes \
36+
--info-xml https://raw.githubusercontent.com/cloud-py-api/nc_py_api/main/examples/as_app/talk_bot_ai/appinfo/info.xml
37+
38+
.PHONY: run27
39+
run27:
40+
docker exec master-stable27-1 sudo -u www-data php occ app_api:app:unregister talk_bot_ai --silent || true
41+
docker exec master-stable27-1 sudo -u www-data php occ app_api:app:register talk_bot_ai docker_dev -e --force-scopes \
42+
--info-xml https://raw.githubusercontent.com/cloud-py-api/nc_py_api/main/examples/as_app/talk_bot_ai/appinfo/info.xml
43+
44+
.PHONY: manual_register
45+
manual_register:
46+
docker exec master-nextcloud-1 sudo -u www-data php occ app_api:app:unregister talk_bot_ai --silent || true
47+
docker exec master-nextcloud-1 sudo -u www-data php occ app_api:app:register talk_bot_ai manual_install --json-info \
48+
"{\"appid\":\"talk_bot_ai\",\"name\":\"TalkBotAI\",\"daemon_config_name\":\"manual_install\",\"version\":\"1.0.0\",\"secret\":\"12345\",\"host\":\"host.docker.internal\",\"port\":9034,\"scopes\":{\"required\":[\"TALK\", \"TALK_BOT\"],\"optional\":[]},\"protocol\":\"http\",\"system_app\":0}" \
49+
-e --force-scopes

‎examples/as_app/talk_bot_AI/appinfo/info.xml

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
<?xml version="1.0"?>
2+
<info>
3+
<id>talk_bot_ai</id>
4+
<name>TalkBotAI</name>
5+
<summary>Nextcloud TalkBotAI Example</summary>
6+
<description>
7+
<![CDATA[Example of the Nextcloud Talk Bot + LLM written in python]]>
8+
</description>
9+
<version>1.0.0</version>
10+
<licence>MIT</licence>
11+
<author mail="[email protected]" homepage="https://github.com/andrey18106">Andrey Borysenko</author>
12+
<author mail="[email protected]" homepage="https://github.com/bigcat88">Alexander Piskun</author>
13+
<namespace>TalkBotAIExample</namespace>
14+
<category>tools</category>
15+
<website>https://github.com/cloud-py-api/nc_py_api</website>
16+
<bugs>https://github.com/cloud-py-api/nc_py_api/issues</bugs>
17+
<repository type="git">https://github.com/cloud-py-api/nc_py_api</repository>
18+
<dependencies>
19+
<nextcloud min-version="27" max-version="28"/>
20+
</dependencies>
21+
<ex-app>
22+
<docker-install>
23+
<registry>ghcr.io</registry>
24+
<image>cloud-py-api/talk_bot_ai</image>
25+
<image-tag>latest</image-tag>
26+
</docker-install>
27+
<scopes>
28+
<required>
29+
<value>TALK</value>
30+
<value>TALK_BOT</value>
31+
</required>
32+
<optional>
33+
</optional>
34+
</scopes>
35+
<protocol>http</protocol>
36+
<system>0</system>
37+
</ex-app>
38+
</info>

‎examples/as_app/talk_bot_AI/requirements.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
nc_py_api[app]>=0.2.1
2+
transformers>=4.33
3+
torch
4+
torchvision
5+
torchaudio

‎examples/as_app/talk_bot_AI/screenshots/talk_bot_ai.png

86.6 KB
Loading

‎examples/as_app/talk_bot_AI/src/main.py

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
"""Example of an application that uses Python Transformers library with Talk Bot APIs."""
2+
3+
# This line should be on top before any import of the "Transformers" library.
4+
from nc_py_api.ex_app import persist_transformers_cache # noqa # isort:skip
5+
import re
6+
from threading import Thread
7+
from typing import Annotated
8+
9+
import requests
10+
from fastapi import BackgroundTasks, Depends, FastAPI
11+
from transformers import pipeline
12+
13+
from nc_py_api import NextcloudApp, talk_bot
14+
from nc_py_api.ex_app import run_app, set_handlers, talk_bot_app
15+
16+
APP = FastAPI()
17+
AI_BOT = talk_bot.TalkBot("/ai_talk_bot", "AI talk bot", "Usage: `@ai What sounds do cats make?`")
18+
MODEL_NAME = "MBZUAI/LaMini-Flan-T5-77M"
19+
MODEL_INIT_THREAD = None
20+
21+
22+
def ai_talk_bot_process_request(message: talk_bot.TalkBotMessage):
23+
r = re.search(r"@ai\s(.*)", message.object_content["message"], re.IGNORECASE)
24+
if r is None:
25+
return
26+
model = pipeline("text2text-generation", model="MBZUAI/LaMini-Flan-T5-77M")
27+
response_text = model(r.group(1), max_length=64, do_sample=True)[0]["generated_text"]
28+
AI_BOT.send_message(response_text, message)
29+
30+
31+
@APP.post("/ai_talk_bot")
32+
async def ai_talk_bot(
33+
message: Annotated[talk_bot.TalkBotMessage, Depends(talk_bot_app)],
34+
background_tasks: BackgroundTasks,
35+
):
36+
if message.object_name == "message":
37+
background_tasks.add_task(ai_talk_bot_process_request, message)
38+
return requests.Response()
39+
40+
41+
def enabled_handler(enabled: bool, nc: NextcloudApp) -> str:
42+
print(f"enabled={enabled}")
43+
try:
44+
AI_BOT.enabled_handler(enabled, nc)
45+
except Exception as e:
46+
return str(e)
47+
return ""
48+
49+
50+
def download_models():
51+
pipeline("text2text-generation", model=MODEL_NAME)
52+
53+
54+
def heartbeat_handler() -> str:
55+
global MODEL_INIT_THREAD
56+
print("heartbeat_handler: called")
57+
if MODEL_INIT_THREAD is None:
58+
MODEL_INIT_THREAD = Thread(target=download_models)
59+
MODEL_INIT_THREAD.start()
60+
print("heartbeat_handler: started initialization thread")
61+
r = "init" if MODEL_INIT_THREAD.is_alive() else "ok"
62+
print(f"heartbeat_handler: result={r}")
63+
return r
64+
65+
66+
@APP.on_event("startup")
67+
def initialization():
68+
set_handlers(APP, enabled_handler, heartbeat_handler)
69+
70+
71+
if __name__ == "__main__":
72+
run_app("main:APP", log_level="trace")

‎nc_py_api/ex_app/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
"""All possible ExApp stuff for NextcloudApp that can be used."""
22
from .defs import ApiScope, LogLvl
33
from .integration_fastapi import nc_app, set_handlers, talk_bot_app
4+
from .misc import persistent_storage
45
from .ui.files import UiActionFileInfo, UiFileActionHandlerInfo
56
from .uvicorn_fastapi import run_app

0 commit comments

Comments
 (0)