chore(model): Update the default model to glm-4-9b-chat (eosphoros-ai…

…#1629)

.env.template

@@ -17,15 +17,15 @@
 #**                         LLM MODELS                            **#
 #*******************************************************************#
 # LLM_MODEL, see dbgpt/configs/model_config.LLM_MODEL_CONFIG
-LLM_MODEL=vicuna-13b-v1.5
+LLM_MODEL=glm-4-9b-chat
 ## LLM model path, by default, DB-GPT will read the model path from LLM_MODEL_CONFIG based on the LLM_MODEL.
 ## Of course you can specify your model path according to LLM_MODEL_PATH
 ## In DB-GPT, the priority from high to low to read model path:
 ##    1. environment variable with key: {LLM_MODEL}_MODEL_PATH (Avoid multi-model conflicts)
 ##    2. environment variable with key: MODEL_PATH
 ##    3. environment variable with key: LLM_MODEL_PATH
 ##    4. the config in dbgpt/configs/model_config.LLM_MODEL_CONFIG
-# LLM_MODEL_PATH=/app/models/vicuna-13b-v1.5
+# LLM_MODEL_PATH=/app/models/glm-4-9b-chat
 # LLM_PROMPT_TEMPLATE=vicuna_v1.1
 MODEL_SERVER=http://127.0.0.1:8000
 LIMIT_MODEL_CONCURRENCY=5

dbgpt/_private/config.py

@@ -194,7 +194,7 @@ def __init__(self) -> None:
         self.CHAT_HISTORY_STORE_TYPE = os.getenv("CHAT_HISTORY_STORE_TYPE", "db")
 
         ### LLM Model Service Configuration
-        self.LLM_MODEL = os.getenv("LLM_MODEL", "vicuna-13b-v1.5")
+        self.LLM_MODEL = os.getenv("LLM_MODEL", "glm-4-9b-chat")
         self.LLM_MODEL_PATH = os.getenv("LLM_MODEL_PATH")
 
         ### Proxy llm backend, this configuration is only valid when "LLM_MODEL=proxyllm"

dbgpt/agent/expand/resources/search_tool.py

@@ -26,6 +26,8 @@ def baidu_search(
         "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:112.0) "
         "Gecko/20100101 Firefox/112.0"
     }
+    if num_results < 8:
+        num_results = 8
     url = f"https://www.baidu.com/s?wd={query}&rn={num_results}"
     response = requests.get(url, headers=headers)
     response.encoding = "utf-8"

docker-compose.yml

@@ -24,7 +24,7 @@ services:
       - LOCAL_DB_HOST=db
       - LOCAL_DB_PASSWORD=aa123456
       - ALLOWLISTED_PLUGINS=db_dashboard
-      - LLM_MODEL=vicuna-13b-v1.5
+      - LLM_MODEL=glm-4-9b-chat
     depends_on:
       - db
     volumes:

docker/base/run_sqlite.sh

@@ -4,7 +4,7 @@ docker run --ipc host --gpus all -d \
     -p 5000:5000 \
     -e LOCAL_DB_TYPE=sqlite \
     -e LOCAL_DB_PATH=data/default_sqlite.db \
-    -e LLM_MODEL=vicuna-13b-v1.5 \
+    -e LLM_MODEL=glm-4-9b-chat \
     -e LANGUAGE=zh \
     -v /data:/data \
     -v /data/models:/app/models \

docker/compose_examples/cluster-docker-compose.yml

@@ -19,7 +19,7 @@ services:
       - 8100:8100/tcp
   llm-worker:
     image: eosphorosai/dbgpt:latest
-    command: dbgpt start worker --model_name vicuna-13b-v1.5 --model_path /app/models/vicuna-13b-v1.5 --port 8001 --controller_addr http://controller:8000
+    command: dbgpt start worker --model_name glm-4-9b-chat --model_path /app/models/glm-4-9b-chat --port 8001 --controller_addr http://controller:8000
     environment:
       - DBGPT_LOG_LEVEL=DEBUG
     depends_on:
@@ -66,7 +66,7 @@ services:
       - LOCAL_DB_PATH=data/default_sqlite.db
       - LOCAL_DB_TYPE=sqlite
       - ALLOWLISTED_PLUGINS=db_dashboard
-      - LLM_MODEL=vicuna-13b-v1.5
+      - LLM_MODEL=glm-4-9b-chat
       - MODEL_SERVER=http://controller:8000
     depends_on:
       - controller

docs/docs/installation/advanced_usage/OpenAI_SDK_call.md

@@ -38,7 +38,7 @@ Chat
 curl http://127.0.0.1:8100/api/v1/chat/completions \
 -H "Authorization: Bearer EMPTY" \
 -H "Content-Type: application/json" \
--d '{"model": "vicuna-13b-v1.5", "messages": [{"role": "user", "content": "hello"}]}'
+-d '{"model": "glm-4-9b-chat", "messages": [{"role": "user", "content": "hello"}]}'
 ```
 
 :::tip
@@ -61,7 +61,7 @@ curl http://127.0.0.1:8100/api/v1/embeddings \
 import openai
 openai.api_key = "EMPTY"
 openai.api_base = "http://127.0.0.1:8100/api/v1"
-model = "vicuna-13b-v1.5"
+model = "glm-4-9b-chat"
 
 completion = openai.ChatCompletion.create(
   model=model,

docs/docs/installation/advanced_usage/vLLM_inference.md

@@ -4,15 +4,17 @@ DB-GPT supports [vLLM](https://github.com/vllm-project/vllm) inference, a fast a
 ## Install dependencies
 `vLLM` is an optional dependency in DB-GPT. You can install it manually through the following command.
 
-```python
-$ pip install -e ".[vllm]"
+```bash
+pip install -e ".[vllm]"
 ```
 
 ## Modify configuration file
 In the `.env` configuration file, modify the inference type of the model to start `vllm` inference.
-```python
-LLM_MODEL=vicuna-13b-v1.5
+```bash
+LLM_MODEL=glm-4-9b-chat
 MODEL_TYPE=vllm
+# modify the following configuration if you possess GPU resources
+# gpu_memory_utilization=0.8
 ```
 
 For more information about the list of models supported by `vLLM`, please refer to the [vLLM supported model document](https://docs.vllm.ai/en/latest/models/supported_models.html#supported-models).

docs/docs/installation/docker.md

@@ -5,15 +5,15 @@
 There are two ways to prepare a Docker image. 1. Pull from the official image 2. Build locally. You can **choose any one** during actual use.
 
 1.Pulled from the official image repository, [Eosphoros AI Docker Hub](https://hub.docker.com/u/eosphorosai)
-```python
+```bash
 docker pull eosphorosai/dbgpt:latest
 ```
 2.local build(optional)
-```python
+```bash
 bash docker/build_all_images.sh
 ```
 Check the Docker image
-```python
+```bash
 # command
 docker images | grep "eosphorosai/dbgpt"
 
@@ -24,12 +24,12 @@ eosphorosai/dbgpt                latest    eb3cdc5b4ead   About a minute ago   1
 ```
 `eosphorosai/dbgpt` is the base image, which contains project dependencies and the sqlite database. The `eosphorosai/dbgpt-allinone` image is built from `eosphorosai/dbgpt`, which contains a MySQL database. Of course, in addition to pulling the Docker image, the project also provides Dockerfile files, which can be built directly through scripts in DB-GPT. Here are the build commands:
 
-```python
+```bash
 bash docker/build_all_images.sh
 ```
 When using it, you need to specify specific parameters. The following is an example of specifying parameter construction:
 
-```python
+```bash
 bash docker/build_all_images.sh \
 --base-image nvidia/cuda:11.8.0-runtime-ubuntu22.04 \
 --pip-index-url https://pypi.tuna.tsinghua.edu.cn/simple \
@@ -42,53 +42,53 @@ You can view the specific usage through the command `bash docker/build_all_image
 ### Run through Sqlite database
 
 
-```python
+```bash
 docker run --ipc host --gpus all -d \
 -p 5670:5670 \
 -e LOCAL_DB_TYPE=sqlite \
 -e LOCAL_DB_PATH=data/default_sqlite.db \
--e LLM_MODEL=vicuna-13b-v1.5 \
+-e LLM_MODEL=glm-4-9b-chat \
 -e LANGUAGE=zh \
 -v /data/models:/app/models \
 --name dbgpt \
 eosphorosai/dbgpt
 ```
 Open the browser and visit [http://localhost:5670](http://localhost:5670)
 
-- `-e LLM_MODEL=vicuna-13b-v1.5`, which means the base model uses `vicuna-13b-v1.5`. For more model usage, you can view the configuration in `/pilot/configs/model_config.LLM_MODEL_CONFIG`.
+- `-e LLM_MODEL=glm-4-9b-chat`, which means the base model uses `glm-4-9b-chat`. For more model usage, you can view the configuration in `/pilot/configs/model_config.LLM_MODEL_CONFIG`.
 - `-v /data/models:/app/models`, specifies the model file to be mounted. The directory `/data/models` is mounted in `/app/models` of the container. Of course, it can be replaced with other paths.
 
 After the container is started, you can view the logs through the following command
-```python
+```bash
 docker logs dbgpt -f
 ```
 
 ### Run through MySQL database
 
-```python
+```bash
 docker run --ipc host --gpus all -d -p 3306:3306 \
 -p 5670:5670 \
 -e LOCAL_DB_HOST=127.0.0.1 \
 -e LOCAL_DB_PASSWORD=aa123456 \
 -e MYSQL_ROOT_PASSWORD=aa123456 \
--e LLM_MODEL=vicuna-13b-v1.5 \
+-e LLM_MODEL=glm-4-9b-chat \
 -e LANGUAGE=zh \
 -v /data/models:/app/models \
 --name db-gpt-allinone \
 db-gpt-allinone
 ```
 Open the browser and visit [http://localhost:5670](http://localhost:5670)
 
-- `-e LLM_MODEL=vicuna-13b-v1.5`, which means the base model uses `vicuna-13b-v1.5`. For more model usage, you can view the configuration in `/pilot/configs/model_config.LLM_MODEL_CONFIG`.
+- `-e LLM_MODEL=glm-4-9b-chat`, which means the base model uses `glm-4-9b-chat`. For more model usage, you can view the configuration in `/pilot/configs/model_config.LLM_MODEL_CONFIG`.
 - `-v /data/models:/app/models`, specifies the model file to be mounted. The directory `/data/models` is mounted in `/app/models` of the container. Of course, it can be replaced with other paths.
 
 After the container is started, you can view the logs through the following command
-```python
+```bash
 docker logs db-gpt-allinone -f
 ```
 
 ### Run through the OpenAI proxy model
-```python
+```bash
 PROXY_API_KEY="You api key"
 PROXY_SERVER_URL="https://api.openai.com/v1/chat/completions"
 docker run --gpus all -d -p 3306:3306 \

docs/docs/installation/model_service/cluster.md

@@ -25,12 +25,12 @@ By default, `Model Server` will start on port `8000`
 ## Start Model Worker
 
 :::tip
-Start `chatglm2-6b` model Worker
+Start `glm-4-9b-chat` model Worker
 :::
 
 ```shell
-dbgpt start worker --model_name chatglm2-6b \
---model_path /app/models/chatglm2-6b \
+dbgpt start worker --model_name glm-4-9b-chat \
+--model_path /app/models/glm-4-9b-chat \
 --port 8001 \
 --controller_addr http://127.0.0.1:8000
 ```
@@ -92,7 +92,7 @@ $ dbgpt model list
 +-------------------+------------+------------+------+---------+---------+-----------------+----------------------------+
 |    Model Name     | Model Type |    Host    | Port | Healthy | Enabled | Prompt Template |       Last Heartbeat       |
 +-------------------+------------+------------+------+---------+---------+-----------------+----------------------------+
-|   chatglm2-6b     |    llm     | 172.17.0.2 | 8001 |   True  |   True  |                 | 2023-09-12T23:04:31.287654 |
+|   glm-4-9b-chat     |    llm     | 172.17.0.2 | 8001 |   True  |   True  |                 | 2023-09-12T23:04:31.287654 |
 |  WorkerManager    |  service   | 172.17.0.2 | 8001 |   True  |   True  |                 | 2023-09-12T23:04:31.286668 |
 |  WorkerManager    |  service   | 172.17.0.2 | 8003 |   True  |   True  |                 | 2023-09-12T23:04:29.845617 |
 |  WorkerManager    |  service   | 172.17.0.2 | 8002 |   True  |   True  |                 | 2023-09-12T23:04:24.598439 |
@@ -124,7 +124,7 @@ MODEL_SERVER=http://127.0.0.1:8000
 
 Or it can be started directly by command to formulate the model.
 ```shell
-LLM_MODEL=chatglm2-6b dbgpt start webserver --light
+LLM_MODEL=glm-4-9b-chat dbgpt start webserver --light
 ```
 
 ## Command line usage

docs/docs/installation/model_service/stand_alone.md

@@ -1,26 +1,26 @@
 # Stand-alone Deployment
 
 ## Preparation
-```python
+```bash
 # download source code
-$ git clone https://github.com/eosphoros-ai/DB-GPT.git
+git clone https://github.com/eosphoros-ai/DB-GPT.git
 
-$ cd DB-GPT
+cd DB-GPT
 ```
 
 ## Environment installation
 
-```python
+```bash
 # create a virtual environment
-$ conda create -n dbgpt_env python=3.10
+conda create -n dbgpt_env python=3.10
 
 # activate virtual environment
-$ conda activate dbgpt_env
+conda activate dbgpt_env
 ```
 
 ## Install dependencies
 
-```python
+```bash
 pip install -e ".[default]"
 ```
 
@@ -34,19 +34,19 @@ Download LLM and Embedding model
 :::
 
 
-```python
-$ mkdir models && cd models
+```bash
+mkdir models && cd models
 
 # download embedding model, eg: text2vec-large-chinese
-$ git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
+git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
 ```
 
 :::tip
 
 Set up proxy API and modify `.env`configuration
 :::
 
-```python
+```bash
 #set LLM_MODEL TYPE
 LLM_MODEL=proxyllm
 #set your Proxy Api key and Proxy Server url
@@ -58,23 +58,23 @@ PROXY_SERVER_URL=https://api.openai.com/v1/chat/completions
 ⚠️ If you have GPU resources, you can use local models to deploy
 :::
 
-```python
-$ mkdir models && cd models
+```bash
+mkdir models && cd models
 
-# # download embedding model, eg: vicuna-13b-v1.5 or  
-$ git clone https://huggingface.co/lmsys/vicuna-13b-v1.5
+# # download embedding model, eg: glm-4-9b-chat or  
+git clone https://huggingface.co/THUDM/glm-4-9b-chat
 
 # download embedding model, eg: text2vec-large-chinese
-$ git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
+git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
 
-$ popd
+popd
 
 ```
 
 ## Command line startup
 
-```python
-LLM_MODEL=vicuna-13b-v1.5 
+```bash
+LLM_MODEL=glm-4-9b-chat 
 dbgpt start webserver --port 6006
 ```
 By default, the `dbgpt start webserver command` will start the `webserver`, `model controller`, and `model worker` through a single Python process. In the above command, port `6006` is specified.
@@ -86,16 +86,16 @@ By default, the `dbgpt start webserver command` will start the `webserver`, `mod
 :::tip
 view and display all model services
 :::
-```python
+```bash
 dbgpt model list 
 ```
 
-```python
+```bash
 # result
 +-----------------+------------+------------+------+---------+---------+-----------------+----------------------------+
 |    Model Name   | Model Type |    Host    | Port | Healthy | Enabled | Prompt Template |       Last Heartbeat       |
 +-----------------+------------+------------+------+---------+---------+-----------------+----------------------------+
-| vicuna-13b-v1.5 |    llm     | 172.17.0.9 | 6006 |   True  |   True  |                 | 2023-10-16T19:49:59.201313 |
+| glm-4-9b-chat |    llm     | 172.17.0.9 | 6006 |   True  |   True  |                 | 2023-10-16T19:49:59.201313 |
 |  WorkerManager  |  service   | 172.17.0.9 | 6006 |   True  |   True  |                 | 2023-10-16T19:49:59.246756 |
 +-----------------+------------+------------+------+---------+---------+-----------------+----------------------------+
 
@@ -105,14 +105,14 @@ Where `WorkerManager` is the management process of `Model Workers`
 :::tip
 check and verify model serving
 :::
-```python
-dbgpt model chat --model_name vicuna-13b-v1.5
+```bash
+dbgpt model chat --model_name glm-4-9b-chat
 ```
 
 The above command will launch an interactive page that allows you to talk to the model through the terminal.
 
-```python
-Chatbot started with model vicuna-13b-v1.5. Type 'exit' to leave the chat.
+```bash
+Chatbot started with model glm-4-9b-chat. Type 'exit' to leave the chat.
 
 
 You: Hello