Skip to content

Commit 91e9751

Browse files
Jintao-HuangJintao-Huang
authored
update quant_device_map (#2154)
1 parent 774374f commit 91e9751

File tree

7 files changed

+19
-72
lines changed

7 files changed

+19
-72
lines changed

‎docs/source/Instruction/命令行参数.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -382,7 +382,7 @@ export参数继承了infer参数, 除此之外增加了以下参数:
382382
- `--quant_n_samples`: 量化参数, 默认为`256`. 当设置为`--quant_method awq`时, 如果出现量化的时候OOM, 可以适度降低`--quant_n_samples`和`--quant_seqlen`. `--quant_method gptq`通常不会出现量化OOM.
383383
- `--quant_seqlen`: 量化参数, 默认为`2048`.
384384
- `--quant_batch_size`: 量化数据集的batch_size，默认为`1`.
385-
- `--quant_device_map`: 默认为`'cpu'`, 节约显存. 你可以指定为'cuda:0', 'auto', 'cpu'等, 表示量化时模型导入的设备. 该参数与实际执行量化的设备无关, 例如awq和gptq会在cuda:0中进行量化.
385+
- `--quant_device_map`: 默认为`None`. 你可以指定为'cuda:0', 'auto', 'cpu'等, 表示量化时模型导入的设备.
386386
- `--quant_output_dir`: 默认为`None`, 默认的quant_output_dir会被打印在命令行中.
387387
- `--push_to_hub`: 默认为`False`. 是否将最后的`ckpt_dir`push到ModelScope Hub中. 如果你指定了`merge_lora`, 则将推送全量参数; 如果你还指定了`quant_bits`, 则将推送量化后的模型.
388388
- `--hub_model_id`: 默认为`None`. 推送到的ModelScope Hub的model_id. 如果`push_to_hub`设置为True, 该参数必须被设置.

‎docs/source/Instruction/支持的模型和数据集.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -442,7 +442,7 @@
442442
|qwen2-vl-72b-instruct-gptq-int8|[qwen/Qwen2-VL-72B-Instruct-GPTQ-Int8](https://modelscope.cn/models/qwen/Qwen2-VL-72B-Instruct-GPTQ-Int8/summary)|^(model)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|qwen2-vl|✔|✔|✘|✘|transformers>=4.45.dev.0, qwen_vl_utils, auto_gptq>=0.5|vision, video|[Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int8](https://huggingface.co/Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int8)|
443443
|qwen2-vl-72b-instruct-awq|[qwen/Qwen2-VL-72B-Instruct-AWQ](https://modelscope.cn/models/qwen/Qwen2-VL-72B-Instruct-AWQ/summary)|^(model)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|qwen2-vl|✔|✔|✘|✘|transformers>=4.45.dev.0, qwen_vl_utils, autoawq|vision, video|[Qwen/Qwen2-VL-72B-Instruct-AWQ](https://huggingface.co/Qwen/Qwen2-VL-72B-Instruct-AWQ)|
444444
|glm4v-9b-chat|[ZhipuAI/glm-4v-9b](https://modelscope.cn/models/ZhipuAI/glm-4v-9b/summary)|^(transformer.encoder)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|glm4v|✘|✘|✘|✘|transformers>=4.42|vision|[THUDM/glm-4v-9b](https://huggingface.co/THUDM/glm-4v-9b)|
445-
|llama3_2-11b-visiont|[LLM-Research/Llama-3.2-11B-Vision](https://modelscope.cn/models/LLM-Research/Llama-3.2-11B-Vision/summary)|^(language_model\|multi_modal_projector)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|llama3_2-vision-generation|✔|✔|✘|✘|transformers>=4.45|vision|[meta-llama/Llama-3.2-11B-Vision](https://huggingface.co/meta-llama/Llama-3.2-11B-Vision)|
445+
|llama3_2-11b-vision|[LLM-Research/Llama-3.2-11B-Vision](https://modelscope.cn/models/LLM-Research/Llama-3.2-11B-Vision/summary)|^(language_model\|multi_modal_projector)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|llama3_2-vision-generation|✔|✔|✘|✘|transformers>=4.45|vision|[meta-llama/Llama-3.2-11B-Vision](https://huggingface.co/meta-llama/Llama-3.2-11B-Vision)|
446446
|llama3_2-11b-vision-instruct|[LLM-Research/Llama-3.2-11B-Vision-Instruct](https://modelscope.cn/models/LLM-Research/Llama-3.2-11B-Vision-Instruct/summary)|^(language_model\|multi_modal_projector)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|llama3_2-vision|✔|✔|✘|✘|transformers>=4.45|vision|[meta-llama/Llama-3.2-11B-Vision-Instruct](https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instruct)|
447447
|llama3_2-90b-vision|[LLM-Research/Llama-3.2-90B-Vision](https://modelscope.cn/models/LLM-Research/Llama-3.2-90B-Vision/summary)|^(language_model\|multi_modal_projector)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|llama3_2-vision-generation|✔|✔|✘|✘|transformers>=4.45|vision|[meta-llama/Llama-3.2-90B-Vision](https://huggingface.co/meta-llama/Llama-3.2-90B-Vision)|
448448
|llama3_2-90b-vision-instruct|[LLM-Research/Llama-3.2-90B-Vision-Instruct](https://modelscope.cn/models/LLM-Research/Llama-3.2-90B-Vision-Instruct/summary)|^(language_model\|multi_modal_projector)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|llama3_2-vision|✔|✔|✘|✘|transformers>=4.45|vision|[meta-llama/Llama-3.2-90B-Vision-Instruct](https://huggingface.co/meta-llama/Llama-3.2-90B-Vision-Instruct)|
@@ -653,6 +653,7 @@
653653
|cosmopedia-100k|[swift/cosmopedia-100k](https://modelscope.cn/datasets/swift/cosmopedia-100k/summary)||100000|1024.5±243.1, min=239, max=2981|multi-domain, en, qa|[HuggingFaceTB/cosmopedia-100k](https://huggingface.co/datasets/HuggingFaceTB/cosmopedia-100k)|
654654
|dolma|[swift/dolma](https://modelscope.cn/datasets/swift/dolma/summary)|v1_7|-|Dataset is too huge, please click the original link to view the dataset stat.|pretrain, quality|[allenai/dolma](https://huggingface.co/datasets/allenai/dolma)|
655655
|dolphin|[swift/dolphin](https://modelscope.cn/datasets/swift/dolphin/summary)|flan1m-alpaca-uncensored<br>flan5m-alpaca-uncensored|-|Dataset is too huge, please click the original link to view the dataset stat.|en|[cognitivecomputations/dolphin](https://huggingface.co/datasets/cognitivecomputations/dolphin)|
656+
|duet|[AI-ModelScope/Duet-v0.5](https://modelscope.cn/datasets/AI-ModelScope/Duet-v0.5/summary)||5000|1157.4±189.3, min=657, max=2344|CoT, en|[G-reen/Duet-v0.5](https://huggingface.co/datasets/G-reen/Duet-v0.5)|
656657
|evol-instruct-v2|[AI-ModelScope/WizardLM_evol_instruct_V2_196k](https://modelscope.cn/datasets/AI-ModelScope/WizardLM_evol_instruct_V2_196k/summary)||109184|480.9±333.1, min=26, max=4942|chat, en|[WizardLM/WizardLM_evol_instruct_V2_196k](https://huggingface.co/datasets/WizardLM/WizardLM_evol_instruct_V2_196k)|
657658
|fineweb|[None](https://modelscope.cn/datasets/None/summary)||-|Dataset is too huge, please click the original link to view the dataset stat.|pretrain, quality|[HuggingFaceFW/fineweb](https://huggingface.co/datasets/HuggingFaceFW/fineweb)|
658659
|gen-qa|[swift/GenQA](https://modelscope.cn/datasets/swift/GenQA/summary)||-|Dataset is too huge, please click the original link to view the dataset stat.|qa, quality, multi-task|[tomg-group-umd/GenQA](https://huggingface.co/datasets/tomg-group-umd/GenQA)|

‎docs/source_en/Instruction/Command-line-parameters.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -383,7 +383,7 @@ export parameters inherit from infer parameters, with the following added parame
383383
- `--quant_n_samples`: Quantization parameter, default is `256`. When set to `--quant_method awq`, if OOM occurs during quantization, you can moderately reduce `--quant_n_samples` and `--quant_seqlen`. `--quant_method gptq` generally does not encounter quantization OOM.
384384
- `--quant_seqlen`: Quantization parameter, default is `2048`.
385385
- `--quant_batch_size`: Calibrating batch_size，Default `1`.
386-
- `--quant_device_map`: Default is `'cpu'`, to save memory. You can specify 'cuda:0', 'auto', 'cpu', etc., representing the device to load model during quantization. This parameter is independent of the actual device that performs the quantization, such as AWQ and GPTQ which will carry out quantization on cuda:0.
386+
- `--quant_device_map`: Default is `None`, to save memory. You can specify 'cuda:0', 'auto', 'cpu', etc., representing the device to load model during quantization.
387387
- `quant_output_dir`: Default is `None`, the default quant_output_dir will be printed in the command line.
388388
- `--push_to_hub`: Default is `False`. Whether to push the final `ckpt_dir` to ModelScope Hub. If you specify `merge_lora`, full parameters will be pushed; if you also specify `quant_bits`, quantized model will be pushed.
389389
- `--hub_model_id`: Default is `None`. Model_id to push to on ModelScope Hub. If `push_to_hub` is set to True, this parameter must be set.

‎docs/source_en/Instruction/Supported-models-datasets.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -442,7 +442,7 @@ The table below introcudes all models supported by SWIFT:
442442
|qwen2-vl-72b-instruct-gptq-int8|[qwen/Qwen2-VL-72B-Instruct-GPTQ-Int8](https://modelscope.cn/models/qwen/Qwen2-VL-72B-Instruct-GPTQ-Int8/summary)|^(model)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|qwen2-vl|&#x2714;|&#x2714;|&#x2718;|&#x2718;|transformers>=4.45.dev.0, qwen_vl_utils, auto_gptq>=0.5|vision, video|[Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int8](https://huggingface.co/Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int8)|
443443
|qwen2-vl-72b-instruct-awq|[qwen/Qwen2-VL-72B-Instruct-AWQ](https://modelscope.cn/models/qwen/Qwen2-VL-72B-Instruct-AWQ/summary)|^(model)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|qwen2-vl|&#x2714;|&#x2714;|&#x2718;|&#x2718;|transformers>=4.45.dev.0, qwen_vl_utils, autoawq|vision, video|[Qwen/Qwen2-VL-72B-Instruct-AWQ](https://huggingface.co/Qwen/Qwen2-VL-72B-Instruct-AWQ)|
444444
|glm4v-9b-chat|[ZhipuAI/glm-4v-9b](https://modelscope.cn/models/ZhipuAI/glm-4v-9b/summary)|^(transformer.encoder)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|glm4v|&#x2718;|&#x2718;|&#x2718;|&#x2718;|transformers>=4.42|vision|[THUDM/glm-4v-9b](https://huggingface.co/THUDM/glm-4v-9b)|
445-
|llama3_2-11b-visiont|[LLM-Research/Llama-3.2-11B-Vision](https://modelscope.cn/models/LLM-Research/Llama-3.2-11B-Vision/summary)|^(language_model\|multi_modal_projector)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|llama3_2-vision-generation|&#x2714;|&#x2714;|&#x2718;|&#x2718;|transformers>=4.45|vision|[meta-llama/Llama-3.2-11B-Vision](https://huggingface.co/meta-llama/Llama-3.2-11B-Vision)|
445+
|llama3_2-11b-vision|[LLM-Research/Llama-3.2-11B-Vision](https://modelscope.cn/models/LLM-Research/Llama-3.2-11B-Vision/summary)|^(language_model\|multi_modal_projector)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|llama3_2-vision-generation|&#x2714;|&#x2714;|&#x2718;|&#x2718;|transformers>=4.45|vision|[meta-llama/Llama-3.2-11B-Vision](https://huggingface.co/meta-llama/Llama-3.2-11B-Vision)|
446446
|llama3_2-11b-vision-instruct|[LLM-Research/Llama-3.2-11B-Vision-Instruct](https://modelscope.cn/models/LLM-Research/Llama-3.2-11B-Vision-Instruct/summary)|^(language_model\|multi_modal_projector)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|llama3_2-vision|&#x2714;|&#x2714;|&#x2718;|&#x2718;|transformers>=4.45|vision|[meta-llama/Llama-3.2-11B-Vision-Instruct](https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instruct)|
447447
|llama3_2-90b-vision|[LLM-Research/Llama-3.2-90B-Vision](https://modelscope.cn/models/LLM-Research/Llama-3.2-90B-Vision/summary)|^(language_model\|multi_modal_projector)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|llama3_2-vision-generation|&#x2714;|&#x2714;|&#x2718;|&#x2718;|transformers>=4.45|vision|[meta-llama/Llama-3.2-90B-Vision](https://huggingface.co/meta-llama/Llama-3.2-90B-Vision)|
448448
|llama3_2-90b-vision-instruct|[LLM-Research/Llama-3.2-90B-Vision-Instruct](https://modelscope.cn/models/LLM-Research/Llama-3.2-90B-Vision-Instruct/summary)|^(language_model\|multi_modal_projector)(?!.\*(lm_head\|output\|emb\|wte\|shared)).\*|llama3_2-vision|&#x2714;|&#x2714;|&#x2718;|&#x2718;|transformers>=4.45|vision|[meta-llama/Llama-3.2-90B-Vision-Instruct](https://huggingface.co/meta-llama/Llama-3.2-90B-Vision-Instruct)|
@@ -653,6 +653,7 @@ The table below introduces the datasets supported by SWIFT:
653653
|cosmopedia-100k|[swift/cosmopedia-100k](https://modelscope.cn/datasets/swift/cosmopedia-100k/summary)||100000|1024.5±243.1, min=239, max=2981|multi-domain, en, qa|[HuggingFaceTB/cosmopedia-100k](https://huggingface.co/datasets/HuggingFaceTB/cosmopedia-100k)|
654654
|dolma|[swift/dolma](https://modelscope.cn/datasets/swift/dolma/summary)|v1_7|-|Dataset is too huge, please click the original link to view the dataset stat.|pretrain, quality|[allenai/dolma](https://huggingface.co/datasets/allenai/dolma)|
655655
|dolphin|[swift/dolphin](https://modelscope.cn/datasets/swift/dolphin/summary)|flan1m-alpaca-uncensored<br>flan5m-alpaca-uncensored|-|Dataset is too huge, please click the original link to view the dataset stat.|en|[cognitivecomputations/dolphin](https://huggingface.co/datasets/cognitivecomputations/dolphin)|
656+
|duet|[AI-ModelScope/Duet-v0.5](https://modelscope.cn/datasets/AI-ModelScope/Duet-v0.5/summary)||5000|1157.4±189.3, min=657, max=2344|CoT, en|[G-reen/Duet-v0.5](https://huggingface.co/datasets/G-reen/Duet-v0.5)|
656657
|evol-instruct-v2|[AI-ModelScope/WizardLM_evol_instruct_V2_196k](https://modelscope.cn/datasets/AI-ModelScope/WizardLM_evol_instruct_V2_196k/summary)||109184|480.9±333.1, min=26, max=4942|chat, en|[WizardLM/WizardLM_evol_instruct_V2_196k](https://huggingface.co/datasets/WizardLM/WizardLM_evol_instruct_V2_196k)|
657658
|fineweb|[None](https://modelscope.cn/datasets/None/summary)||-|Dataset is too huge, please click the original link to view the dataset stat.|pretrain, quality|[HuggingFaceFW/fineweb](https://huggingface.co/datasets/HuggingFaceFW/fineweb)|
658659
|gen-qa|[swift/GenQA](https://modelscope.cn/datasets/swift/GenQA/summary)||-|Dataset is too huge, please click the original link to view the dataset stat.|qa, quality, multi-task|[tomg-group-umd/GenQA](https://huggingface.co/datasets/tomg-group-umd/GenQA)|

‎swift/llm/export.py

Lines changed: 1 addition & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,9 @@
11
# Copyright (c) Alibaba, Inc. and its affiliates.
22
import os
3-
from typing import Dict, List, Optional
3+
from typing import List, Optional
44

55
import json
66
import torch
7-
import transformers
8-
from packaging import version
97

108
from swift.llm import get_model_tokenizer, get_template
119
from swift.utils import (check_json_format, get_logger, get_main, get_model_info, push_to_ms_hub, seed_everything,
@@ -66,67 +64,8 @@ def _get_dataset(*args, **kwargs):
6664

6765
def awq_model_quantize(awq_model, tokenizer, batch_size) -> None:
6866

69-
def _llama_rotary_emb_forward(self, x, position_ids):
70-
with torch.no_grad():
71-
if 'dynamic' in self.rope_type:
72-
self._dynamic_frequency_update(position_ids, device=x.device)
73-
74-
# Core RoPE block
75-
inv_freq_expanded = self.inv_freq[None, :, None].float().expand(position_ids.shape[0], -1, 1)
76-
position_ids_expanded = position_ids[:, None, :].float()
77-
# Force float32 (see https://github.com/huggingface/transformers/pull/29285)
78-
device_type = x.device.type
79-
device_type = device_type if isinstance(device_type, str) and device_type != 'mps' else 'cpu'
80-
with torch.autocast(device_type=device_type, enabled=False):
81-
inv_freq_expanded = inv_freq_expanded.to(position_ids_expanded.device)
82-
freqs = (inv_freq_expanded.float() @ position_ids_expanded.float()).transpose(1, 2)
83-
emb = torch.cat((freqs, freqs), dim=-1)
84-
cos = emb.cos()
85-
sin = emb.sin()
86-
87-
# Advanced RoPE types (e.g. yarn) apply a post-processing scaling factor, equivalent to scaling attention
88-
cos = cos * self.attention_scaling
89-
sin = sin * self.attention_scaling
90-
91-
return cos.to(dtype=x.dtype), sin.to(dtype=x.dtype)
92-
93-
@torch.no_grad()
94-
def _module_forward(self, x: torch.Tensor, module: torch.nn.Module, module_kwargs: Dict) -> torch.Tensor:
95-
# The original code of awq.AwqQuantizer._module_forward has a bug with n_parallel_calib_samples
96-
if self.n_parallel_calib_samples is None:
97-
# runs through all samples at once
98-
module_output = module(x, **module_kwargs)
99-
if isinstance(module_output, tuple):
100-
module_output = module_output[0]
101-
else:
102-
# memory efficiently runs through all calibration samples
103-
# but only n_parallel_calib_samples at a time
104-
module_output = []
105-
partitioned_inputs = torch.split(x, self.n_parallel_calib_samples)
106-
for idx, x_partial in enumerate(partitioned_inputs):
107-
tmp_module_kwargs = {**module_kwargs}
108-
if tmp_module_kwargs.get('attention_mask'):
109-
tmp_module_kwargs['attention_mask'] = tmp_module_kwargs['attention_mask'][idx:idx + self.
110-
n_parallel_calib_samples]
111-
partial_output = module(x_partial, **tmp_module_kwargs)
112-
113-
if isinstance(partial_output, tuple):
114-
partial_output = partial_output[0]
115-
116-
module_output.append(partial_output.cpu())
117-
118-
module_output = torch.cat(module_output, dim=0)
119-
120-
return module_output
121-
122-
import awq
12367
from awq.quantize import quantizer
12468
from transformers import AwqConfig
125-
if version.parse(awq.__version__) >= version.parse('0.2.6'):
126-
quantizer.AwqQuantizer._module_forward = _module_forward
127-
128-
if version.parse(transformers.__version__) >= version.parse('4.43.0'):
129-
transformers.models.llama.modeling_llama.LlamaRotaryEmbedding.forward = _llama_rotary_emb_forward
13069

13170
assert _args is not None
13271
logger.info(f'Quantization dataset: {_args.dataset}')
@@ -257,7 +196,6 @@ def llm_export(args: ExportArguments) -> None:
257196
model.config.quantization_config.pop('dataset', None)
258197
gptq_quantizer.save(model, args.quant_output_dir)
259198
elif args.quant_method == 'bnb':
260-
args.quant_device_map = 'auto' # cannot use cpu on bnb
261199
args.quantization_bit = args.quant_bits
262200
args.bnb_4bit_compute_dtype, args.load_in_4bit, args.load_in_8bit = args.select_bnb()
263201
model, template = prepare_model_template(args, device_map=args.quant_device_map, verbose=False)

‎swift/llm/utils/argument.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1546,7 +1546,7 @@ def handle_infer_backend(self):
15461546
self.infer_media_type = 'interleave'
15471547
self.media_type = template_info.get('media_type', 'image')
15481548
self.media_key = MediaTag.media_keys.get(self.media_type, 'images')
1549-
if self.merge_device_map is None:
1549+
if self.merge_device_map is None and not isinstance(self, ExportArguments):
15501550
self.merge_device_map = 'cpu'
15511551

15521552
@staticmethod
@@ -1661,7 +1661,7 @@ class ExportArguments(InferArguments):
16611661
quant_method: Literal['awq', 'gptq', 'bnb'] = 'awq'
16621662
quant_n_samples: int = 256
16631663
quant_seqlen: int = 2048
1664-
quant_device_map: str = 'cpu' # e.g. 'cpu', 'auto'
1664+
quant_device_map: Optional[str] = None # e.g. 'cpu', 'auto'
16651665
quant_output_dir: Optional[str] = None
16661666
quant_batch_size: int = 1
16671667

@@ -1684,8 +1684,8 @@ class ExportArguments(InferArguments):
16841684
# merge_lora, hub_token
16851685

16861686
def __post_init__(self):
1687-
if self.merge_device_map is None:
1688-
self.merge_device_map = 'cpu' if self.quant_bits > 0 else 'auto'
1687+
if self.merge_device_map is None and self.quant_bits > 0:
1688+
self.merge_device_map = 'cpu'
16891689
if self.quant_bits > 0 and self.dtype == 'AUTO':
16901690
self.dtype = 'fp16'
16911691
logger.info(f'Setting args.dtype: {self.dtype}')

‎tests/llm/test_ollama_export.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,9 @@
33
import tempfile
44
import unittest
55

6+
import transformers
7+
from packaging import version
8+
69
from swift.llm import ExportArguments, export_main
710

811
if __name__ == '__main__':
@@ -16,7 +19,8 @@ def setUp(self):
1619
self.tmp_dir = tempfile.TemporaryDirectory().name
1720

1821
def tearDown(self):
19-
shutil.rmtree(self.tmp_dir)
22+
if os.path.exists(self.tmp_dir):
23+
shutil.rmtree(self.tmp_dir)
2024
super().tearDown()
2125

2226
def test_llama3(self):
@@ -36,6 +40,9 @@ def test_llama3(self):
3640
self.assertTrue(stop in content)
3741

3842
def test_glm4(self):
43+
if version.parse(transformers.__version__) >= version.parse('4.45'):
44+
return
45+
3946
args = ExportArguments(model_type='glm4-9b-chat', to_ollama=True, ollama_output_dir=self.tmp_dir)
4047
export_main(args)
4148

0 commit comments

Comments
 (0)