使用TurboMind 推理 + Python 代码集成的方式报错 #1835
Description
Checklist
- 1. I have searched related issues but cannot get the expected help.
- 2. The bug has not been fixed in the latest version.
Describe the bug
Traceback (most recent call last):
File "/mnt/data/data/user/zhanghanwei/LLM_model/SpeechGPT-main/speechgpt/src/infer/cli_infer-lmdeploy.py", line 269, in interact
self.forward([prompt])
File "/mnt/data/data/user/zhanghanwei/LLM_model/SpeechGPT-main/speechgpt/src/infer/cli_infer-lmdeploy.py", line 186, in forward
for a, b in enumerate(outputs):
TypeError: 'EngineOutput' object is not iterable
'EngineOutput' object is not iterable
Reproduction
下面是我的代码，参考自（https://xujinzh.github.io/2024/01/13/ai-internlm-lmdeploy/index.html）：
for outputs in self.generator.stream_infer(
session_id=0,
input_ids=[input_ids]):
res, tokens = outputs[0]
Environment
Package Version Editable project location
----------------------------- --------------------- ---------------------------------------------------------------------------
absl-py 2.0.0
accelerate 0.31.0
adaseq 0.6.6
addict 2.4.0
aiofiles 23.2.1
aiohttp 3.9.3
aioprometheus 23.12.0
aiosignal 1.3.1
albumentations 1.3.1
alias-free-torch 0.0.6
aliyun-python-sdk-core 2.15.1
aliyun-python-sdk-kms 2.16.2
altair 5.3.0
aniso8601 9.0.1
annotated-types 0.6.0
antlr4-python3-runtime 4.9.3
anyio 4.2.0
apex 0.1
appdirs 1.4.4
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
asttokens 2.4.1
astunparse 1.6.3
async-lru 2.0.4
async-timeout 4.0.3
attrs 23.2.0
audioread 3.0.1
auto-gptq 0.6.0
autoawq 0.2.4
autoawq_kernels 0.0.6
av 12.0.0
Babel 2.14.0
basicsr 1.4.2
beartype 0.18.5
beautifulsoup4 4.12.3
bidict 0.23.1
biopython 1.82
bitarray 2.9.2
bitsandbytes 0.42.0
bitstring 4.2.0
black 23.12.1
bleach 6.1.0
blis 0.7.11
blobfile 2.1.1
bmt-clipit 1.0
boltons 23.0.0
boto3 1.34.88
botocore 1.34.88
brotlipy 0.7.0
cachetools 5.3.2
catalogue 2.0.10
certifi 2023.11.17
cffi 1.15.1
cfgv 3.4.0
charset-normalizer 2.0.4
chumpy 0.70
cityscapesScripts 2.2.3
click 8.1.7
clip 1.0
cloudpathlib 0.16.0
cloudpickle 3.0.0
colorama 0.4.6
coloredlogs 14.0
comm 0.2.1
conda 23.9.0
conda-content-trust 0.2.0
conda-libmamba-solver 23.9.1
conda-package-handling 2.2.0
conda_package_streaming 0.9.0
confection 0.1.4
ConfigArgParse 1.7
contextlib2 21.6.0
contourpy 1.2.0
control-ldm 0.0.1
crcmod 1.7
cryptography 41.0.3
cycler 0.12.1
cymem 2.0.8
Cython 0.29.36
dacite 1.8.1
dataclasses 0.6
datasets 2.16.1
ddpm-guided-diffusion 0.0.0
debugpy 1.8.0
decorator 4.4.2
decord 0.6.0
deepspeed 0.12.6
defusedxml 0.7.1
descartes 1.1.0
detectron2 0.6
dgl 1.1.3+cu121
diffusers 0.27.2
dill 0.3.7
Distance 0.1.3
distlib 0.3.8
dnspython 2.3.0
docstring_parser 0.16
easydict 1.13
easyrobust 0.2.4
edit-distance 1.0.6
editdistance 0.5.2
einops 0.7.0
embeddings 0.0.8
emoji 2.11.1
espnet-tts-frontend 0.0.3
et-xmlfile 1.1.0
evaluate 0.4.2
eventlet 0.36.1
exceptiongroup 1.2.0
executing 2.0.1
expecttest 0.2.1
face-alignment 1.4.1
fairscale 0.4.13
fairseq 0.12.2
fastai 2.7.14
fastapi 0.109.0
fastcore 1.5.29
fastdownload 0.0.7
fastjsonschema 2.19.1
fastprogress 1.0.3
fasttext 0.9.2
ffmpeg 1.4
ffmpeg-python 0.2.0
ffmpy 0.3.2
filelock 3.13.1
fire 0.6.0
flake8 7.0.0
flash-attn 2.5.2
Flask 2.2.5
Flask-Cors 4.0.0
Flask-RESTful 0.3.10
Flask-SocketIO 5.3.6
flask-talisman 1.1.0
flatbuffers 23.5.26
fonttools 4.47.0
fqdn 1.5.1
frozenlist 1.4.1
fsspec 2023.10.0
ftfy 6.2.0
funasr 1.0.14
funcodec 0.2.0
funtextprocessing 0.1.1
future 0.18.3
fvcore 0.1.5.post20221221
g2p 2.0.0
g2p-en 2.1.0
gast 0.5.4
gekko 1.0.6
google-auth 2.26.1
google-auth-oauthlib 1.0.0
google-pasta 0.2.0
gradio 3.50.2
gradio_client 0.6.1
greenlet 3.0.3
grpcio 1.60.0
h11 0.14.0
h5py 3.10.0
hdbscan 0.8.33
hjson 3.1.0
httpcore 1.0.5
httptools 0.6.1
httpx 0.27.0
huggingface-hub 0.20.3
humanfriendly 10.0
hydra-core 1.3.2
HyperPyYAML 1.2.2
identify 2.5.36
idna 3.4
imageio 2.33.1
imageio-ffmpeg 0.4.9
imgaug 0.4.0
importlib-metadata 7.0.1
importlib_resources 6.4.0
inflect 7.0.0
iniconfig 2.0.0
iopath 0.1.9
ipdb 0.13.13
ipykernel 6.28.0
ipython 8.19.0
isoduration 20.11.0
isort 5.13.2
itsdangerous 2.2.0
jaconv 0.3.4
jamo 0.4.1
jedi 0.19.1
jieba 0.42.1
Jinja2 3.1.2
jmespath 0.10.0
joblib 1.3.2
json-tricks 3.17.3
json5 0.9.14
jsonpatch 1.32
jsonplus 0.8.0
jsonpointer 2.1
jsonschema 4.21.1
jsonschema-specifications 2023.12.1
jupyter_client 8.6.0
jupyter_core 5.7.0
jupyter-events 0.9.0
jupyter-lsp 2.2.2
jupyter_server 2.12.5
jupyter_server_terminals 0.5.2
jupyterlab 4.0.12
jupyterlab_pygments 0.3.0
jupyterlab_server 2.25.2
kaldiio 2.18.0
kantts 1.0.1
keras 2.14.0
kiwisolver 1.4.5
kornia 0.7.2
kornia_rs 0.1.3
kwsbp 0.0.6
langcodes 3.3.0
lap 0.4.0
lazy_loader 0.3
libclang 16.0.6
libmambapy 1.5.1
librosa 0.10.1
lightning-utilities 0.11.2
llvmlite 0.41.1
lmdb 1.4.1
lmdeploy 0.4.2 /mnt/data/data/user/zhanghanwei/LLM_model/SpeechGPT-main/speechgpt/lmdeploy
local-attention 1.9.1
lpips 0.1.4
lxml 4.9.4
lyft-dataset-sdk 0.0.8
Markdown 3.5.1
markdown-it-py 3.0.0
MarkupSafe 2.1.3
matplotlib 3.5.3
matplotlib-inline 0.1.6
mccabe 0.7.0
mdurl 0.1.2
megatron-util 1.3.2
MinDAEC 0.0.2
mir-eval 0.7
mistune 3.0.2
ml-collections 0.1.1
ml-dtypes 0.2.0
mmcls 0.25.0
mmcv-full 1.7.0+torch2.1.1cu121
mmdet 2.28.2
mmdet3d 1.0.0a1
mmengine-lite 0.10.4
mmsegmentation 0.30.0
mock 5.1.0
modelscope 1.14.0
more-itertools 10.2.0
moviepy 1.0.3
mpi4py 3.1.5
mpmath 1.3.0
ms-swift 2.0.2
msgpack 1.0.7
multidict 6.0.4
multiprocess 0.70.15
munkres 1.1.4
murmurhash 1.0.10
mypy-extensions 1.0.0
nbclient 0.9.0
nbconvert 7.14.2
nbformat 5.9.2
nerfacc 0.2.2
nest-asyncio 1.5.8
networkx 3.2.1
ninja 1.11.1.1
nltk 3.8.1
nodeenv 1.8.0
notebook_shim 0.2.3
numba 0.58.1
numpy 1.26.3
nuscenes-devkit 1.1.11
nvdiffrast 0.3.1
oauthlib 3.2.2
omegaconf 2.3.0
onnx 1.16.0
onnxruntime 1.17.3
onnxsim 0.4.36
open-clip-torch 2.24.0
openai-whisper 20231117
opencv-python 4.9.0.80
opencv-python-headless 4.9.0.80
openpyxl 3.1.2
opt-einsum 3.3.0
optimum 1.19.1
orjson 3.9.12
oss2 2.18.4
overrides 7.7.0
packaging 23.1
pai-easycv 0.11.6
paint-ldm 0.0.0
pandas 2.2.0
pandocfilters 1.5.1
panopticapi 0.1
panphon 0.20.0
parso 0.8.3
pathspec 0.12.1
peft 0.4.0
pexpect 4.9.0
phaseaug 1.0.1
pickleshare 0.7.5
pillow 10.2.0
pip 24.1
platformdirs 4.1.0
plotly 5.21.0
pluggy 1.5.0
plyfile 1.0.3
pointnet2 0.0.0
pooch 1.8.0
portalocker 2.8.2
pre-commit 3.7.0
preshed 3.0.9
prettytable 3.10.0
proglog 0.1.10
prometheus-client 0.19.0
prompt-toolkit 3.0.43
protobuf 3.20.3
psutil 5.9.7
ptflops 0.7.2.2
ptyprocess 0.7.0
pure-eval 0.2.2
py-cpuinfo 9.0.0
py-sound-connect 0.2.1
pyarrow 15.0.0
pyarrow-hotfix 0.6
pyasn1 0.5.1
pyasn1-modules 0.3.0
pybind11 2.11.1
pyclipper 1.3.0.post5
pycocoevalcap 1.2
pycocotools 2.0.7
pycodestyle 2.11.1
pycosat 0.6.6
pycparser 2.21
pycryptodome 3.20.0
pycryptodomex 3.20.0
pydantic 2.5.3
pydantic_core 2.14.6
pyDeprecate 0.3.2
pydot 2.0.0
pydub 0.25.1
pyflakes 3.2.0
Pygments 2.17.2
PyMCubes 0.1.4
pynini 2.1.5
pynndescent 0.5.12
pynvml 11.5.0
pyOpenSSL 23.2.0
pyparsing 3.1.1
pypinyin 0.44.0
pyquaternion 0.9.9
PySocks 1.7.1
pysptk 0.1.18
pytest 8.1.1
pythainlp 5.0.2
python-crfsuite 0.9.10
python-dateutil 2.8.2
python-dotenv 1.0.1
python-engineio 4.9.0
python-json-logger 2.0.7
python-multipart 0.0.9
python-rapidjson 1.17
python-socketio 5.11.2
pytorch-lightning 1.7.7
pytorch-metric-learning 2.5.0
pytorch-wavelets 1.3.0
pytorch-wpe 0.0.1
pytorch3d 0.7.5
pytz 2023.4
pyvi 0.1.1
PyWavelets 1.6.0
PyYAML 6.0.1
pyzmq 25.1.2
quantile-python 1.1
qudida 0.0.4
rapidfuzz 3.8.1
ray 2.9.1
referencing 0.33.0
regex 2023.12.25
requests 2.31.0
requests-oauthlib 1.3.1
resampy 0.4.2
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rich 13.7.1
rotary-embedding-torch 0.5.3
rouge 1.0.1
rouge-score 0.0.4
rpds-py 0.17.1
rsa 4.9
ruamel.yaml 0.18.6
ruamel.yaml.clib 0.2.8
ruff 0.4.9
s3transfer 0.10.1
sacrebleu 2.4.0
sacremoses 0.1.1
safetensors 0.4.1
scikit-image 0.19.3
scikit-learn 1.3.2
scipy 1.11.4
seaborn 0.13.2
semantic-version 2.10.0
Send2Trash 1.8.2
sentencepiece 0.1.99
seqeval 1.2.2
setuptools 68.0.0
Shapely 1.8.4
shellingham 1.5.4
shortuuid 1.0.13
shotdetect-scenedetect-lgss 0.0.4
shtab 1.7.1
simple-websocket 1.0.0
simplejson 3.19.2
six 1.16.0
sklearn-crfsuite 0.3.6
smart-open 6.4.0
smplx 0.1.28
sniffio 1.3.0
sortedcontainers 2.4.0
soundfile 0.12.1
soupsieve 2.5
sox 1.5.0
soxr 0.3.7
spacy 3.7.4
spacy-legacy 3.0.12
spacy-loggers 1.0.5
speechbrain 1.0.0
srsly 2.4.8
sse-starlette 2.1.0
stack-data 0.6.3
stanza 1.8.2
starlette 0.35.1
subword-nmt 0.3.8
sympy 1.12
tabulate 0.9.0
taming-transformers-rom1504 0.0.6
tenacity 8.2.3
tensorboard 2.16.2
tensorboard-data-server 0.7.2
tensorboardX 2.6.2.2
tensordict 0.2.1
tensorflow 2.14.0
tensorflow-estimator 2.14.0
tensorflow-io-gcs-filesystem 0.35.0
termcolor 2.4.0
terminado 0.18.0
terminaltables 3.1.10
text-unidecode 1.3
text2sql-lgesql 1.3.0
tf_keras-nightly 2.16.0.dev2024013110
tf-slim 1.1.0
thinc 8.2.3
thop 0.1.1.post2209072238
threadpoolctl 3.2.0
tifffile 2024.1.30
tiktoken 0.5.2
timm 0.9.16
tinycss2 1.2.1
tinycudann 1.7
tokenizers 0.15.1
toml 0.10.2
tomli 2.0.1
tomlkit 0.12.0
toolz 0.12.1
torch 2.1.2+cu121
torch-complex 0.4.3
torch-scatter 2.1.2
torchaudio 2.1.2+cu121
torchdata 0.7.1
torchmetrics 0.11.4
torchsde 0.2.6
torchsummary 1.5.1
torchvision 0.16.2+cu121
tornado 6.4
tqdm 4.65.0
traitlets 5.14.1
trampoline 0.1.2
transformers 4.38.2
transformers-stream-generator 0.0.4
trimesh 2.35.39
triton 2.1.0
tritonclient 2.46.0
trl 0.8.6
truststore 0.8.0
ttsfrd 0.2.1
typeguard 2.13.3
typer 0.12.3
types-python-dateutil 2.8.19.20240106
typing 3.7.4.3
typing_extensions 4.9.0
tyro 0.8.3
tzdata 2023.4
ujson 5.9.0
umap-learn 0.5.6
unicodecsv 0.14.1
unicodedata2 15.1.0
unicore 1.2.1
Unidecode 1.3.8
uri-template 1.3.0
urllib3 2.2.1
utils 1.0.2
uvicorn 0.27.0.post1
uvloop 0.19.0
videofeatures-clipit 1.0
virtualenv 20.25.3
vllm 0.3.0
wasabi 1.1.2
watchfiles 0.21.0
wcwidth 0.2.12
weasel 0.3.4
webcolors 1.13
webencodings 0.5.1
websocket-client 1.7.0
websockets 11.0.3
Werkzeug 3.0.1
wget 3.2
wheel 0.41.2
wrapt 1.14.1
wsproto 1.2.0
xformers 0.0.23.post1
xtcocotools 1.14
xxhash 3.4.1
yacs 0.1.8
yapf 0.30.0
yarl 1.9.4
zhconv 1.4.3
zipp 3.17.0
zstandard 0.19.0Error traceback
No response