v0.4.4

What's new in 0.4.4 (2023-09-19)

These are the changes in inference v0.4.4.

Bug fixes

• BUG: stop auto download from self-hosted storage for locale zh_CN by @UranusSeven in #465

Full Changelog: v0.4.3...v0.4.4

v0.4.3

What's new in 0.4.3 (2023-09-16)

These are the changes in inference v0.4.3.

Others

• CHORE: Specify xoscar version by @aresnow1 in #457

Full Changelog: v0.4.2...v0.4.3

v0.4.2

What's new in 0.4.2 (2023-09-15)

These are the changes in inference v0.4.2.

New features

• FEAT: concurrent generation by @codingl2k1 in #417
• FEAT: Support gguf by @aresnow1 in #446
• FEAT: Support OpenBuddy by @codingl2k1 in #444

Enhancements

• ENH: client support desc model by @UranusSeven in #442
• ENH: caching from self-hosted storage by @UranusSeven in #419
• ENH: Assign worker sub pool at runtime instead of pre-allocated by @ChengjieLi28 in #437
• ENH: add benchmark script by @UranusSeven in #451

Bug fixes

• BUG: Fix restful client for embedding models by @aresnow1 in #439
• BUG: cmdline double line breaker by @UranusSeven in #441
• BUG: no error raised on unsupported fmt by @UranusSeven in #443
• BUG: Xinferecen list failed if embedding models are launched by @aresnow1 in #452

Tests

• TST: skip self-hosted storage tests by @UranusSeven in #453

Documentation

• DOC: fix baichuan-2 and make naming consistent by @UranusSeven in #432
• DOC: update hot topics by @UranusSeven in #456

Others

• CI: Fix Windows CI by @codingl2k1 in #440

New Contributors

• @ChengjieLi28 made their first contribution in #437

Full Changelog: v0.4.1...v0.4.2

v0.4.1

What's new in 0.4.1 (2023-09-07)

These are the changes in inference v0.4.1.

Bug fixes

• BUG: Searching in UI results in white screen by @Bojun-Feng in #431
• BUG: Include json in MANIFEST.in by @aresnow1 in #435

Documentation

• DOC: Embedding models by @aresnow1 in #421

Full Changelog: v0.4.0...v0.4.1

v0.4.0

What's new in 0.4.0 (2023-09-06)

These are the changes in inference v0.4.0.

New features

• FEAT: Support CodeLlama-Instruct by @jiayini1119 in #414
• FEAT: Add embedding models support by @aresnow1 in #418
• FEAT: Support replica by @codingl2k1 in #410
• FEAT: support baichuan2 by @UranusSeven in #425

Bug fixes

• BUG: cmdline chat duplicates user msg by @UranusSeven in #428
• BUG: llama_cpp model context length by @UranusSeven in #429

Documentation

• DOC: update readme by @UranusSeven in #423

New Contributors

• @codingl2k1 made their first contribution in #410

Full Changelog: v0.3.0...v0.4.0

v0.3.0

What's new in 0.3.0 (2023-09-04)

These are the changes in inference v0.3.0.

Enhancements

• ENH: help message for CLI by @Bojun-Feng in #367

Bug fixes

• BUG: Asking to pad but the tokenizer does not have a padding token by @jiayini1119 in #407
• BUG: empty results for non-stream inference by @UranusSeven in #415
• BUG: Make context_length optional in model family by @Bojun-Feng in #394

Others

• EHN: auto retry download on network errors by @jiayini1119 in #405
• FEAT : Add Model Dashboard by @Bojun-Feng in #366

Full Changelog: v0.2.3...v0.3.0

v0.2.3

What's new in 0.2.3 (2023-08-30)

These are the changes in inference v0.2.3.

Bug fixes

• BUG: fix subprocess log on linux by @UranusSeven in #357

Others

• CHORE: lock llama-cpp-python version by @UranusSeven in #406

Full Changelog: v0.2.2...v0.2.3

v0.2.2

What's new in 0.2.2 (2023-08-25)

These are the changes in inference v0.2.2.

New features

• FEAT: Support Llama-2 PyTorch model by @jiayini1119 in #387
• FEAT: code-llama by @UranusSeven in #402

Enhancements

• ENH: Update max_tokens to 32k by @Bojun-Feng in #386

Bug fixes

• BUG: last token is duplicated by @UranusSeven in #398

Documentation

• DOC: readme enhancements by @aresnow1 in #390

Others

• fix chatglm params by @Bojun-Feng in #400

Full Changelog: v0.2.1...v0.2.2

v0.2.1

What's new in 0.2.1 (2023-08-23)

These are the changes in inference v0.2.1.

New features

• FEAT: Adding support for stream returning completion results by @takatost in #368

Enhancements

• ENH: including default context length into model family by @Bojun-Feng in #374

Bug fixes

• BUG: PyTorch generate config max_new_tokens not compatible with RESTful API by @Bojun-Feng in #373
• BUG: llm class match by @UranusSeven in #383
• BUG: return chat model handle by @UranusSeven in #382
• BUG: xinference cache dir doesn't exist by @UranusSeven in #380

New Contributors

• @takatost made their first contribution in #368

Full Changelog: v0.2.0...v0.2.1

v0.2.0

What's new in 0.2.0 (2023-08-19)

These are the changes in inference v0.2.0.

New features

• FEAT:Support Starchat-Beta and StarCoderPlus with Pytorch. by @RayJi01 in #333
• FEAT: Support Ctransformers by @RayJi01 in #289
• FEAT: internlm by @UranusSeven in #352
• FEAT: Support Vicuna-v1.5 and Vicuna-v1.5-16k by @RayJi01 in #343
• FEAT: wizardmath by @UranusSeven in #351
• FEAT: support generate/chat/create_embedding/register/unregister/registrations method in cmdline by @pangyoki in #363

Enhancements

• ENH: Use Llama 2 chat for inference in LangChain QA demo by @jiayini1119 in #324
• ENH: cache from URI by @UranusSeven in #350
• ENH: Update System Prompt for llama-2-chat by @Bojun-Feng in #359
• ENH: RESTful client supports custom model APIs by @jiayini1119 in #360
• BLD: fix readthedocs by @UranusSeven in #340
• BLD: fix readthedocs by @UranusSeven in #342

Bug fixes

• BUG: Chatglm max_length doesn't work by @Bojun-Feng in #349
• BUG: builtin stop_token_ids changes by @UranusSeven in #353
• BUG: custom model related bugs by @UranusSeven in #364

Documentation

• DOC: framework by @UranusSeven in #332
• DOC: models by @UranusSeven in #338
• DOC: fix README.md by @UranusSeven in #354
• DOC: update builtin models by @UranusSeven in #365

Others

• FEAT : Add Model Dashboard by @Bojun-Feng in #334
• Revert "FEAT : Add Model Dashboard" by @UranusSeven in #362

Full Changelog: v0.1.3...v0.2.0