v0.5.1:bug fix release

What's Changed

• bump version into v0.5.0 by @XuehaoSun in #538
• fix triton multiple gpus and some other issues by @wenhuach21 in #539

Full Changelog: v0.5.0...v0.5.1

v0.5.0

Highlights

• refine autoround format inference, support 2,3,4,8 bits and marlin kernel and fix several bugs in auto-round format
• support xpu in tuning and inference by @wenhuach21 in #481
• support for more vlms by @n1ck-guo in #390
• change quantization method name and made several refinements by @wenhuach21 in #500
• support rtn via iters==0 by @wenhuach21 in #510
• fix bug of mix calib dataset by @n1ck-guo in #492

What's Changed

• support xpu in tuning and inference by @wenhuach21 in #481
• add light ut, fixtypos by @WeiweiZhang1 in #483
• bump into v0.4.7 by @XuehaoSun in #487
• fix dataset combine bug by @wenhuach21 in #489
• fix llama 8b time cost by @WeiweiZhang1 in #490
• update 2bits acc results by @WeiweiZhang1 in #491
• fix bug of mix calib dataset by @n1ck-guo in #492
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #494
• [GGUF support step3]patch for double quant by @n1ck-guo in #473
• refine inference backend/code step 1 by @wenhuach21 in #486
• refine inference step 2 by @wenhuach21 in #498
• change quantization method name and made several refinements by @wenhuach21 in #500
• fix bug of awq/gptq modules_to_not_convert by @n1ck-guo in #501
• use --tasks to control evaluation enabling by @wenhuach21 in #505
• fix gguf eval regression bug by @n1ck-guo in #506
• change to new api in readme by @wenhuach21 in #507
• fix setup issue on cuda machine by @wenhuach21 in #511
• support rtn via iters==0 by @wenhuach21 in #510
• fix critical bug of get_multimodal_block_names by @n1ck-guo in #509
• Update requirements-lib.txt by @yiliu30 in #513
• add group_size divisible check in backend by @wenhuach21 in #512
• support for more vlms by @n1ck-guo in #390
• move gguf-dq test to cuda by @n1ck-guo in #520
• fix bs!=1 for gemma and MiniMax-Text-01 by @wenhuach21 in #515
• add regex support in layer_config setting by @wenhuach21 in #519
• patch for vlm by @n1ck-guo in #518
• rename backend to packing_format in config.json by @wenhuach21 in #521
• fix example's model_dtype by @WeiweiZhang1 in #523
• rm fp16 export in autoround format by @wenhuach21 in #525
• update convert_hf_to_gguf to support more models by @n1ck-guo in #524
• fix light config by @WeiweiZhang1 in #526
• fix typos, add model card link for VLMs by @WeiweiZhang1 in #527
• add backend readme by @wenhuach21 in #528
• update mllm readme by @WeiweiZhang1 in #530
• fix bug of cuda ut by @n1ck-guo in #532
• fix inference issue by @wenhuach21 in #529
• update readme by @wenhuach21 in #531
• refine readme by @WeiweiZhang1 in #536
• fix cuda ut by @n1ck-guo in #537

Full Changelog: v0.4.7...v0.5.0

v0.4.7

Highlights

Support W4AFP8 for HPU. Please refer to Intel Neural Compressor for guidance on running these models. by @yiliu30 in #467

Support packing immediately in new quantization api to save ram usage by @wenhuach21 in #466

20x for awq and 4x for gptq packing speedup on cuda by @wenhuach21 in #459

Support auto-round-light to speed up the tuning process @WeiweiZhang1 in #454

Fix critic bug of mxfp4 in tuningby @wenhuach21 in #451

What's Changed

• step-1 support naive double quant in tuning by @wenhuach21 in #442
• fix critic bug of mxfp4 by @wenhuach21 in #451
• update readme by @wenhuach21 in #455
• update eval by @n1ck-guo in #450
• awq exporting bugfix by @WeiweiZhang1 in #456
• Support force loading into autoround Format by @WeiweiZhang1 in #453
• 20x for awq and 4x for gptq packing speedup by @wenhuach21 in #459
• fixl eval bug by @n1ck-guo in #461
• [STEP-1]W4Afp8 export by @wenhuach21 in #378
• [HPU] Update W4A8 for HPU by @yiliu30 in #467
• support for gemma3 by @n1ck-guo in #468
• upload_auto-round-light results by @WeiweiZhang1 in #454
• GGUF support step2: add naive Q2_KS and Q4_KS by @n1ck-guo in #448
• fix incorrect recipe data by @WeiweiZhang1 in #471
• support for mistral3 by @n1ck-guo in #472
• support to export gemma3 gguf format by @n1ck-guo in #470
• Increase unit test timeout from 120 to 240 minutes by @XuehaoSun in #474
• support packing immediately in new quantization api to save ram usage by @wenhuach21 in #466
• rm redundant line break by @WeiweiZhang1 in #475
• Temporarily close qxk api for new release by @n1ck-guo in #478
• add restrict for exporting act-quant models by @n1ck-guo in #480

Full Changelog: v0.4.6...v0.4.7

v0.4.6

Highlights:

1 set torch compile to false by default in #447
2 Fix packing hang and force to fp16 at exporting in #430
3 align auto_quantizer with Transformers 4.49 in #437

What's Changed

• Fix packing hang, torch compile and force to fp16 at exporting by @wenhuach21 in #430
• fix nblocks issues by @wenhuach21 in #432
• rm gc collect in packing by @wenhuach21 in #438
• align auto_quantizer with main branch in Transformers by @WeiweiZhang1 in #437
• [HPU]Fix compile bug when quant layer by @yiliu30 in #441
• remove tricky setting in mxfp4 by @wenhuach21 in #445
• fix bug of evaluate user model by @n1ck-guo in #444
• Refine funcs by @WeiweiZhang1 in #446
• set torch compile to false by default by @WeiweiZhang1 in #447

Full Changelog: v0.4.5...v0.4.6

v0.4.5

Highlights:
We have enhanced support for extremely large models with the following updates:

Multi-Card Tuning Support: Added basic support for multi-GPU tuning. #415 support naive multi-card tuning

Accelerated Packing Stage: Improved the packing speed (2X-4X)for AutoGPTQ and AutoAWQ formats by leveraging cuda. #407 speedup packing stage for autogptq and autoawq forma

Deepseek V3 GGUF Export: Introduced support for exporting models to the Deepseek V3 GGUF format. #416 support to export deepseek v3 gguf format

What's Changed

• update format readme by @wenhuach21 in #411
• fix log bug and device "auto" bug by @n1ck-guo in #409
• speedup packing stage for autogptq and autoawq format by @wenhuach21 in #407
• support naive multi-card tuning by @wenhuach21 in #415
• support bf16 inference for autoround format by @wenhuach21 in #420
• enable backup pile dataset loading by @WeiweiZhang1 in #417
• fix evaluation device bug, relate to issue 413 by @n1ck-guo in #419
• support to export deepseek v3 gguf format by @n1ck-guo in #416
• fix cuda UT torch_dtype by @WeiweiZhang1 in #423
• fix eval trust_remote_code by @n1ck-guo in #424

Full Changelog: v0.4.4...v0.4.5

v0.4.4 release

Highlights:
1 Fix install issue in #387
2 support to export gguf q4_0 and q4_1 format in #393
3 fix llm cmd line seqlen issue in #399

What's Changed

• fix a critic bug of static activation quantization by @wenhuach21 in #392
• vlm 70B+ in single card by @n1ck-guo in #395
• enhance calibration dataset and add awq pre quantization warning by @wenhuach21 in #396
• support awq format for vlms by @WeiweiZhang1 in #398
• [critic bug]fix llm example seqlen issue by @WeiweiZhang1 in #399
• fix device auto issue by @wenhuach21 in #400
• Fix auto-round install & bump into 0.4.4 by @XuehaoSun in #387
• fix dtype converting issue by @wenhuach21 in #403
• support for deepseek vl2 by @n1ck-guo in #401
• llm_layer_config_bugfix by @WeiweiZhang1 in #406
• support awq with qbits, only support sym by @wenhuach21 in #402
• support to export gguf q4_0 and q4_1 format by @n1ck-guo in #393

Full Changelog: v0.4.3...v0.4.4

v0.4.3: bug fix release

Highlights:
fix incorrect device setting in autoround format inference by @WeiweiZhang1 in #383
remove the dependency on AutoGPTQ by @XuehaoSun in #380

What's Changed

• support_llava_hf_vlm_example by @WeiweiZhang1 in #381
• fix block_name_to_quantize by @WeiweiZhang1 in #382
• fix incorrect device setting in autoround format inference by @WeiweiZhang1 in #383
• refine homepage, update model links by @WeiweiZhang1 in #385
• update eval basic usage by @n1ck-guo in #384
• refine error msg and dump more log in the tuning by @wenhuach21 in #386
• remove the dependency on AutoGPTQ for CPU and bump to V0.4.3 by @XuehaoSun in #380

Full Changelog: v0.4.2...v0.4.3

v0.4.2: bug fix release

Highlights

1 Fix autoawq exporting issue
2 remove bias exporting if possible in autogptq format

What's Changed

• bump version into v0.4.1 by @XuehaoSun in #350
• Update docker user and remove baseline UT by @XuehaoSun in #347
• delete llm example and refine readme by @wenhuach21 in #354
• Simulated W4Afp8 Quantization by @wenhuach21 in #331
• add QWQ-32B, VLM, Qwen2.5, Llama3.1 int4 models by @wenhuach21 in #356
• fix awq exporting by @wenhuach21 in #358
• Tensor reshape bugfix by @WeiweiZhang1 in #364
• fix awq backend and fp_layers issue by @wenhuach21 in #363
• fix awq exporting bugs by @wenhuach21 in #365
• fix bug of only_text_test check due to inference issue on cpu by @n1ck-guo in #362
• add gpu test by @wenhuach21 in #367
• using multicard when device set to "auto" by @n1ck-guo in #368
• quant_block_names enhancement by @WeiweiZhang1 in #369
• [HPU] Add lazy mode back by @yiliu30 in #371
• remove bias exporting if possible in autogptq format by @wenhuach21 in #375
• save processor automatically by @n1ck-guo in #372
• Add gpu ut by @wenhuach21 in #370
• fix gpu ut by @n1ck-guo in #376
• fix typos by @wenhuach21 in #377

Full Changelog: v0.4.1...v0.4.2

v0.4.1: bug fix release

Highlights:

• Fixed vllm calibration infinite loop issue
• Corrected the default value for the sym argument in the API configuration.

What's Changed

• fix typo by @wenhuach21 in #342
• vllm/llama-vision llava calibration infinite loop fix by @WeiweiZhang1 in #343
• [HPU]Enhance numba check by @yiliu30 in #345
• [VLM]fix bs and grad reset by @n1ck-guo in #344
• [HPU]Enhance installation check by @yiliu30 in #346
• [Critical Bug]API use sym as default by @wenhuach21 in #349
• triton backend requires< 3.0 by @wenhuach21 in #348

Full Changelog: v0.4...v0.4.1

v0.4

Highlights

[Experimental Feature] We provide API support for VLM models
[Kernel] We add ipex support for intel cpu
[Bug fix] We fix tuning bug for glm4 model
[Enhancement] better align gradient_accumulate_steps behavior for varied length input

What's Changed

• refine AuoRound format and support marlin repacking by @wenhuach21 in #280
• update readme for v0.3.1 release by @wenhuach21 in #283
• update readme for cpu inference by @wenhuach21 in #284
• avoid deterministic algorithm warning in inference by @wenhuach21 in #285
• fix mx_fp issues by @wenhuach21 in #286
• update torch ao integration information by @wenhuach21 in #287
• Refine code by @wenhuach21 in #291
• Add ipex support for intel cpu by @wenhuach21 in #292
• fix ipex tqdm mismatch issue by @wenhuach21 in #293
• fix bug of backend by @wenhuach21 in #294
• [Experimental Feature]support for common hf multimodel by @n1ck-guo in #276
• use torch.compile by default for PyTorch versions 2.6 and above by @wenhuach21 in #295
• refine forward hook by @WeiweiZhang1 in #290
• eval for MLLMs by @n1ck-guo in #296
• mllm eval bug fix by @n1ck-guo in #297
• Port Numba-based packing from INC by @yiliu30 in #301
• refine model config file for mixed precision quantization by @wenhuach21 in #300
• fix glm4-9b batch dim issue by @wenhuach21 in #304
• better align gradient_accumulate_steps for varied length input by @wenhuach21 in #309
• Enable torch.compile on HPU by @yiliu30 in #307
• Update autogptq exporting by @wenhuach21 in #310
• fix typo by @wenhuach21 in #311
• qwen2 vision quantization bugfix by @WeiweiZhang1 in #313
• multiple gpu evaluation/calibration refine by @wenhuach21 in #312
• HPU only release binary by @yiliu30 in #302
• patch 1 for mllm by @n1ck-guo in #298
• add torch compile arg by @wenhuach21 in #314
• fix merge error by @n1ck-guo in #316
• Update the check for HPU by @yiliu30 in #318
• fix eval device issue by @wenhuach21 in #319
• fix multiple device bug by @wenhuach21 in #321
• add warning for no gptq exllamav2 kernel by @wenhuach21 in #324
• add pile calib, rename quant_block_list to to_quant_block_names by @WeiweiZhang1 in #322
• fix autogptq version error by @wenhuach21 in #325
• new mllm eval by @n1ck-guo in #317
• Add cpu only version by @XuehaoSun in #315
• set default mllm dataset by @n1ck-guo in #327
• fix fp_layers issue and force to FP16 on cuda for autoround format inference by @wenhuach21 in #326
• fix the bug of test model support for test-only by @n1ck-guo in #328
• Increase unit test timeout to 120 minutes by @XuehaoSun in #330
• fix mllm dataset config bug and add gptq cuda backend by @wenhuach21 in #329
• add tips and tricks for llm&mllm quantization by @wenhuach21 in #333
• fix eval_bs in fake format and reset auto-gptq exporting max_shard_size by @wenhuach21 in #332
• fix model_dtype issue and reformat mllm code by @wenhuach21 in #335
• Exclude markdown files from unit test pipelines by @XuehaoSun in #337
• refine mllm docs by @WeiweiZhang1 in #336
• cogvlm doc by @n1ck-guo in #339
• add qwen2.5 recipe and refine readme by @WeiweiZhang1 in #338
• add cogvlm recipe and refine readme by @WeiweiZhang1 in #340
• refine mllm API and add help info by @n1ck-guo in #334

Full Changelog: v0.3.1...v0.4