Search results

int4_weight_only get plain weight are padded

quantize

question

Status: Open.

#2249 In pytorch/ao;

· jiqing-feng opened on May 23, 2025

AO/GemLite tensors produce incorrect outputs in vLLM

integrationIssues related to integrations with other libraries, like huggingface, vllm, sglang, gemlite etc.Issues related to integrations with other libraries, like huggingface, vllm, sglang, gemlite etc.

quantize

triaged

Status: Open.

#2141 In pytorch/ao;

· mobicham opened on Apr 28, 2025

int8 quantization with FSDP for inference error

distributed

quantize

triaged

Status: Open.

#2127 In pytorch/ao;

· Andy0422 opened on Apr 25, 2025

Question about dtype check in marlin_qqq validation for w4a8 functionality

quantize

triaged

Status: Open.

#2115 In pytorch/ao;

· xxw11 opened on Apr 23, 2025

Deprecate/Remove GPTQ.py

quantize

tracker

triaged

Status: Open.

#2089 In pytorch/ao;

· jerryzh168 opened on Apr 21, 2025

fp8 quantization with FSDP2 error

float8

quantize

triaged

Status: Open.

#1929 In pytorch/ao;

· happynear opened on Mar 20, 2025

An error was encountered setting torch._dynamo.decorators.mark_unbacked

quantize

question

triaged

Status: Open.

#1790 In pytorch/ao;

· songh11 opened on Feb 27, 2025

[Question] Static Quantization for Open-Source LLMs

quantize

question

Status: Open.

#1724 In pytorch/ao;

· yang-ahuan opened on Feb 18, 2025

Model size after quantization

quantize

question

Status: Open.

#1701 In pytorch/ao;

· TaylorYangX opened on Feb 11, 2025

[Needs more investigation] <code>int8_weight_only</code> via <code>quantize_()</code> API on <code>torch.float16</code> models results in NaN values across multiple CPU architectures

bugSomething isn't workingSomething isn't working

quantize

Status: Open.

#1662 In pytorch/ao;

· vmpuri opened on Feb 4, 2025

Fail to reproduce benchmark results

benchmark

quantize

Status: Open.

#1135 In pytorch/ao;

· ThisisBillhe opened on Oct 22, 2024

RuntimeError: Error(s) in loading state_dict for QuantizedFasterRCNN:

quantize

Status: Open.

#1123 In pytorch/ao;

· adityapandey369 opened on Oct 20, 2024