Search results

MXFP Inference Tracking Doc

mx

performance

topic: performanceUse this tag if this PR improves the performance of a featureUse this tag if this PR improves the performance of a feature

tracker

Status: Open.

#2229 In pytorch/ao;

· drisspg opened on May 21, 2025

Add MXFP casting kernels from triton Repro

mx

performance

Status: Open.

#2217 In pytorch/ao;

· drisspg opened on May 16, 2025

Got unexpected low speed using quantization inference on qwen models.

performance

triaged

Status: Open.

#2102 In pytorch/ao;

· HaoKang-Timmy opened on Apr 22, 2025

What kind of layers are optimized by torchao on a RTX 4090?

performance

question

triaged

Status: Open.

#1805 In pytorch/ao;

· naiveen opened on Mar 1, 2025

Performance comparison <code>NF4Tensor</code> vs. BNB <code>Params4bit</code>

performance

Status: Open.

#1686 In pytorch/ao;

· psinger opened on Feb 10, 2025

<code>int4_weight_only</code> Slows Down <code>torch.nn.Linear</code> for Llama2 7B Shapes

high priority

performance

triaged

Status: Open.

#1606 In pytorch/ao;

· mostafaelhoushi opened on Jan 23, 2025

QLoRA Worse Memory When linear_nf4 Used on Output

performance

Status: Open.

#1433 In pytorch/ao;

· pbontrager opened on Dec 18, 2024

Why is the inference speed of the quantized model using QAT so slow?

performance

qat

Status: Open.

#1050 In pytorch/ao;

· elfisworking opened on Oct 10, 2024