Search results

Add sliding windowed attention and different block sizes per layer.

Status: Open.

#434 In ReaLLMASIC/nanoGPT;

· gkielian opened on Apr 22, 2025

Option to monitor specific learned parameters during training

Status: Open.

#429 In ReaLLMASIC/nanoGPT;

· gkielian opened on Apr 12, 2025

Bugs when running on Multiple A100

Status: Open.

#426 In ReaLLMASIC/nanoGPT;

· Mars-Cat2023 opened on Apr 7, 2025

Ssm has following error when running locally on tip of tree

Status: Open.

#412 In ReaLLMASIC/nanoGPT;

· klei22 opened on Mar 4, 2025

Add entropy colorization of output

Status: Open.

#407 In ReaLLMASIC/nanoGPT;

· gkielian opened on Feb 26, 2025

Add Softcapping implementation

Status: Open.

#397 In ReaLLMASIC/nanoGPT;

· gkielian opened on Feb 13, 2025

Add visualizations for attention

Status: Open.

#389 In ReaLLMASIC/nanoGPT;

· gkielian opened on Feb 6, 2025

Implement Early Exit

Status: Open.

#388 In ReaLLMASIC/nanoGPT;

· gkielian opened on Feb 5, 2025

Add Delta Attention variation

Status: Open.

#376 In ReaLLMASIC/nanoGPT;

· gkielian opened on Jan 27, 2025

Add support for Maximal Update of Parameters - MUP

Status: Open.

#372 In ReaLLMASIC/nanoGPT;

· gkielian opened on Jan 23, 2025

Learned KV Cache Prefix

Status: Open.

#369 In ReaLLMASIC/nanoGPT;

· klei22 opened on Jan 21, 2025

Add CJK IPA Tokenizer

Status: Open.

#367 In ReaLLMASIC/nanoGPT;

· gkielian opened on Jan 19, 2025