v0.13.0

What's Changed

• Make the geometric median dataset optional by @alan-cooney in #110
• Improve log names by @alan-cooney in #109
• Add replace and zero activations hooks by @alan-cooney in #111
• Add a model reconstruction validation metric by @alan-cooney in #112

Full Changelog: v0.12.0...v0.13.0

v0.12.0

What's Changed

• Typo in adam_with_reset.py by @HoagyC in #99
• Add resample metrics by @alan-cooney in #98
• Fix numerical stability issues with linear encoder by @alan-cooney in #100
• Fix maths details in contributing guide by @alan-cooney in #101
• Fix store size calc by @alan-cooney in #106
• Move other train metrics from pipeline to implement abstract metric c… by @alan-cooney in #107
• Replicate Neel's GeLU work by @alan-cooney in #108

Full Changelog: v0.11.0...v0.12.0

v0.11.0

What's Changed

• Improve the docs by @alan-cooney in #95
• Add file checkpointing by @alan-cooney in #96
• Fix all docstrings for docs by @alan-cooney in #97

Full Changelog: v0.10.0...v0.11.0

v0.10.0

What's Changed

• Rename the source model directory by @alan-cooney in #94

Full Changelog: v0.9.0...v0.10.0

v0.9.0

What's Changed

• Prevent num_resample_inputs > len(activation_store) by @HoagyC in #69
• no auto formatting in notebooks by @jbloomAus in #70
• Dataset classes by @jbloomAus in #71
• improve demo by @jbloomAus in #72
• Move towards abstract final pattern by @alan-cooney in #74
• Docs generation by @bryce13950 in #60
• Docs deploy by @bryce13950 in #81
• removed deploy action by @bryce13950 in #82
• added generation again by @bryce13950 in #83
• Improve optimizer documentation by @alan-cooney in #79
• Add abstract model class by @alan-cooney in #80
• Cover notebooks with checks by @alan-cooney in #84
• Move pipeline to simpler OO approach by @alan-cooney in #85
• Make the CD consistent with the poe commands by @alan-cooney in #93

New Contributors

• @bryce13950 made their first contribution in #60

Full Changelog: v0.8.0...v0.9.0

v0.8.0

What's Changed

• Improve performance & logging by @alan-cooney in #62
• Add neuron resampling by @alan-cooney in #56
• Switch loss to mean by @alan-cooney in #67
• Simplify logging by @alan-cooney in #65

Full Changelog: v0.7.0...v0.8.0

v0.7.0

What's Changed

• Simplify poe commands by @alan-cooney in #61
• Implement Capacity Metric and W&B Histogram Logging by @HoagyC in #48

Full Changelog: v0.6.0...v0.7.0

v0.6.0

What's Changed

• L1 averaged over batch by @ArthurConmy in #28
• Add names to model children modules by @alan-cooney in #57
• Move loss averaging to log step by @alan-cooney in #59
• Improve sweep defaults by @alan-cooney in #58

Full Changelog: v0.5.0...v0.6.0

v0.5.0

Add stateful iterator to the pipeline (#55)

v0.4.0

What's Changed

• Add note about local venv to the readme by @alan-cooney in #27
• remove old import by @jbloomAus in #21
• Add abstract dataset by @alan-cooney in #35
• Add Pile Uncopyrighted source data by @alan-cooney in #36
• Set all dataclasses to frozen by @alan-cooney in #37
• Add Neel's C4 Dataset by @alan-cooney in #38
• Remove old src data functionality by @alan-cooney in #39
• Add get_dataloader method to source dataset by @alan-cooney in #41
• Add mixed precision manual settings throughout SAE by @alan-cooney in #42
• Make the pile tests more robust by @alan-cooney in #43
• Restrict Python version to <3.12 due to aiohttp compatibility by @HoagyC in #40
• Pin old transformers and tokenizers versions to fix bug with Neel's tokenizer by @ArthurConmy in #24
• Add alias poe commands by @alan-cooney in #44
• Add pre commit hooks quick setup by @alan-cooney in #46
• Add batch size option to config by @ArthurConmy in #25
• Fix demo pipeline by @alan-cooney in #50

New Contributors

• @jbloomAus made their first contribution in #21
• @HoagyC made their first contribution in #40
• @ArthurConmy made their first contribution in #24

Full Changelog: v0.3.0...v0.4.0