Skip to content

Dynamic cache resize #156

Open
Open
@Conchylicultor

Description

@Conchylicultor

In the ChatSampler, it would be good if users could resize the cache, like:

sampler = gm.text.ChatSampler(cache_length=1024)
sampler.resize_cache(2048)

On Colab when reaching the full cache length, it would allow to still run more prompts without having to restart from scratch

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions