MOE models and just active experts in RAM ( or VRAM)

[ZorXL]

In llamacpp is it possible to do following?

The complete model, including all experts, is stored on a drive (SSD or HDD) out of RAM. . This includes both the parameters of the experts and the gating network. Ideally, just gatting network is initially loaded into RAM and locked there by mlock or similar.

Activation Process: When an input is received, the gating network evaluates it and determines which experts should be activated based on the input's characteristics.

Loading Active Experts: Only the parameters of the selected experts are loaded into RAM for processing.

For the next query gatting network again checks what experts will be activated. If it happens they are different then those already stored in in RAM from previous use and activation, then they are just replaced.with newly activated for processing..