DeepSeek V4 Flash FP4 Weights

[groknt]

I am inquiring about whether Heretic, in its current state, is usable, or even effective, against the DeepSeek V4 Flash model.
The first thing that caught me off guard was that the file size is relatively small compared to the parameter size - unlike other models.

There's not much material online, but discovering this: https://outcomeschool.com/blog/decoding-deepseek-v4
It seems DeepSeek used 'FP4 Quantization-Aware Training', name seems to imply the model was trained with FP4 as the base.

This is quite alarming because I am unsure how Heretic would work. Would it even work? If it does, would it be less precise than usual?
Forgive me since I don't know the intricacies regarding this, especially since I've never seen a model like this.

[coder3101]

DSv4 uses Manifold-Constrained Hyper-Connections (mHC) which breaks abliteration

[groknt]

Right, I was focusing on one aspect, but mHC may be even more of a blocker. At least I know there are seemingly one or more blockers now, before committing to compute and attempting heretic on the DeepSeek models.

[rootacc3ss]

Curious if the newer additions and experimental methodologies -- partially added due to similar roadblock for Kimi, see #221 and #211 -- would help any with DSv4 at all. Or if we can manage to make progress "unbreaking" abliteration for mhc / contributing to some form of fix, at least

[baysonfox]

Came across with https://huggingface.co/apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8, which leads to https://github.com/drowzeys/DeepSeek-V4-Flash-DSpark-Abliterated-Uncensored-1M-57toks.

Could be useful for a potential solution i guess……?

[wtdcode]

I noticed https://huggingface.co/squanchyzx/DeepSeek-V4-Flash-0731-HERETIC-Abliterated-FP8

Seems a promising attempt?

[accemlcc]

DSv4 uses Manifold-Constrained Hyper-Connections (mHC) which breaks abliteration

Thanks for the link, very interesting. It would indeed be an interesting case to solve, but unfortunately its size makes it rather unwieldy for local experiments—or impossible, at least for me. And experiments via clouds would likely become very expensive very quickly.

By the way, I’ve seen heavily quantized (Q2) models that can be loaded on an AI studio or AMD Strix Halo and are supposedly 'uncensored.' I suspect the heavy quantization simply makes them uncensored. Of course, it’s reasonable to assume that it’s not just the censorship that has suffered.