Sample Music

[Nurb4000]

Thought some might be interested in what this stuff can do. I have been posting albums up on YT since last year with 1.0, and started posting more with 1.5 this year.

Disclaimer: I used an external locally run LLM ( mostly cogito ) to generate song structure, caption, lyrics, etc. After hundreds ( yes, literally, 100s ) of generations id pick the best, clean them up a little when needed then group them by genre and post.

The external LLM created the titles, the stable diffusion image prompt, etc. I only steered it with the genre, perhaps clean it slightly, and of course weeding out the garbage. Acestep did all the audio.

The first 6 were done with v1.0, the next batch of 4 was v1.5, all from the 'base' acestep. Will be posting some .cpp creations after i stop futzing with my GUI project.

https://www.youtube.com/@ZoeandGreystones/playlists

and if you get the hidden ( sort of ) reference. 2 points for you. :)

[ServeurpersoCom]

Cool, you already sent me your channel but I didn't see that it was also posted here :) Having examples of use is great!

[ServeurpersoCom]

Non-XL DiT (2B)
Turbo Q8_3362391695.mp3

New XL DiT (4B)
XL Turbo Q4_3362391695.mp3
XL Turbo Q8_3362391695.mp3
XL Turbo BF16_3362391695.mp3

Request :
Turbo Q8.json

As expected, the larger the model, the more interesting it is to quantify it :) We can't hear the difference between Q4_K_M and BF16

[Nurb4000]

Initial post is good :) juste screenshot me a full log please, and try with last master / rebuild :)

Ended up with an unexpected break and it seems i'm a moron or something. I did a new pull ( but it was the latest at the time i spoke up ) and it now works.. I do apologize. i should have tried again before i said anything.

[ServeurpersoCom]

What matters is that it works in the end!!! It worked on my phone too :) I had never tested the XL, because too slow on smartphone, and from a code perspective, based on my experience with past issues, I was 100% certain that it would work on ARM.

[Nurb4000]

What matters is that it works in the end!!! It worked on my phone too :) I had never tested the XL, because too slow on smartphone, and from a code perspective, based on my experience with past issues, I was 100% certain that it would work on ARM.

Ya it made no sense. All i can guess is i was on one revision back, and didn't realize it. I have been deleting and pulling from scratch each time. But my timing must have been off by a few minutes or something.

not that it matters now, but was is the error in context: ( i saved it just before i wiped and pull/compiled )

[Init-Noise Batch0] Philox noise seed=1314755417, [400, 64]
[Init-Noise] Starting: T=400, S=200, enc_S=69, steps=8, batch=1
[DiT] Batch N=1, T=400, S=200, enc_S=69
/home/xxx/src/acestep.cpp/ggml/src/ggml.c:2004: GGML_ASSERT(ggml_can_repeat(b, a)) failed

[ServeurpersoCom]

Computation from my smartphone
Chiptune jap_3128399667.mp3
🤣

[ServeurpersoCom]

What matters is that it works in the end!!! It worked on my phone too :) I had never tested the XL, because too slow on smartphone, and from a code perspective, based on my experience with past issues, I was 100% certain that it would work on ARM.

Ya it made no sense. All i can guess is i was on one revision back, and didn't realize it. I have been deleting and pulling from scratch each time. But my timing must have been off by a few minutes or something.

not that it matters now, but was is the error in context: ( i saved it just before i wiped and pull/compiled )

[Init-Noise Batch0] Philox noise seed=1314755417, [400, 64] [Init-Noise] Starting: T=400, S=200, enc_S=69, steps=8, batch=1 [DiT] Batch N=1, T=400, S=200, enc_S=69 /home/xxx/src/acestep.cpp/ggml/src/ggml.c:2004: GGML_ASSERT(ggml_can_repeat(b, a)) failed

Mystery solved: you were one commit behind 0308fa5. The XL model has asymmetric dimensions: encoder is 2048, decoder is 2560, with a Linear(2048, 2560) bridging them. Before that commit, encoder buffers were allocated at decoder size (2560), so when a 2048-dim encoder tensor hit a ggml_add against a 2560 buffer: ggml_can_repeat(2048, 2560) = false = boom. The 2B model never had this problem because encoder and decoder are both 2048!

[ServeurpersoCom]

You have to try the cover-nofsq mode. It's a "hack" that lets you remix existing (even commercial one) songs, using "cover strength" as a potentiometer for proximity to the original track versus prompt. I listen to my AI-transformed music all day at work with it. Infinite remixing!

[Nurb4000]

Will do that this weekend. Been messing with the 'stock' remix on some 'human music' out of my collection, and it has come up with some interesting stuff too.

[ServeurpersoCom]

I managed to translate English music into French using LLM while keeping the original music exactly or very close to it, to stress test the flow matching, and it works! You can also force the style to drift, it produces incredible results.

[ServeurpersoCom]

Basically, the model far surpasses SUNO in electronic music, for vocals and rock it is very close, the only point where it is less good or does not do it at all, is rapped lyrics which do not sing.

[ServeurpersoCom]

cover-nofsq : Close to the original, but with a prompt to make it diverge a little more to house style, (XL Turbo, Cover strength 0.45)

Daft Punk - One More Time_818673224.mp3

[Nurb4000]

I had a few free minutes today and i agree "cover-nofsq" is cool. Will have to play more when i have real time.

Tho i had one operational question, a bit confused. If load a song, it goes and analyses it, as id expect. But if i leave it at text2music, and change the duration ( either shorter or longer ), it ignores my setting and retains the reference duration when generated. Same if i mark it as reference. I get it would do that for some of them ( like the covers and repaint ) but id not expect that on basic text2music, with or without it being a reference.

even if i delete the original load, it still retains the original length. Of course if i reset, it changes.. im sure is just me misunderstanding the intent.

[ServeurpersoCom]

The "Understand" mode also writes the 5 Hz LM code. You cannot change the duration of the codes; else they must be regenerated (Compose)
Also, the Source audio ("Src") forces the system to produce audio of that duration.

However, the Ref audio ("Ref", Timbre conditioning) does not impose a duration.
Also, you can only input one 5 Hz code source into the Synth part (text-encoder->DiT->VAE), which is why the bottom indicator toggles between the LM codes and the music codes selected as "Ref".

[ServeurpersoCom]

generate:

  Prompt -> Text-Enc --------v
  Tokens 5Hz -> FSQ(detok) --v
                       LoRA -v
                            DiT -> FSQ 25Hz -> VAE(decode) -> Audio

cover:

  Prompt -> Text-Enc --------v
  Tokens 5Hz -> FSQ(detok) --v
  Src Audio -> VAE(encode) --v
  Ref Audio (timbre) --------v
                       LoRA -v
                            DiT -> FSQ 25Hz -> VAE(decode) -> Audio

cover-nofsq:

  Prompt -> Text-Enc --------v
  Src Audio -> VAE(encode) --v
  Ref Audio (timbre) --------v
                       LoRA -v
                            DiT -> FSQ 25Hz -> VAE(decode) -> Audio

understand:

  Audio -> VAE(encode) -> FSQ(tokenize) -> Tokens 5Hz -> LM -> Prompt


Tokens 5Hz = audio codes (LM domain, 5 tokens per second)
FSQ 25Hz   = latents (DiT/VAE domain, 25 frames per second)
Text-Enc   = Qwen3 Embedding (caption + lyrics conditioning)
FSQ(detok) = detokenizer neural network (weights in each DiT GGUF)

[Nurb4000]

oh it makes sense from your first note, in mine i wrote the stuff out when you finally hit generate so you could change on the fly, yours does before that, so it makes sense why it was not really honoring my setting changes. Same idea, just different order :)

[ServeurpersoCom]

Since this is a GitHub discussion, I'll share some of the documentation. I'll redo it completely at the end.

[Nurb4000]

Short delay, been super busy around here but posted up an new set on YT. Using 1.5xl turbo, and the cpp engine. Part of the riotgrrl theme.

playlist - "Wild Heart Riot"

Only difference in setup this time from the last sets, was that i made a few manual changes to lyric structure, my external LLM was failing me on things like solos and stuff.

Still not 100% sold on XL. Going to take some more experimentation.

[Undrsiege]

Vampire (xl-sft complete).mp3 No cover,All prompted

[Nurb4000]

And another just went up. This time 'symphonic rock' genre. All Xl turbo + cpp engine.

[Nurb4000]

tossed up another riotgrrrl, pro environment theme ( was getting ready for earth day :) )

[Nurb4000]

And another just went up this weekend. Back to post-rock. This one, its hard to explain the significance, but back in the 80s i had a band. We had plans. But they never happened. So some 40 years later, here is the ghost of the album we had planned, its damned close to what we were doing. Even the cover is modeled after an old pencil sketch i did. And yes, the band name, is what i called us.

( 1.5cpp + XLturbo )

[Nurb4000]

Not sure if anyone is keeping track, but might be some more soon, been a pause as i have spent the last couple of days adapting my GUI for the semi-recent changes to the cli to move most options to json instead of command line. Still intent to be KISS and not fancy with all the bells and whistles like ServeurpersoCom's webUI, but added a couple of new items to it that i would find useful. ( like better cover mode support, and 'llm analyze' of reference audio tracks. )

[ServeurpersoCom]

I don't know if you've tested the HTTP API, but it's KISS and scalable + LLM friendly! You can also send the post-DiT (or pre-VAE) latents to avoid the lossy of the VAE-encoder/VAE-decoder roundtrip + the losses from any potential MP3 encoding!

[Nurb4000]

I haven't yet, but yes i was thinking about looking at it. Just been lazy ( and sort of busy lately, its that time of the year ) to start something 'new'.

[ServeurpersoCom]

You should do it when you feel like it, it's a hobby :)