How to switch to the Rid, wich was not send by browser yet?

[Razzwan]

I am trying to implement simulcast with 3 Rids: low, mid, max.

The browser, by default, does not send the max Rid stream.

Is there a way to request the max Rid using the str0m API?

If I try to request a keyframe for the max Rid that was not sent by the browser client, like:

writer.request_keyframe(Rid::from("max"), Pli)

It shows the error:

ERROR: NoReceiverSource(Some(Rid(max)))

So, it seems like it must be requested from the client by sending a specific request somehow?

[k0nserv]

Browsers will not always send all layers of a simulcast track. This can happen, for example, because the browser estimates that there isn't enough bandwidth to send the layer or if there isn't enough CPU to encode it. It's also possible that your configuration of sendEncodings is invalid for the max layer.

Look at RTCQualityLimitationReason in chrome://webtc-internals under the outbound rtp streams and it should tell you what's happening.

You can look for the Event::StreamPaused to determine when a layer becomes unavailable. Initially str0m assume all layers are unpaused, but after a while you should see this event

[Razzwan]

As I understood from rfc8853, if the offer and the answer have defined all the rids being sent/received, then we must create a transport layer for each of these rids. If any of the rids is stopped by the sender, then the receiving party can request the sender to start transmitting that rid using an RTCP message (Pli).

So, seems like we rather must create ReceiverSource for all the rids described in offer/answer, and must be able to request sender to start specific rid using Pli RTCP request, than throw NoReceiverSource(Some(Rid(max))) error if it was not send by sender. What do you think?

[k0nserv]

Can you share the partial offer and answer when you set up this track?

I assume the browser offers and str0m answers

As I understood from rfc8853, if the offer and the answer have defined all the rids being sent/received, then we must create a transport layer for each of these rids. If any of the rids is stopped by the sender, then the receiving party can request the sender to start transmitting that rid using an RTCP message (Pli).

The RFC only says the answerer must be prepared to receive the RID.

The answerer MUST be prepared to receive any of the receive-direction rid-id alternatives and MAY send any of the "send"-direction alternatives that are part of the answer.

So, seems like we rather must create ReceiverSource for all the rids described in offer/answer, and must be able to request sender to start specific rid using Pli RTCP request, than throw NoReceiverSource(Some(Rid(max))) error if it was not send by sender. What do you think?

I don't think this is correct. PLIs don't force the sender to start the stream, the sender can still have reason to not do that as I outlined prior. In general senders don't have to do anything in response to a PLI, it's best effort.

In our SFU we wait for the first packet on a given RID before we start using it.

If you watch Chrome's behaviour closely you'll see that is start by sending low and then later, when the bandwidth estimate ramps up, mid, and lastly max. This can take many seconds to happen. My experience is that typically the lowest quality layer starts fairly immediately

[Razzwan]

@k0nserv yes, seems like it should not be available

that's part of sdp:

a=rid:low recv
a=rid:mid recv
a=simulcast:recv low;mid

From a logical point of view, there is nothing stopping us from requesting another Rid later if it was not sent by publisher. In some cases, this may be convenient, because of priority changed, but the specification does not include such a scenario...

[k0nserv]

That's from a str0m answer? It doesn't even mention max. It sounds like you expect to be able to add an entirely new rid later (without negotiation?). If so can you outline how this is supposed to work and the parts of the specs that allow it

[Razzwan]

That's from a str0m answer? It doesn't even mention max. It sounds like you expect to be able to add an entirely new rid later (without negotiation?). If so can you outline how this is supposed to work and the parts of the specs that allow it

Sory, no I defined all the rids at the beginning. It is latest version, where I removed max rid.

[k0nserv]

Sory, no I defined all the rids at the beginning. It is latest version, where I removed max rid.

Okay, in that case I don't think I follow the issue. For str0m to send a PLI it needs to know the SSRC of the stream, but we don't have this until we have discovered the stream. If there are never any packets sent on the max stream then the error you see makes sense.

I'd recommend ignoring that error or not attempting to issue a PLI until you have seen at least some data on that stream. You also need to handle the stream being paused and then unpaused again later.

To answer the original question: You cannot switch to rid before the the browser has started sending it.

For example, you might need 2.5MBit/s of upstream bandwidth to send all 3 layers, but the network can only handle 2Mbit/s. In this scenario the browser will never send max and you cannot force it to

[Razzwan]

I'd recommend ignoring that error or not attempting to issue a PLI until you have seen at least some data on that stream. You also need to handle the stream being paused and then unpaused again later.

Yes, I am ignoring this error for now and subscribing to the pause/resume events. It seems to work fine. Thank you.

But still, if you think about it logically, it would be better to be able to request a keyframe to obtain the max Rid, since SDP contains the SSRC for all Rids.

I mean, it would be great to amend the RFC8853 specification so that the receiver could request the sender to start transmitting a specific Rid using the Pli keyframe. The sender would still not be obligated to send it, but if their bandwidth allows, it could do so. This would make it possible to stop receiving streams that no one is consuming and dynamically request them again as soon as a subscriber appears who needs that specific stream.

[Razzwan]

I've edited the text above to more clearly describe the problem and possible solution (my English isn't perfect).

I think browsers might support this feature (answer to Pli request if rid was described in SDP); at least, I'm willing to risk it and test it. I'll let you know if I succeed.