How to mute/unmute track?

[Razzwan]

I would like to implement ability to mute and unmute track based on example. Thats how I try to mute the track:

let id_for_remove = 1;
let media_kind_for_remove = MediaKind::Autio;

for client in clients.iter_mut() {
    client.tracks_out.retain(|to| {
        if let Some(track_in) =
            to.track_in.clone().upgrade()
        {
            if track_in.kind == media_kind_for_remove
                && track_in.origin == id_for_remove
            {
                false
            } else {
                true
            }
        } else {
            true
        }
    });
}

And this is good works for Audio, but if I try to do the same thing for the video, the video is corrupted. What should I do to restore the video? Just send PLI to publisher? How to do that correctly? Should I change somehow timestamp/sequence number of rtp packages?

[algesten]

To fully mute, you'd change the direction on a Media/m-line to inactive. One side needs to do it, create the OFFER, and the other side applies the ANSWER.

Javascript side initiates mute

• On the javascript side, set direction property. Follow by SDP negotiation. Get the OFFER
• On str0m side accept OFFER, generate ANSWER
• Apply ANSWER to javascript

str0m side initiates mute

• str0m get the SdpApi and do set_direction() followed by apply to get the OFFER
• javascript apply the OFFER and get the ANSWER
• str0m accept the ANSWER

[Razzwan]

Thank you.

This describes the process of turning off the media stream at the negotiation level, and negotiations sometimes take more than a second. In my case, I need to turn off the sound for individual participants in the room with the precision of a word. In other words, after turning off the sound, the person should not be able to say even a short word. The method of disconnecting a participant from the stream (remove media from tracks_out in example words) has shown the best results in previous tests (in my previous implementation using webrtc rs) and does not require renegotiations. The only thing I need to understand in order to implement it (by the way, if you are interested, I can add this implementation to the library) is:

1. How to request a keyframe from the code?
2. In the previous implementation, in order to avoid defects, it was necessary to correctly overwrite the sequence number and timestamp of RTP packets. Perhaps something similar is also necessary. Although, this already works in the example above. Perhaps, some problems with sequence number and timestamp was already solved within str0m library...

[algesten]

In many hardware, you have an LED that shows your camera is on, which means "muting" the camera should also turn off the LED. If this is the case, you probably want to stop the media on the client side.

On an RTP level there isn't a way to signal "mute". You could stop sending media (or forward RTP packets), but that will force the other side initially send NACK/FIR/PLI, however it will give up in time.

Another option is to have some ersatz media that is "blank"/"silence" that you insert instead. It's easy enough with audio, and more complex with video.

Request keyframe using the writer on the receiver side.

Are you using str0m in RTP mode or not?

[Razzwan]

In many hardware, you have an LED that shows your camera is on, which means "muting" the camera should also turn off the LED. If this is the case, you probably want to stop the media on the client side.

Yes. Thank you. For my tasks, it's not a problem that the LED is on. Because in the game I'm making, someone always sees someone, but not everyone. So, no need to switch off LED.

On an RTP level there isn't a way to signal "mute". You could stop sending media (or forward RTP packets), but that will force the other side initially send NACK/FIR/PLI, however it will give up in time.

Ideally, I should know if we've been disconnected from receiving video, and I should ignore all requests from peers that are disconnected. In other words, if any of the disconnected peers are still requesting PLI, FIR, or NACK, I should simply ignore those requests. At least, that's what I did in my previous implementation. However, NACK requests are not sent if there are no missing sequence numbers, and when we're subsequently reconnected, we should first send a PLI to avoid seeing broken video. That's how I would like to implement this functionality.

Are you using str0m in RTP mode or not?

I still using default settings, because I don't know, how to switch to RTP mode. I mean, it should lead many changes in many places of code. Are you using RTP mode for your SFU server? I would like to use the most flexible approach

Another option is to have some ersatz media that is "blank"/"silence" that you insert instead. It's easy enough with audio, and more complex with video.

This seems like a more complex solution without any advantages compared to simply disabling data sending. What do you think?

Request keyframe using the writer on the receiver side.

Thank you! Thats exactly what I need!

[algesten]

I still using default settings, because I don't know, how to switch to RTP mode. I mean, it should lead many changes in many places of code. Are you using RTP mode for your SFU server? I would like to use the most flexible approach

Hard for me to say which is best. It depends on situation. RTP mode is a lower level of API, so you need to do more: https://github.com/algesten/str0m?tab=readme-ov-file#frame-or-rtp-level We use Frame Level in our SFU because we want to record streams (save to disk on the server).