RTMP: Use extended timestamp as delta when chunk fmt=1/2. v6.0.167 v7.0.37

[pzx601917159]

1. When the chunk message header employs type 1 and type 2, the extended timestamp denotes the time delta.
2. When the DTS (Decoding Time Stamp) experiences a jump and exceeds 16777215, there can be errors in DTS calculation, and if the audio and video delta differs, it may result in audio-video synchronization issues.

---

TRANS_BY_GPT4

[winlinvip]

1. Please explain more about background and context.
2. Please show how to reproduce this issue.
3. Please add utests to cover it.

[pzx601917159]

1. During the process of switching between primary and backup streams for live streaming, the stream pushed to SRS experienced a discontinuity in DTS (Decoding Time Stamp), leading to errors in DTS calculation. If the format (fmt) is 1, the DTS delta values for audio and video are inconsistent. These values are also being treated as absolute timestamps, which can result in audio-video synchronization issues.

2. As demonstrated in the unit test file, specifically the ProtocolRecvAVExtTimedelta test case, the DTS discontinuity can be replicated by using the CDN's primary and backup stream functionality to push the stream to SRS. This can be achieved by setting the output_ts_offset parameter in ffmpeg, or by using the provided file dts_large_gap.zip. The command to push the stream to a local SRS using ffmpeg version 4.3.2 is: ffmpeg.exe -i .\dts_skip_large.flv -dts_error_threshold 100000000 -c copy -f flv "rtmp:/xxxx:xxxx/live/livestream".

3. The unit test has been added to the srs_protocol_rtmp_stack under the name ProtocolRecvAVExtTimedelta.

TRANS_BY_GPT4

[winlinvip]

From your utest, I understand that the problem is from the vague specification: If chunk fmt=1 and extended timestamp present, is it a delta or value?

For example, if we got too packets here:

    uint8_t video_packet0[] = {
            // 12bytes header, 1byts chunk header, 11bytes msg heder
            0x04, // fmt 0
            0x00, 0x2a, 0xf8, // timestamp 11000
            0x00, 0x00, 0x80, // length, 128
            0x09, // message_type
            0x00, 0x00, 0x00, 0x01, // stream_id
            // msg payload start
            ..
    };

    uint8_t video_packet1[] = {
            // 12bytes header, 1byts chunk header, 11bytes msg heder
            0x44, // fmt 1
            0xff, 0xff, 0xff, // indicates extended timestamp 0xffffff(16777215)
            0x00, 0x00, 0x80, // length, 128
            0x09, // message_type
            0x00, 0xff, 0xfc, 0x18, // extended timestamp delta:16776216
            // msg payload start
            ...
    };

In the video_packet1, the extended timestamp 0xfffc18(16776216) is a delta or absolute value?

• If delta, the timestamp of video_packet1 should be 16776216+11000
• If absolute value, the timestamp of video_packet1 should be 16776216

Unfortunately, SRS followed the 2019 version rtmp.part1.Chunk-Stream.pdf. In fact, RTMP v1 2012 version is quiet clear than 2009:

RTMP v1 2012 version

5.3.1.2.1. Type 0
timestamp (3 bytes): For a type-0 chunk, the absolute timestamp of
 the message is sent here. If the timestamp is greater than or
 equal to 16777215 (hexadecimal 0xFFFFFF), this field MUST be
 16777215, indicating the presence of the Extended Timestamp field
 to encode the full 32 bit timestamp. Otherwise, this field SHOULD
 be the entire timestamp.

5.3.1.2.5. Common Header Fields
timestamp delta (3 bytes): For a type-1 or type-2 chunk, the
 difference between the previous chunk’s timestamp and the current
 chunk’s timestamp is sent here. If the delta is greater than or
 equal to 16777215 (hexadecimal 0xFFFFFF), this field MUST be
 16777215, indicating the presence of the Extended Timestamp field
 to encode the full 32 bit delta. Otherwise, this field SHOULD be
 the actual delta.

I think it makes sense to consider extended timestamp as delta when fmt=1 (type-1) in chunk header.

I have also filed an issue at veovera/enhanced-rtmp#42

TRANS_BY_GPT4