Skip to content

Conversation

@seydx
Copy link
Contributor

@seydx seydx commented May 12, 2025

This PR adds comprehensive support for Tuya cameras in go2rtc, with both Tuya Cloud API and Tuya Smart API integration methods. It builds upon the previous PR that introduced initial Tuya integration.

Key Improvements

  • Implemented tuya:// protocol instead of webrtc: for better clarity and separation of concerns
  • Added support for both Tuya Cloud API and Tuya Smart API integration methods:
    • Tuya Cloud API: Requires cloud project setup
    • Tuya Smart API: No cloud project required, cameras added via email/password
  • H265 support
  • Two-way audio support
  • Support for shared cameras (Tuya Smart Api)
  • Resolution selection (resolution=hd|sd)
  • Proper handling of multiple concurrent streams

This implementation has been tested with several Tuya camera models and provides a more robust experience while maintaining all functionality from the original PR.

Usage Examples

streams:
  # Tuya Cloud API: WebRTC main stream
  tuya_webrtc:
   - tuya://openapi.tuyaus.com?device_id=XXX&uid=XXX&client_id=XXX&client_secret=XXX
  
  # Tuya Cloud API: WebRTC sub stream
  tuya_webrtc_sd:
   - tuya://openapi.tuyaus.com?device_id=XXX&uid=XXX&client_id=XXX&client_secret=XXX&resolution=sd

  # Tuya Smart API: WebRTC main stream
  tuya:
    - tuya://protect-us.ismartlife.me?device_id=XXX&email=XXX&password=XXX

  # Tuya Smart API: WebRTC sub stream
  tuya_sd:
    - tuya://protect-us.ismartlife.me?device_id=XXX&email=XXX&password=XXX&resolution=sd

Closes #315 #1379

@seydx seydx mentioned this pull request May 12, 2025
@felipecrs
Copy link
Contributor

felipecrs commented May 12, 2025

I got this error when trying it:

panic: runtime error: index out of range [0] with length 0

goroutine 43 [running]:
github.com/AlexxIT/go2rtc/pkg/tuya.(*TuyaClient).InitDevice(0xc00024c400)
        github.com/AlexxIT/go2rtc/pkg/tuya/api.go:305 +0x885
github.com/AlexxIT/go2rtc/pkg/tuya.NewTuyaClient({0xc0001ea487, 0x12}, {0xc0001ea4a4, 0x16}, {0xc0001ea4bf, 0x14}, {0xc0001ea4de, 0x14}, {0xc0001ea4fa, 0x20}, ...)
        github.com/AlexxIT/go2rtc/pkg/tuya/api.go:169 +0x2bd
github.com/AlexxIT/go2rtc/pkg/tuya.Dial({0xc0001ea480, 0xb3})
        github.com/AlexxIT/go2rtc/pkg/tuya/client.go:51 +0x379
main.main.Init.func5({0xc0001ea480?, 0xc0001c42a0?})
        github.com/AlexxIT/go2rtc/internal/tuya/tuya.go:11 +0x18
github.com/AlexxIT/go2rtc/internal/streams.GetProducer({0xc0001ea480, 0xb3})
        github.com/AlexxIT/go2rtc/internal/streams/handlers.go:49 +0xcc
github.com/AlexxIT/go2rtc/internal/streams.(*Producer).Dial(0xc0000ec630)
        github.com/AlexxIT/go2rtc/internal/streams/producer.go:62 +0x8a
github.com/AlexxIT/go2rtc/internal/streams.(*Stream).AddConsumer(0xc0001d4300, {0xee4508, 0xc00023c120})
        github.com/AlexxIT/go2rtc/internal/streams/add_consumer.go:36 +0x676
github.com/AlexxIT/go2rtc/internal/mp4.handlerWSMSE(0xc0005b8000, 0xc0005ba000)
        github.com/AlexxIT/go2rtc/internal/mp4/ws.go:29 +0x145
github.com/AlexxIT/go2rtc/internal/api/ws.apiWS.func2()
        github.com/AlexxIT/go2rtc/internal/api/ws/ws.go:134 +0x39
created by github.com/AlexxIT/go2rtc/internal/api/ws.apiWS in goroutine 52
        github.com/AlexxIT/go2rtc/internal/api/ws/ws.go:133 +0x357

The config:

streams:
  rua_tuya:
    - tuya://openapi.tuyaus.com?device_id=x&uid=x&client_id=x&secret=x

Maybe it's something related to my camera specifically, but it shouldn't panic anyway.

PS: I got it working for another camera.

@seydx
Copy link
Contributor Author

seydx commented May 12, 2025

I got this error when trying it:

panic: runtime error: index out of range [0] with length 0

goroutine 43 [running]:
github.com/AlexxIT/go2rtc/pkg/tuya.(*TuyaClient).InitDevice(0xc00024c400)
        github.com/AlexxIT/go2rtc/pkg/tuya/api.go:305 +0x885
github.com/AlexxIT/go2rtc/pkg/tuya.NewTuyaClient({0xc0001ea487, 0x12}, {0xc0001ea4a4, 0x16}, {0xc0001ea4bf, 0x14}, {0xc0001ea4de, 0x14}, {0xc0001ea4fa, 0x20}, ...)
        github.com/AlexxIT/go2rtc/pkg/tuya/api.go:169 +0x2bd
github.com/AlexxIT/go2rtc/pkg/tuya.Dial({0xc0001ea480, 0xb3})
        github.com/AlexxIT/go2rtc/pkg/tuya/client.go:51 +0x379
main.main.Init.func5({0xc0001ea480?, 0xc0001c42a0?})
        github.com/AlexxIT/go2rtc/internal/tuya/tuya.go:11 +0x18
github.com/AlexxIT/go2rtc/internal/streams.GetProducer({0xc0001ea480, 0xb3})
        github.com/AlexxIT/go2rtc/internal/streams/handlers.go:49 +0xcc
github.com/AlexxIT/go2rtc/internal/streams.(*Producer).Dial(0xc0000ec630)
        github.com/AlexxIT/go2rtc/internal/streams/producer.go:62 +0x8a
github.com/AlexxIT/go2rtc/internal/streams.(*Stream).AddConsumer(0xc0001d4300, {0xee4508, 0xc00023c120})
        github.com/AlexxIT/go2rtc/internal/streams/add_consumer.go:36 +0x676
github.com/AlexxIT/go2rtc/internal/mp4.handlerWSMSE(0xc0005b8000, 0xc0005ba000)
        github.com/AlexxIT/go2rtc/internal/mp4/ws.go:29 +0x145
github.com/AlexxIT/go2rtc/internal/api/ws.apiWS.func2()
        github.com/AlexxIT/go2rtc/internal/api/ws/ws.go:134 +0x39
created by github.com/AlexxIT/go2rtc/internal/api/ws.apiWS in goroutine 52
        github.com/AlexxIT/go2rtc/internal/api/ws/ws.go:133 +0x357

The config:

streams:
  rua_tuya:
    - tuya://openapi.tuyaus.com?device_id=x&uid=x&client_id=x&secret=x

Maybe it's something related to my camera specifically, but it shouldn't panic anyway.

yeah it shouldn't panic, will take a look

@felipecrs
Copy link
Contributor

Can you please also add an example on how to change the resolution to your PR description?

@seydx
Copy link
Contributor Author

seydx commented May 12, 2025

Can you please also add an example on how to change the resolution to your PR description?

done

@seydx
Copy link
Contributor Author

seydx commented May 12, 2025

@felipecrs

For the camera where go2rtc crashes with a panic, there seems to be no "skill" entry in the response from your camera

https://github.com/seydx/go2rtc/blob/ff00484328d2e2cf3f75dea4616529f52dbf356d/pkg/tuya/api.go#L305

It looks like I need to rewrite this somewhat so that it falls back to default video/audio.

@felipecrs
Copy link
Contributor

Thank you! I guess my camera doesn't support changing the resolution through the WebRTC API because regardless of resolution=0 or resolution=1 I always get the SD stream.


Some other feedback:

  1. Supposing you're using the same API I am using in my script when use_rtsp=1, you should not need the user to pass uid.
  2. Maybe you can simplify use_rtsp=1 into rtsp=1
  3. It would be great if you could also allow use_hls=1 or hls=1. Some cameras only work through the HLS stream. It should be as simple as passing "type": "HLS" instead of "type": "RTSP" when requesting the stream (reference from my script).

@felipecrs
Copy link
Contributor

felipecrs commented May 12, 2025

For the camera where go2rtc crashes with a panic, there seems to be no "skill" entry in the response from your camera

seydx/go2rtc@ff00484/pkg/tuya/api.go#L305

It looks like I need to rewrite this somewhat so that it falls back to default video/audio.

Do you want me to run some Tuya API against my camera? I can do it relatively easy through the Tuya IOT Debug function.

@seydx
Copy link
Contributor Author

seydx commented May 12, 2025

For the camera where go2rtc crashes with a panic, there seems to be no "skill" entry in the response from your camera
seydx/go2rtc@ff00484/pkg/tuya/api.go#L305
It looks like I need to rewrite this somewhat so that it falls back to default video/audio.

Do you want me to run some Tuya API against my camera? I can do it relatively easy through the Tuya IOT Debug function.

Yeah that would be nice, this is the endpoint:

https://github.com/seydx/go2rtc/blob/ff00484328d2e2cf3f75dea4616529f52dbf356d/pkg/tuya/api.go#L258

should look like:

{
    "result":{
        "audio_attributes":{
            "call_mode":[
                1,
                2
            ],
            "hardware_capability":[
                1,
                2
            ]
        },
        "auth":"h85L4pljbuHFR0a/iTgViwA35xi3yTl3NyMsFQL5****",
        "id":"6cf2b6d2b09a2f8597****",
        "moto_id":"moto_cnpre002",
        "p2p_config":{
            "ices":[
                {
                    "urls":"stun:49.234.141.77:3478"
                },
                {
                    "urls":"stun:tx1stun.tuyacn.com:3478"
                },
                {
                    "urls":"nat:tx1nat.tuyacn.com:3478"
                },
                {
                    "urls":"nat:tx2nat.tuyacn.com:3478"
                },
                {
                    "credential":"kb/EA2whGCcNSM5FjXV2dxAM1MU=",
                    "ttl":36000,
                    "urls":"turn:49.234.141.77:3478",
                    "username":"1600883205:6cf2b6d2b09a2f8597****"
                },
                {
                    "credential":"kb/EA2whGCcNSM5FjXV2dxAM****",
                    "ttl":36000,
                    "urls":"turn:tx1turn.tuyacn.com:3478",
                    "username":"1600883205:6cf2b6d2b09a2f8597****"
                }
            ]
        },
        "skill":"{"webrtc":3,"audios":[{"channels":1,"dataBit":16,"codecType":101,"sampleRate":8000}],"videos":[{"streamType":2,"profileId":"","width":1920,"codecType":2,"sampleRate":90000,"height":1080},{"streamType":4,"width":640,"codecType":2,"height":360}]}",
        "supports_webrtc":true,
        "video_clarity":4
    },
    "success":true,
    "t":1600847205437
}

@felipecrs
Copy link
Contributor

This is the output:

{
  "result": {
    "audio_attributes": {
      "call_mode": [
        1
      ],
      "hardware_capability": [
        1,
        2
      ]
    },
    "auth": "xx=",
    "id": "xx",
    "moto_id": "signaling14723",
    "p2p_config": {
      "auth": "xx=",
      "ices": [
        {
          "urls": "stun:44.240.73.31:3478"
        },
        {
          "urls": "stun:[2600:1f14:305f:5a01:3a96:c309:6fd1:4847]:3478"
        },
        {
          "credential": "xx",
          "ttl": 36000,
          "urls": "turn:15.204.9.11:3478",
          "username": "1747054638:eb05a8c8501ca6b9784gfw"
        }
      ],
      "moto_id": "signaling14723"
    },
    "protocol_version": "2.2",
    "skill": "{\"webrtc\":3}",
    "support_webrtc_record": false,
    "supports_webrtc": true,
    "vedio_clarity": 2,
    "vedio_claritys": [
      2,
      4
    ],
    "video_clarity": 2
  },
  "success": true,
  "t": 1747018638601,
  "tid": "d1178ccb2edc11f0b3586285baec33dd"
}

@felipecrs
Copy link
Contributor

felipecrs commented May 12, 2025

Out of curiosity, this is the output for the camera which only works through HLS (at least in my tests):

{
  "result": {
    "audio_attributes": {
      "call_mode": [],
      "hardware_capability": [
        1
      ]
    },
    "auth": "xx",
    "id": "xx",
    "moto_id": "signaling14724",
    "p2p_config": {
      "auth": "xx",
      "ices": [
        {
          "urls": "stun:54.148.238.22:3478"
        },
        {
          "urls": "stun:[2600:1f14:305f:5a01:def3:5ab9:10ea:974e]:3478"
        },
        {
          "credential": "xx",
          "ttl": 36000,
          "urls": "turn:15.204.8.186:3478",
          "username": "1747054814:ebf031220b3de54904xgqs"
        }
      ],
      "moto_id": "signaling14724"
    },
    "protocol_version": "2.2",
    "skill": "{\"webrtc\":2}",
    "support_webrtc_record": false,
    "supports_webrtc": true,
    "vedio_clarity": 4,
    "vedio_claritys": [
      2,
      4
    ],
    "video_clarity": 4
  },
  "success": true,
  "t": 1747018814643,
  "tid": "3a0a92932edd11f0bc8ac69aa343f8ac"
}

PS: It's a different camera, but it also panics in the same place.

@seydx
Copy link
Contributor Author

seydx commented May 12, 2025

@felipecrs

panic should be fixed, can you try pls, changed also the rtsp query to rtsp and added a new hls query hls

@felipecrs
Copy link
Contributor

panic should be fixed, can you try pls, changed also the rtsp query to rtsp and added a new hls query hls

Not only the panic is gone, I can now stream that camera over WebRTC. Thanks!!

@seydx
Copy link
Contributor Author

seydx commented May 12, 2025

panic should be fixed, can you try pls, changed also the rtsp query to rtsp and added a new hls query hls

Not only the panic is gone, I can now stream that camera over WebRTC. Thanks!!

Tested HLS? HLS does not work for my camera, have only black screen. Using ffmpeg as internal producer with disabled audio, because with audio the stream was not playable

@felipecrs
Copy link
Contributor

Wow, the stream for my camera which previously only worked through HLS is now possible with WebRTC too!

@felipecrs
Copy link
Contributor

The HLS mode is kinda weird though:

image

@felipecrs
Copy link
Contributor

Honestly I will probably never need to use HLS anymore, since WebRTC is working. HLS is such a mess.

@seydx
Copy link
Contributor Author

seydx commented May 12, 2025

The HLS mode is kinda weird though:

image

Ffmpeg version? It seems it doesnt support https protocol

can you show me the go2rtc stream config related to the camera pls

@felipecrs
Copy link
Contributor

felipecrs commented May 12, 2025

Ffmpeg version? It seems it doesnt support https protocol

Yeah, that's probably it, I'm using it through pkgx. Probably they built it without HTTPS support. Tested both 5 and 7 though.

@felipecrs
Copy link
Contributor

can you show me the go2rtc stream config related to the camera pls

streams:
  bebe:
    - tuya://openapi.tuyaus.com?device_id=xx&uid=xx&client_id=xx&secret=xx&hls=1

@felipecrs
Copy link
Contributor

because with audio the stream was not playable

Yeah, looks like go2rtc doesn't play well with ADTS audio.

@felipecrs
Copy link
Contributor

If I were you, I'd remove ffmpeg from the HLS stream and just let it be. The problem is that go2rtc cannot play it, and maybe someday it will.

I had the same issue when using my script. Was never able to get the .m3u8 HLS stream working in go2rtc, but in VLC it plays (it just takes some seconds in the black loading screen).

BTW I tried your PR in my Frigate instance, with a proper ffmpeg, and I was able to get the black screen, just like I used to do using my script in HLS mode in go2rtc.

So, again, I think your PR is doing what it's supposed to for HLS mode. The rest is up to go2rtc itself.

@felipecrs
Copy link
Contributor

felipecrs commented May 12, 2025

because with audio the stream was not playable

Yeah, looks like go2rtc doesn't play well with ADTS audio.

Actually I cannot reproduce that. Maybe your camera outputs a different audio codec when in HLS mode than mine. For me the stream plays the same way with or without ffmpeg when using my script.

@felipecrs
Copy link
Contributor

felipecrs commented May 12, 2025

@seydx thinking more about the parameters, I think the following is what makes the most sense:

Instead of rtsp=1 or hls=1, you can have type=rtsp, type=hls, or type=webrtc, where if none is set, type=webrtc is assumed.

@AlexxIT AlexxIT self-assigned this May 12, 2025
@seydx
Copy link
Contributor Author

seydx commented May 12, 2025

@seydx thinking more about the parameters, I think the following is what makes the most sense:

Instead of rtsp=1 or hls=1, you can have type=rtsp, type=hls, or type=webrtc, where if none is set, type=webrtc is assumed.

done

@seydx
Copy link
Contributor Author

seydx commented May 12, 2025

If I were you, I'd remove ffmpeg from the HLS stream and just let it be. The problem is that go2rtc cannot play it, and maybe someday it will.

I had the same issue when using my script. Was never able to get the .m3u8 HLS stream working in go2rtc, but in VLC it plays (it just takes some seconds in the black loading screen).

BTW I tried your PR in my Frigate instance, with a proper ffmpeg, and I was able to get the black screen, just like I used to do using my script in HLS mode in go2rtc.

So, again, I think your PR is doing what it's supposed to for HLS mode. The rest is up to go2rtc itself.

done

@felipecrs
Copy link
Contributor

Not sure if it's worth it, probably not, but you could get the user id using client_id and client_secret through this API, and looking for the first user returned:

https://developer.tuya.com/en/docs/archived-documents/user-management?id=K95ztzvgwnshy#title-9-Get%20the%20user%20list

@felipecrs
Copy link
Contributor

Another recommendation I would give is to name the secret parameter as client_secret.

client_secret makes a better direct relationship to how it is named in Tuya IOT, and also this is how it is called in other interfaces like Home Assistant and Local Tuya:

image

I guess keeping the same terminology across different interfaces helps increase predictability and reduce confusion.

@p0is0n
Copy link

p0is0n commented Nov 16, 2025

Hi, I tried using this PR, but I got the error:

[rtsp] error="streams: tuya: failed to start MQTT: network Error : dial tcp 45.195.222.93:443: i/o timeout" stream=cam_1

Connect via email/password region EU:

tuya://protect-eu.ismartlife.me?device_id=XXX&email=XXX&password=XXX

@AlexxIT
Copy link
Owner

AlexxIT commented Nov 25, 2025

Well. This code don't work for my SmartLife account and for my Tuya account. I still don't know why.

@p0is0n
Copy link

p0is0n commented Nov 25, 2025

Well. This code don't work for my SmartLife account and for my Tuya account. I still don't know why.

I tested this with Tuya. Using the Tuya cloud has been working for about two weeks. Direct login with login/password does not work when the EU region is selected.

@seydx
Copy link
Contributor Author

seydx commented Nov 25, 2025

Well. This code don't work for my SmartLife account and for my Tuya account. I still don't know why.

I tested this with Tuya. Using the Tuya cloud has been working for about two weeks. Direct login with login/password does not work when the EU region is selected.

No issues for me with EU data center

@felipecrs
Copy link
Contributor

felipecrs commented Nov 25, 2025

Maybe some of you can share your Tuya email and password with @seydx privately so he could try to reproduce the issue himself. Note Smart Life accounts are not supported.

@seydx
Copy link
Contributor Author

seydx commented Nov 25, 2025

Hi, I tried using this PR, but I got the error:

[rtsp] error="streams: tuya: failed to start MQTT: network Error : dial tcp 45.195.222.93:443: i/o timeout" stream=cam_1

Connect via email/password region EU:

tuya://protect-eu.ismartlife.me?device_id=XXX&email=XXX&password=XXX

which country?

@p0is0n
Copy link

p0is0n commented Nov 25, 2025

Hi, I tried using this PR, but I got the error:
[rtsp] error="streams: tuya: failed to start MQTT: network Error : dial tcp 45.195.222.93:443: i/o timeout" stream=cam_1
Connect via email/password region EU:

tuya://protect-eu.ismartlife.me?device_id=XXX&email=XXX&password=XXX

which country?

Kazakhstan. Ping to this IP is ok:

PING 45.195.222.93 (45.195.222.93): 56 data bytes
64 bytes from 45.195.222.93: icmp_seq=0 ttl=34 time=53.768 ms
64 bytes from 45.195.222.93: icmp_seq=1 ttl=34 time=56.417 ms

@felipecrs
Copy link
Contributor

Ping works but the 443 port doesn't seem open/accessible:

~ ❯ ping 45.195.222.93
Pinging 45.195.222.93 with 32 bytes of data:
Reply from 45.195.222.93: bytes=32 time=358ms TTL=237

~ ❯ telnet 45.195.222.93 443
Connecting To 45.195.222.93...Could not open connection to the host, on port 443: Connect failed

From Brazil.

@p0is0n
Copy link

p0is0n commented Nov 25, 2025

Ping works but the 443 port doesn't seem open/accessible:

~ ❯ ping 45.195.222.93
Pinging 45.195.222.93 with 32 bytes of data:
Reply from 45.195.222.93: bytes=32 time=358ms TTL=237

~ ❯ telnet 45.195.222.93 443
Connecting To 45.195.222.93...Could not open connection to the host, on port 443: Connect failed

From Brazil.

Same. @seydx there is problem from Tuya? Why I got wrong MQTT server from Tuya?

@AlexxIT
Copy link
Owner

AlexxIT commented Nov 25, 2025

No problem. I've already found a working account and fixed this issue. It'll be in the update.

@felipecrs
Copy link
Contributor

felipecrs commented Nov 25, 2025

I found an interesting project:

Could be useful for adding support for Smart Life accounts, maybe.

@AlexxIT AlexxIT merged commit 6d1a95a into AlexxIT:master Nov 25, 2025
@AlexxIT
Copy link
Owner

AlexxIT commented Nov 25, 2025

Thanks! I think we can finally accept this.

@AlexxIT AlexxIT added this to the master milestone Nov 25, 2025
@felipecrs
Copy link
Contributor

I believe this PR makes the world a better place. 😊

Thanks a lot for reviewing and merging it, @AlexxIT!

And thanks @seydx for spending so much time on this, including reverse engineering the Tuya IPC Client. Much respect!

@AlexxIT
Copy link
Owner

AlexxIT commented Nov 25, 2025

Getting a stream without using a stupid dev account is, of course, a game changer for the Tuya API.

@pergolafabio
Copy link

Much appreciated 👍👍

@p0is0n
Copy link

p0is0n commented Nov 25, 2025

Wow! Thanks for push to master, I can switch to.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Tuya source