Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to scale tb-node in monolith type #102

Open
xiddjp opened this issue Aug 21, 2023 · 9 comments
Open

How to scale tb-node in monolith type #102

xiddjp opened this issue Aug 21, 2023 · 9 comments

Comments

@xiddjp
Copy link

xiddjp commented Aug 21, 2023

How to scale tb-node in monolith type? When i scale tb-node to 2 or 3 pod, telemetry not update any more.

@dkim-stratuscent
Copy link

listening

@icanos
Copy link

icanos commented Sep 17, 2023

Can't seem to figure this out neither. Any thoughts?

@xiddjp
Copy link
Author

xiddjp commented Sep 18, 2023

I dont't know,it's hard to fix.

@aistisdev
Copy link

How to scale tb-node in monolith type? When i scale tb-node to 2 or 3 pod, telemetry not update any more.

What do you mean by "not update anymore"? In dashboard charts (real time or history?), in device latest values tab or what?

@xiddjp
Copy link
Author

xiddjp commented Oct 9, 2023

How to scale tb-node in monolith type? When i scale tb-node to 2 or 3 pod, telemetry not update any more.

What do you mean by "not update anymore"? In dashboard charts (real time or history?), in device latest values tab or what?
In device details page, Latest telemetry tab. telemetry data will no longer be updated. Reopening this tab will display the telemetry value for the current time again, but it will not continue to be updated

@aistisdev
Copy link

In device details page, Latest telemetry tab. telemetry data will no longer be updated. Reopening this tab will display the telemetry value for the current time again, but it will not continue to be updated

I think you need to use zookeeper+kafka for the scaled pods to communicate with each other (even on monolith mode), so to get real time updates on websocket queries. Otherwise the nodes will have their own queues and rule chain exectution logic, which will not trigger the websocket services on one of the nodes that is not receiving data. Also you might not be able to register new rule chains on both nodes if they have no way to communicate with each other.

@xiddjp
Copy link
Author

xiddjp commented Oct 11, 2023

I use zookeeper and kafka accord thingsboard-ce-k8s/helm folder. but when i scale tb-node up to 2 or 3 node telemetry not update.

@aistisdev
Copy link

aistisdev commented Oct 12, 2023

Do the telemetry values also not update in real time dashboards? Also when you go to systemadmin home page, do you see all your scaled nodes likes this?:
image

@jairohg
Copy link

jairohg commented Mar 26, 2024

I've tested scaling by changing the default value of the environment variable "TB_SERVICE_TYPE" from "monolith" to "tb-core" on the tb-node StatefulSet, then deployed another StatefulSet for the rule engine with TB_SERVICE_TYPE set to "tb-rule-engine". Still haven't configured the liveness probe for the rule engine StatefulSet.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants