Support show bytes sent/received on the connection

[lance6716]

Due to some billing requirements, I want to know how many bytes this client sent/received with TiKV. Currrently I want to wrap a net.Conn with customized Read Write method to add the IO byte number to an atomic variable, like

https://github.com/pingcap/tiflow/blob/7f27730235f94859692b9da561f50c491e483427/dm/syncer/dbconn/utils.go#L102-L127

Is this OK? can you give some advice

[lance6716]

@Defined2014 @xiongjiwei ptal

[Defined2014]

Is this added to every TCP connection? Will it be too heavy to add atomic variables here which have a big impact on performance?

[lance6716]

Is this added to every TCP connection? Will it be too heavy to add atomic variables here which have a big impact on performance?

It should be an option like WithDialer or something, but when it enables, all connections to the endpoint should use the atomic counter.

And I think the performance is not a big problem, because IO duration is much higher than atomic operations

[sticnarf]

It sounds okay but I'm not sure whether it is a better idea to count at upper layers for more flexibility. If one client could support multiple tenants, counting bytes by connection may be not enough.

[Defined2014]

It sounds okay but I'm not sure whether it is a better idea to count at upper layers for more flexibility. If one client could support multiple tenants, counting bytes by connection may be not enough.

Agree. Do you have a spec about this requirement? I am not fully understand about this feature.

According to my understanding, a TiDB-Server may only have one tenant, do we need to count traffic by TCP connection, maybe by instance is enough (not sure)?.

[lance6716]

As my plan, it's used by lightning to import data to tikv. We want to know the network IO of this import task

[sticnarf]

Is precision so important that we must wrap the connection? The protobuf bytes or the bytes you just want to import (oh it will) don't seem to differ a lot from the real IO bytes 🤔

[disksing]

Is precision so important that we must wrap the connection? The protobuf bytes or the bytes you just want to import (oh it will) don't seem to differ a lot from the real IO bytes 🤔

good point. I'm thinking of another example: if there are problems with environment and lead to network reconnection or data retransmission, we should not charge users for it.