Support splice in http blind tunnel #11890
Conversation
|
Passes unit test and manual integration test on Debian 12. Performance benchmark shows that splice could improve maximum blind tunnel throughput from 300 MB/s to 575 MB/s and reduce latency by 40% for MB level payload on C6in.large EC2 instance. Feel free to benchmark this CR. |
YIHONG-JIN
force-pushed
the
zero-copy-pr
branch
from
339f013 to
5a1b3ee
Compare
| @@ -7274,6 +7274,32 @@ HttpSM::setup_blind_tunnel(bool send_response_hdr, IOBufferReader *initial) | |||
| // header buffer into new buffer | |||
| client_request_body_bytes += from_ua_buf->write(_ua.get_txn()->get_remote_reader()); | |||
|
|
|||
| #if TS_USE_LINUX_SPLICE | |||
| MIOBuffer *from_ua_pipe_buf = new_PipeIOBuffer(BUFFER_SIZE_INDEX_32K); | |||
There was a problem hiding this comment.
BUFFER_SIZE_INDEX_32K is equal to 8. Is this the capacity of the pipe that we're requesting from the kernel?
There was a problem hiding this comment.
Yes, it is the same as the MIOBuffer size we use for blind tunnel. The default linux pipe size is 16 pages so I am trying to save memory by requesting only 8 pages here. However, the system admin still need to lift pipe-user-pages-soft limit to avoid exceptions.
|
|
||
| if (r <= 0) { | ||
| // Temporary Unavailable, Non-Blocking I/O | ||
| if (r == -EAGAIN || r == -ENOTCONN) { |
There was a problem hiding this comment.
Is it possible that we get EAGAIN here because the pipe is at capacity? How do we handle that case?
There was a problem hiding this comment.
It is actually impossible to get EAGAIN here because the pipe is at capacity. We will disable the read vio after each successful read and only reenable it when its corresponding pipe is empty again.
However, it is possible that we get EAGAIN because the socket is somehow unavailable. In that case, we wait for next epoll edge trigger same as the logic without zero copy
| @@ -508,6 +508,81 @@ UnixNetVConnection::net_read_io(NetHandler *nh) | |||
| read_disable(nh, this); | |||
| return; | |||
| } | |||
| #if TS_USE_LINUX_SPLICE | |||
There was a problem hiding this comment.
This function is getting quite long, with two independent code paths to read from socket to buffer. I would prefer using polymorphism to handle the different socket-to-buffer copies, but at least we should split this function up so that it's more readable.
There was a problem hiding this comment.
I have considered using polymorphism here but it looks impossible without significant refactoring because of exposure of low level io operations in this function (we will need a new member function in both MIOBuffer and PipeIOBuffer). Will split this function up for now
| @@ -656,6 +656,15 @@ new_MIOBuffer_internal(const char *location, int64_t size_index) | |||
| TS_INLINE void | |||
| free_MIOBuffer(MIOBuffer *mio) | |||
| { | |||
| #if TS_USE_LINUX_SPLICE | |||
| // check if mio is PipeIOBuffer using dynamic_cast | |||
| PipeIOBuffer *pipe_mio = dynamic_cast<PipeIOBuffer *>(mio); | |||
There was a problem hiding this comment.
Relying on a runtime type check with dynamic_cast to select the proper deallocation mechanism is generally considered an anti-pattern. Instead, ClassAllocator has a Destruct_on_free_ parameter that allows you to use a virtual destructor to clean up properly depending on the underlying type.
There was a problem hiding this comment.
The PipeIOBuffer is managed by pipeIOAllocator instead of ioAllocator. I introduced pipeIOAllocator because it is a new class with additional attributes. The ClassAllocator appears to be designed for concrete classes rather than abstract classes or interfaces.
- I didn’t put the cleanup logic to the virtual destructor because the existing logic for MIOBuffer doesn’t use MIOBuffer's destructor. Instead, it manually handles cleanup in free_MIOBuffer with the following steps:
mio->_writer = nullptr;
mio->dealloc_all_readers();
Interestingly, the destructor is designed to perform this same cleanup, but for some reason, the existing code avoids relying on it. If anyone knows why, I’d appreciate the insight.
- Even if we moved the cleanup logic to the virtual destructor, we would still need a runtime type check since real polymorphism is not yet in place. For example, when dealing with a PipeIOBuffer, we would need to use
THREAD_FREE(mio, pipeIOAllocator, this_thread());
instead of
THREAD_FREE(mio, ioAllocator, this_thread());
to make sure the correct allocator is used.
There was a problem hiding this comment.
Will keep it as it was until figure out how to implement real polymorphism here. More background info required.
|
Thanks for the comments @moonchen. I will start to resolve the comments |
1. Add TS_USE_LINUX_SPLICE as a compilation option. 2. Make MIOBuffer and MIOBufferReader polymorphic classes making their member function virtual. 3. Create PipeIOBuffer and PipeIOBufferReader as derived classed, encapsulating Linux pipe. 4. Use dynamic_cast to enable logic switch in state machines and continuations.
YIHONG-JIN
force-pushed
the
zero-copy-pr
branch
from
5a1b3ee to
0f7fc70
Compare
|
Slightly reorganized the file structure to decouple PipieIOBuffer from original IOBuffer. |
| TS_INLINE char * | ||
| PipeIOBufferReader::start() | ||
| { | ||
| throw std::runtime_error("Not applicable for PipeIOBufferReader"); |
There was a problem hiding this comment.
Hmm, this PR is pretty interesting as concept. However, overriding bunch of methods and throwing runtime errors like this is indicating something is wrong with class design. ( Also, please note that ATS doesn't use Exception in these fundamental code )
Documentations:
ATS_splice_runbook.md
ATS Performance Benchmark.pdf
ATS_splice_design_doc.pdf