1.add curlpp::Multi::poll/wait 2.add option:TimeoutMs #173
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2.add option:TimeoutMs 3.add example26,which use wait and TimeoutMs
Mister-Woodpecker
force-pushed
the
add_multi_poll_wait
branch
from
594a009 to
3271b03
Compare
flokaiser
pushed a commit
to OroraTech/minio-cpp
that referenced
this pull request
Jan 5, 2025
There are several issues introduced by minio#29 When running an application that is rarely restarted, that links to this sdk I encountered hangs inside of the select() call inside of http.c multiple times, that required a restart of the application. Looking into the man pages of curl, it is quite obvious, that the usage of select is a bad idea here. On one hand, the requests.fdset() call may return no filedescriptors, if there is currently nothing to wait for. (https://curl.se/libcurl/c/curl_multi_fdset.html) when fdset() returns no filedescriptors, maxfd is set to -1. this leads to a select call, that has nfds set to 0 and a disabled timeout. -> this leads to an infinite select() until either the program is terminated or a signal is received. On the other hand, if your application is running for some time, new sockets may have filedescriptors larger than FD_SETSIZE (1024), according to the docs (https://curl.se/libcurl/c/curl_multi_fdset.html), fdset does not return any filedescriptors in that case, which leads to the infinite select() again. There are two possible solutions to this: 1. waiting for this PR to be merged: jpbarrette/curlpp#173 - Use the poll() or wait() call from there 2. revert to the Easy API again. - If I understand it correctly the only reason for using the Multi API was to be able to abort transfers inside of the ResponseCallback, but that was implemented incorrectly anyway. Currently the request is removed by requests.remove(request) inside of the ResponseCallback, which is forbidden by: 'It is fine to remove a handle at any time during a transfer, just not from within any libcurl callback function.' (https://curl.se/libcurl/c/curl_multi_remove_handle.html) Due to these reasons I propose to revert back to the Easy API. To keep the possibility to abort a running request, CURL_WRITEFUNC_ERROR is returned at the corresponding locations inside of the ResponseCallback() function.
flokaiser
pushed a commit
to OroraTech/minio-cpp
that referenced
this pull request
Jan 5, 2025
There are several issues introduced by minio#29 When running an application that is rarely restarted, that links to this sdk I encountered hangs inside of the select() call inside of http.c multiple times, that required a restart of the application. Looking into the man pages of curl, it is quite obvious, that the usage of select is a bad idea here. On one hand, the requests.fdset() call may return no filedescriptors, if there is currently nothing to wait for. (https://curl.se/libcurl/c/curl_multi_fdset.html) when fdset() returns no filedescriptors, maxfd is set to -1. this leads to a select call, that has nfds set to 0 and a disabled timeout. -> this leads to an infinite select() until either the program is terminated or a signal is received. On the other hand, if your application is running for some time, new sockets may have filedescriptors larger than FD_SETSIZE (1024), according to the docs (https://curl.se/libcurl/c/curl_multi_fdset.html), fdset does not return any filedescriptors in that case, which leads to the infinite select() again. There are two possible solutions to this: 1. waiting for this PR to be merged: jpbarrette/curlpp#173 - Use the poll() or wait() call from there 2. revert to the Easy API again. - If I understand it correctly the only reason for using the Multi API was to be able to abort transfers inside of the ResponseCallback, but that was implemented incorrectly anyway. Currently the request is removed by requests.remove(request) inside of the ResponseCallback, which is forbidden by: 'It is fine to remove a handle at any time during a transfer, just not from within any libcurl callback function.' (https://curl.se/libcurl/c/curl_multi_remove_handle.html) Due to these reasons I propose to revert back to the Easy API. To keep the possibility to abort a running request, CURL_WRITEFUNC_ERROR is returned at the corresponding locations inside of the ResponseCallback() function.
flokaiser
pushed a commit
to OroraTech/minio-cpp
that referenced
this pull request
Jan 5, 2025
There are several issues introduced by minio#29 When running an application that is rarely restarted, that links to this sdk I encountered hangs inside of the select() call inside of http.c multiple times, that required a restart of the application. Looking into the man pages of curl, it is quite obvious, that the usage of select is a bad idea here. On one hand, the requests.fdset() call may return no filedescriptors, if there is currently nothing to wait for. (https://curl.se/libcurl/c/curl_multi_fdset.html) when fdset() returns no filedescriptors, maxfd is set to -1. this leads to a select call, that has nfds set to 0 and a disabled timeout. -> this leads to an infinite select() until either the program is terminated or a signal is received. On the other hand, if your application is running for some time, new sockets may have filedescriptors larger than FD_SETSIZE (1024), according to the docs (https://curl.se/libcurl/c/curl_multi_fdset.html), fdset does not return any filedescriptors in that case, which leads to the infinite select() again. There are two possible solutions to this: 1. waiting for this PR to be merged: jpbarrette/curlpp#173 - Use the poll() or wait() call from there 2. revert to the Easy API again. - If I understand it correctly the only reason for using the Multi API was to be able to abort transfers inside of the ResponseCallback, but that was implemented incorrectly anyway. Currently the request is removed by requests.remove(request) inside of the ResponseCallback, which is forbidden by: 'It is fine to remove a handle at any time during a transfer, just not from within any libcurl callback function.' (https://curl.se/libcurl/c/curl_multi_remove_handle.html) Due to these reasons I propose to revert back to the Easy API. To keep the possibility to abort a running request, CURL_WRITEFUNC_ERROR is returned at the corresponding locations inside of the ResponseCallback() function.
flokaiser
pushed a commit
to OroraTech/minio-cpp
that referenced
this pull request
Jan 5, 2025
There are several issues introduced by minio#29 When running an application that is rarely restarted, that links to this sdk I encountered hangs inside of the select() call inside of http.c multiple times, that required a restart of the application. Looking into the man pages of curl, it is quite obvious, that the usage of select is a bad idea here. On one hand, the requests.fdset() call may return no filedescriptors, if there is currently nothing to wait for. (https://curl.se/libcurl/c/curl_multi_fdset.html) when fdset() returns no filedescriptors, maxfd is set to -1. this leads to a select call, that has nfds set to 0 and a disabled timeout. -> this leads to an infinite select() until either the program is terminated or a signal is received. On the other hand, if your application is running for some time, new sockets may have filedescriptors larger than FD_SETSIZE (1024), according to the docs (https://curl.se/libcurl/c/curl_multi_fdset.html), fdset does not return any filedescriptors in that case, which leads to the infinite select() again. There are two possible solutions to this: 1. waiting for this PR to be merged: jpbarrette/curlpp#173 - Use the poll() or wait() call from there 2. revert to the Easy API again. - If I understand it correctly the only reason for using the Multi API was to be able to abort transfers inside of the ResponseCallback, but that was implemented incorrectly anyway. Currently the request is removed by requests.remove(request) inside of the ResponseCallback, which is forbidden by: 'It is fine to remove a handle at any time during a transfer, just not from within any libcurl callback function.' (https://curl.se/libcurl/c/curl_multi_remove_handle.html) Due to these reasons I propose to revert back to the Easy API. To keep the possibility to abort a running request, CURL_WRITEFUNC_ERROR is returned at the corresponding locations inside of the ResponseCallback() function.
flokaiser
pushed a commit
to OroraTech/minio-cpp
that referenced
this pull request
Jan 5, 2025
There are several issues introduced by minio#29 When running an application that is rarely restarted, that links to this sdk I encountered hangs inside of the select() call inside of http.c multiple times, that required a restart of the application. Looking into the man pages of curl, it is quite obvious, that the usage of select is a bad idea here. On one hand, the requests.fdset() call may return no filedescriptors, if there is currently nothing to wait for. (https://curl.se/libcurl/c/curl_multi_fdset.html) when fdset() returns no filedescriptors, maxfd is set to -1. this leads to a select call, that has nfds set to 0 and a disabled timeout. -> this leads to an infinite select() until either the program is terminated or a signal is received. On the other hand, if your application is running for some time, new sockets may have filedescriptors larger than FD_SETSIZE (1024), according to the docs (https://curl.se/libcurl/c/curl_multi_fdset.html), fdset does not return any filedescriptors in that case, which leads to the infinite select() again. There are two possible solutions to this: 1. waiting for this PR to be merged: jpbarrette/curlpp#173 - Use the poll() or wait() call from there 2. revert to the Easy API again. - If I understand it correctly the only reason for using the Multi API was to be able to abort transfers inside of the ResponseCallback, but that was implemented incorrectly anyway. Currently the request is removed by requests.remove(request) inside of the ResponseCallback, which is forbidden by: 'It is fine to remove a handle at any time during a transfer, just not from within any libcurl callback function.' (https://curl.se/libcurl/c/curl_multi_remove_handle.html) Due to these reasons I propose to revert back to the Easy API. To keep the possibility to abort a running request, CURL_WRITEFUNC_ERROR is returned at the corresponding locations inside of the ResponseCallback() function.
flokaiser
pushed a commit
to OroraTech/minio-cpp
that referenced
this pull request
Jan 5, 2025
There are several issues introduced by minio#29 When running an application that is rarely restarted, that links to this sdk I encountered hangs inside of the select() call inside of http.c multiple times, that required a restart of the application. Looking into the man pages of curl, it is quite obvious, that the usage of select is a bad idea here. On one hand, the requests.fdset() call may return no filedescriptors, if there is currently nothing to wait for. (https://curl.se/libcurl/c/curl_multi_fdset.html) when fdset() returns no filedescriptors, maxfd is set to -1. this leads to a select call, that has nfds set to 0 and a disabled timeout. -> this leads to an infinite select() until either the program is terminated or a signal is received. On the other hand, if your application is running for some time, new sockets may have filedescriptors larger than FD_SETSIZE (1024), according to the docs (https://curl.se/libcurl/c/curl_multi_fdset.html), fdset does not return any filedescriptors in that case, which leads to the infinite select() again. There are two possible solutions to this: 1. waiting for this PR to be merged: jpbarrette/curlpp#173 - Use the poll() or wait() call from there 2. revert to the Easy API again. - If I understand it correctly the only reason for using the Multi API was to be able to abort transfers inside of the ResponseCallback, but that was implemented incorrectly anyway. Currently the request is removed by requests.remove(request) inside of the ResponseCallback, which is forbidden by: 'It is fine to remove a handle at any time during a transfer, just not from within any libcurl callback function.' (https://curl.se/libcurl/c/curl_multi_remove_handle.html) Due to these reasons I propose to revert back to the Easy API. To keep the possibility to abort a running request, CURL_WRITEFUNC_ERROR is returned at the corresponding locations inside of the ResponseCallback() function.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
1.add curlpp::Multi::poll/wait
2.add option:TimeoutMs
3.add example26,which use wait and TimeoutMs