-
Notifications
You must be signed in to change notification settings - Fork 382
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sockets opened outside profiling do not close properly inside profiling #342
Comments
Mmm... new commands that work
new working terminal session$ cat > x.py << EOF
> import multiprocessing
> import socket
>
> import memory_profiler
>
>
> address = ("127.0.0.1", 33125)
>
>
> def main():
> multiprocessing.set_start_method("spawn")
>
> sock = socket.socket()
> sock.bind(address)
>
> profiled(sock)
>
> sock3 = socket.socket()
> sock3.bind(address)
>
> memory_profiler.memory_usage((profiled, [sock3]))
> print("really made it")
>
>
> def profiled(sock):
> sock.close()
>
> sock2 = socket.socket()
> sock2.bind(address)
> sock2.close()
>
>
> # auaughhghgh multiprocessing!
> if __name__ == "__main__":
> main()
> EOF $ cat x.py
import multiprocessing
import socket
import memory_profiler
address = ("127.0.0.1", 33125)
def main():
multiprocessing.set_start_method("spawn")
sock = socket.socket()
sock.bind(address)
profiled(sock)
sock3 = socket.socket()
sock3.bind(address)
memory_profiler.memory_usage((profiled, [sock3]))
print("really made it")
def profiled(sock):
sock.close()
sock2 = socket.socket()
sock2.bind(address)
sock2.close()
# auaughhghgh multiprocessing!
if __name__ == "__main__":
main() $ python3.9 -m venv venv $ venv/bin/python -m pip install --upgrade pip setuptools wheel
Requirement already satisfied: pip in ./venv/lib/python3.9/site-packages (21.1.1)
Collecting pip
Using cached pip-21.3.1-py3-none-any.whl (1.7 MB)
Requirement already satisfied: setuptools in ./venv/lib/python3.9/site-packages (56.0.0)
Collecting setuptools
Using cached setuptools-60.1.0-py3-none-any.whl (952 kB)
Collecting wheel
Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel, setuptools, pip
Attempting uninstall: setuptools
Found existing installation: setuptools 56.0.0
Uninstalling setuptools-56.0.0:
Successfully uninstalled setuptools-56.0.0
Attempting uninstall: pip
Found existing installation: pip 21.1.1
Uninstalling pip-21.1.1:
Successfully uninstalled pip-21.1.1
Successfully installed pip-21.3.1 setuptools-60.1.0 wheel-0.37.1 $ venv/bin/pip install memory-profiler==0.60.0 psutil==5.8.0
Collecting memory-profiler==0.60.0
Using cached memory_profiler-0.60.0-py3-none-any.whl
Collecting psutil==5.8.0
Using cached psutil-5.8.0-cp39-cp39-manylinux2010_x86_64.whl (293 kB)
Installing collected packages: psutil, memory-profiler
Successfully installed memory-profiler-0.60.0 psutil-5.8.0 $ venv/bin/python x.py
really made it $ venv/bin/python --version --version
Python 3.9.5 (default, Jun 3 2021, 15:18:23)
[GCC 9.3.0] $ venv/bin/pip freeze
memory-profiler==0.60.0
psutil==5.8.0 $ uname -a
Linux p1 5.4.0-91-generic #102-Ubuntu SMP Fri Nov 5 16:31:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux $ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal Ok, so with the cause identified, what next? I'm not exactly sure. Probably some documentation commenting on this with references to each of multiprocessing, spawn, fork, inherit-ed/ing, file descriptors, etc so it is findable by searching. I hesitate to mess with the multiprocessing global state and auto-configure this since code using memory-profiler may choose something else. If we do set the start method then it should only happen if not otherwise configured by the 'outer' code. I'll think it over and see what I can come up with. |
Are |
Creating and binding a socket outside of a call to
memory_profiler.memory_usage()
causes it to not close properly inside the call. After calling.close()
the port is still claimed and a new socket is unable to be bound to the same port. Withoutmemory_profiler
in the middle it works. I haven't gotten to digging into this yet but will share anything I find.Expand the collapsed sections at the bottom for a full set of commands to recreate and a full terminal session.
For linkage, I initially reported this at CFMTech/pytest-monitor#53.
commands
full console session
$ python3.9 -m venv venv
The text was updated successfully, but these errors were encountered: