Install script failing on MacOS

[moth754]

Error below triggered when running python3 crossmgr-install.py install. Was more extensive but updating certificates cleared those.

`File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", line 599, in _read_chunked
value.append(self._safe_read(amt))
~~~~~~~~~~~~~~~^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", line 644, in _safe_read
raise IncompleteRead(data, amt-len(data))
http.client.IncompleteRead: IncompleteRead(5819 bytes read, 2373 more expected)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/timmockford/Downloads/crossmgr-install.py", line 761, in
dispatchargs.sub_command
~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
File "/Users/timmockford/Downloads/crossmgr-install.py", line 755, in
'install': lambda args: install( args.full ),
~~~~~~~^^^^^^^^^^^^^
File "/Users/timmockford/Downloads/crossmgr-install.py", line 641, in install
src_download()
~~~~~~~~~~~~^^
File "/Users/timmockford/Downloads/crossmgr-install.py", line 115, in src_download
urllib.request.urlretrieve( zip_file_url, filename=zip_file_name )
~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/urllib/request.py", line 242, in urlretrieve
while block := fp.read(bs):
~~~~~~~^^^^
File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", line 473, in read
return self._read_chunked(amt)
~~~~~~~~~~~~~~~~~~^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", line 609, in _read_chunked
raise IncompleteRead(b''.join(value)) from exc
http.client.IncompleteRead: IncompleteRead(0 bytes read)
`

[moth754]

Same error occurring in Windows 11

[moth754]

Similar but not exact error on Linux (popOS) - incompleteRead now saying 943 bytes read, 7249 more than expected

[esitarski]

Update. The stack trace says that urllib.request.urlretrieve is throwing an IncompleteRead(0 bytes read) exception. This code works perfectly on my linux machine (see below). After searching the internet, there seem to be a number of possible solutions: 1. Use "requests" module instead of python's built in urllib. Apparently, "requests: handles this more robustly. The downside is that it has to be installed as it isn't part of standard python. There are some examples of automatically installing missing modules, so this might be worth a try. 2. It's caused by a mismatch that urllib uses http protocol 1.1, and the server is using 1.0. The fix here is to change: http.client.HTTPConnection._http_vsn = 10 http.client/.HTTPConnection._http_vsn_str = 'HTTP/1.0' This seems unlikely as the code works in Linux, and I doubt that github is using the 1.0 http protocol. 3. IncompleteRead caused by an unstable internet connection. To handle this, the urllib code needs to be modified to keep trying and to ask for the missing part of the file. See here <https://stackoverflow.com/questions/14442222/how-to-handle-incompleteread-in-python>. It is suspected that this is what the requests module does. 4. It's a virus detector/firewall interfering on MacOS and Windows. My preference would be to try to use "requests" as it is a proven module. Does anyone have any other ideas? [image: image.png]

…

On Tue, Mar 4, 2025 at 12:30 PM Tim ***@***.***> wrote: Error below triggered when running python3 crossmgr-install.py install. Was more extensive but updating certificates cleared those. `File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", line 599, in _read_chunked value.append(self._safe_read(amt)) ~~~~~~~~~~~~~~~^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", line 644, in _safe_read raise IncompleteRead(data, amt-len(data)) http.client.IncompleteRead: IncompleteRead(5819 bytes read, 2373 more expected) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/timmockford/Downloads/crossmgr-install.py", line 761, in dispatchargs.sub_command <http://args> ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ File "/Users/timmockford/Downloads/crossmgr-install.py", line 755, in 'install': lambda args: install( args.full ), ~~~~~~~^^^^^^^^^^^^^ File "/Users/timmockford/Downloads/crossmgr-install.py", line 641, in install src_download() ~~~~~~~~~~~~^^ File "/Users/timmockford/Downloads/crossmgr-install.py", line 115, in src_download urllib.request.urlretrieve( zip_file_url, filename=zip_file_name ) ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/urllib/request.py", line 242, in urlretrieve while block := fp.read(bs): ~~~~~~~^^^^ File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", line 473, in read return self._read_chunked(amt) ~~~~~~~~~~~~~~~~~~^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", line 609, in _read_chunked raise IncompleteRead(b''.join(value)) from exc http.client.IncompleteRead: IncompleteRead(0 bytes read) ` — Reply to this email directly, view it on GitHub <#176>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABGXKLJ7P5W652KQ2FOW6L2SXPJVAVCNFSM6AAAAABYKDDGZ6VHI2DSMVQWIX3LMV43ASLTON2WKOZSHA4TIOJXGQ4DGMA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***> [image: moth754]*moth754* created an issue (esitarski/CrossMgr#176) <#176> Error below triggered when running python3 crossmgr-install.py install. Was more extensive but updating certificates cleared those. `File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", line 599, in _read_chunked value.append(self._safe_read(amt)) ~~~~~~~~~~~~~~~^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", line 644, in _safe_read raise IncompleteRead(data, amt-len(data)) http.client.IncompleteRead: IncompleteRead(5819 bytes read, 2373 more expected) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/timmockford/Downloads/crossmgr-install.py", line 761, in dispatchargs.sub_command <http://args> ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ File "/Users/timmockford/Downloads/crossmgr-install.py", line 755, in 'install': lambda args: install( args.full ), ~~~~~~~^^^^^^^^^^^^^ File "/Users/timmockford/Downloads/crossmgr-install.py", line 641, in install src_download() ~~~~~~~~~~~~^^ File "/Users/timmockford/Downloads/crossmgr-install.py", line 115, in src_download urllib.request.urlretrieve( zip_file_url, filename=zip_file_name ) ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/urllib/request.py", line 242, in urlretrieve while block := fp.read(bs): ~~~~~~~^^^^ File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", line 473, in read return self._read_chunked(amt) ~~~~~~~~~~~~~~~~~~^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", line 609, in _read_chunked raise IncompleteRead(b''.join(value)) from exc http.client.IncompleteRead: IncompleteRead(0 bytes read) ` — Reply to this email directly, view it on GitHub <#176>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABGXKLJ7P5W652KQ2FOW6L2SXPJVAVCNFSM6AAAAABYKDDGZ6VHI2DSMVQWIX3LMV43ASLTON2WKOZSHA4TIOJXGQ4DGMA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- Edward Sitarski

[esitarski]

Update. Switched to the "requests" module, which is automatically installed if missing. On Wed, Mar 5, 2025 at 11:17 AM Edward Sitarski ***@***.***> wrote:

…

Update. The stack trace says that urllib.request.urlretrieve is throwing an IncompleteRead(0 bytes read) exception. This code works perfectly on my linux machine (see below). After searching the internet, there seem to be a number of possible solutions: 1. Use "requests" module instead of python's built in urllib. Apparently, "requests: handles this more robustly. The downside is that it has to be installed as it isn't part of standard python. There are some examples of automatically installing missing modules, so this might be worth a try. 2. It's caused by a mismatch that urllib uses http protocol 1.1, and the server is using 1.0. The fix here is to change: http.client.HTTPConnection._http_vsn = 10 http.client/.HTTPConnection._http_vsn_str = 'HTTP/1.0' This seems unlikely as the code works in Linux, and I doubt that github is using the 1.0 http protocol. 3. IncompleteRead caused by an unstable internet connection. To handle this, the urllib code needs to be modified to keep trying and to ask for the missing part of the file. See here <https://stackoverflow.com/questions/14442222/how-to-handle-incompleteread-in-python>. It is suspected that this is what the requests module does. 4. It's a virus detector/firewall interfering on MacOS and Windows. My preference would be to try to use "requests" as it is a proven module. Does anyone have any other ideas? [image: image.png] On Tue, Mar 4, 2025 at 12:30 PM Tim ***@***.***> wrote: > Error below triggered when running python3 crossmgr-install.py install. > Was more extensive but updating certificates cleared those. > > `File > "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", > line 599, in _read_chunked > value.append(self._safe_read(amt)) > ~~~~~~~~~~~~~~~^^^^^ > File > "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", > line 644, in _safe_read > raise IncompleteRead(data, amt-len(data)) > http.client.IncompleteRead: IncompleteRead(5819 bytes read, 2373 more > expected) > > The above exception was the direct cause of the following exception: > > Traceback (most recent call last): > File "/Users/timmockford/Downloads/crossmgr-install.py", line 761, in > dispatchargs.sub_command <http://args> > ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ > File "/Users/timmockford/Downloads/crossmgr-install.py", line 755, in > 'install': lambda args: install( args.full ), > ~~~~~~~^^^^^^^^^^^^^ > File "/Users/timmockford/Downloads/crossmgr-install.py", line 641, in > install > src_download() > ~~~~~~~~~~~~^^ > File "/Users/timmockford/Downloads/crossmgr-install.py", line 115, in > src_download > urllib.request.urlretrieve( zip_file_url, filename=zip_file_name ) > ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/urllib/request.py", > line 242, in urlretrieve > while block := fp.read(bs): > ~~~~~~~^^^^ > File > "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", > line 473, in read > return self._read_chunked(amt) > ~~~~~~~~~~~~~~~~~~^^^^^ > File > "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", > line 609, in _read_chunked > raise IncompleteRead(b''.join(value)) from exc > http.client.IncompleteRead: IncompleteRead(0 bytes read) > ` > > — > Reply to this email directly, view it on GitHub > <#176>, or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AABGXKLJ7P5W652KQ2FOW6L2SXPJVAVCNFSM6AAAAABYKDDGZ6VHI2DSMVQWIX3LMV43ASLTON2WKOZSHA4TIOJXGQ4DGMA> > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> > [image: moth754]*moth754* created an issue (esitarski/CrossMgr#176) > <#176> > > Error below triggered when running python3 crossmgr-install.py install. > Was more extensive but updating certificates cleared those. > > `File > "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", > line 599, in _read_chunked > value.append(self._safe_read(amt)) > ~~~~~~~~~~~~~~~^^^^^ > File > "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", > line 644, in _safe_read > raise IncompleteRead(data, amt-len(data)) > http.client.IncompleteRead: IncompleteRead(5819 bytes read, 2373 more > expected) > > The above exception was the direct cause of the following exception: > > Traceback (most recent call last): > File "/Users/timmockford/Downloads/crossmgr-install.py", line 761, in > dispatchargs.sub_command <http://args> > ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ > File "/Users/timmockford/Downloads/crossmgr-install.py", line 755, in > 'install': lambda args: install( args.full ), > ~~~~~~~^^^^^^^^^^^^^ > File "/Users/timmockford/Downloads/crossmgr-install.py", line 641, in > install > src_download() > ~~~~~~~~~~~~^^ > File "/Users/timmockford/Downloads/crossmgr-install.py", line 115, in > src_download > urllib.request.urlretrieve( zip_file_url, filename=zip_file_name ) > ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File > "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/urllib/request.py", > line 242, in urlretrieve > while block := fp.read(bs): > ~~~~~~~^^^^ > File > "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", > line 473, in read > return self._read_chunked(amt) > ~~~~~~~~~~~~~~~~~~^^^^^ > File > "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/http/client.py", > line 609, in _read_chunked > raise IncompleteRead(b''.join(value)) from exc > http.client.IncompleteRead: IncompleteRead(0 bytes read) > ` > > — > Reply to this email directly, view it on GitHub > <#176>, or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AABGXKLJ7P5W652KQ2FOW6L2SXPJVAVCNFSM6AAAAABYKDDGZ6VHI2DSMVQWIX3LMV43ASLTON2WKOZSHA4TIOJXGQ4DGMA> > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> > -- Edward Sitarski

-- Edward Sitarski

[moth754]

When the new file is uploaded I'll test

[moth754]

testing on mac results in installation - can't run the programs but I remember reading about something special required with mac, so will have a look around.

On linux the output is below. Have verified pip is installed. My knowledge around python environments is limited.

Archiving current version to: /home/tim/CrossMgr-archive/20250305T174016-145785... Done.
Downloading CrossMgr source to: /home/tim...
Extracting CrossMgr source to: /home/tim/CrossMgr-master... Done.
Using existing python environment /home/tim/CrossMgr-env.
Updating python environment (may take a few minutes, especially on first install)... /home/tim/CrossMgr-env/bin/python3: No module named pip
Traceback (most recent call last):
File "/home/tim/Downloads/crossmgr-install.py", line 788, in
dispatchargs.sub_command
File "/home/tim/Downloads/crossmgr-install.py", line 782, in
'install': lambda args: install( args.full ),
File "/home/tim/Downloads/crossmgr-install.py", line 669, in install
python_exe = env_setup( full )
File "/home/tim/Downloads/crossmgr-install.py", line 218, in env_setup
subprocess.check_output( [python_exe, '-m', 'pip', 'install', '--upgrade', 'pip'] )
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/tim/CrossMgr-env/bin/python3', '-m', 'pip', 'install', '--upgrade', 'pip']' returned non-zero exit status 1.

[moth754]

Working fine on windows. However video does not run. Have tried the update dependencies script.

All other applications run fine.

[esitarski]

On Linux, you need to right-click on the desktop shortcut to enable it for launch. I suspect something similar for MacOS. When you discover what is required, let me know and I will update the README.md. Regarding the failed call to pip on linux, it turns out that you can't just install modules into the global version of python. That was the fail. It makes sense, as a "bad actor" could replace a standard module with an evil one with the same name and compromise the entire system. The good news is that you can install modules automatically in a venv. But that creates a problem - you can't call it from inside the script as you need to call a different python exe. I figured out a way for doing this (an embarrassing hack), but it gets the job done. Latest has been posted.

[moth754]

Understood (I think). Will keep going with the MacOS and try and find it.

On linux - downloaded latest crossmgr-install.py and getting the same error. Distro is a relatively new install of PopOS

[moth754]

Will open a separate issue for Video not opening on windows

[moth754]

@esitarski out of interest, which Linux distro are you using for race timing?

[moth754]

Understood (I think). Will keep going with the MacOS and try and find it.

On linux - downloaded latest crossmgr-install.py and getting the same error. Distro is a relatively new install of PopOS

Have confirmed is also an issue on Linux Mint