`obs build Debian_10` and `obs build Debian_11` failed with `The headers or library files could not be found for zlib`

[hswong3i]

What did you do?

I am packaging pillow as .deb with OBS service (see https://build.opensuse.org/package/show/home:alvistack/python-pillow-Pillow-9.5.0 and alvistack@61ac84f), but only failed for Debian 10/11 with The headers or library files could not be found for zlib, a required dependency when compiling Pillow from source..

After revert setup.py from 9.4.0, the build passed successfully.

What did you expect to happen?

As like as Debian Testing (https://build.opensuse.org/build/home:alvistack/Debian_Testing/x86_64/python-pillow-Pillow-9.5.0/_log), /usr/include/zlib.h should be detected and build should be passing.

What actually happened?

https://build.opensuse.org/build/home:alvistack/Debian_10/x86_64/python-pillow-Pillow-9.5.0/_log:

[  113s] running build_ext
[  113s] 
[  113s] 
[  113s] The headers or library files could not be found for zlib,
[  113s] a required dependency when compiling Pillow from source.
[  113s] 
[  113s] Please see the install instructions at:
[  113s]    https://pillow.readthedocs.io/en/latest/installation.html
[  113s] 
[  113s] Traceback (most recent call last):
[  113s]   File "setup.py", line 997, in <module>
[  113s]     zip_safe=not (debug_build() or PLATFORM_MINGW),
[  113s]   File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 108, in setup
[  113s]     return distutils.core.setup(**attrs)
[  113s]   File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 185, in setup
[  113s]     return run_commands(dist)
[  113s]   File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 201, in run_commands
[  113s]     dist.run_commands()
[  113s]   File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 969, in run_commands
[  113s]     self.run_command(cmd)
[  113s]   File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1221, in run_command
[  113s]     super().run_command(command)
[  113s]   File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
[  113s]     cmd_obj.run()
[  113s]   File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/build.py", line 131, in run
[  113s]     self.run_command(cmd_name)
[  113s]   File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 318, in run_command
[  113s]     self.distribution.run_command(command)
[  113s]   File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1221, in run_command
[  113s]     super().run_command(command)
[  113s]   File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
[  113s]     cmd_obj.run()
[  113s]   File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 84, in run
[  113s]     _build_ext.run(self)
[  113s]   File "/usr/lib/python3/dist-packages/Cython/Distutils/old_build_ext.py", line 186, in run
[  113s]     _build_ext.build_ext.run(self)
[  113s]   File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
[  113s]     self.build_extensions()
[  113s]   File "setup.py", line 809, in build_extensions
[  113s]     raise RequiredDependencyException(f)
[  113s] __main__.RequiredDependencyException: zlib
[  113s] 
[  113s] During handling of the above exception, another exception occurred:
[  113s] 
[  113s] Traceback (most recent call last):
[  113s]   File "setup.py", line 1010, in <module>
[  113s]     raise RequiredDependencyException(msg)
[  113s] __main__.RequiredDependencyException: 
[  113s] 
[  113s] The headers or library files could not be found for zlib,
[  113s] a required dependency when compiling Pillow from source.
[  113s] 
[  113s] Please see the install instructions at:
[  113s]    https://pillow.readthedocs.io/en/latest/installation.html

What are your OS, Python and Pillow versions?

• OS: Debian 10/11
• Python: 3.7/3.9
• Pillow: 9.5.0

$ sudo find /var/tmp/build-root/Debian*-x86_64/.mount/ -type f -name zlib.h
/var/tmp/build-root/Debian_10-x86_64/.mount/usr/include/zlib.h
/var/tmp/build-root/Debian_11-x86_64/.mount/usr/include/zlib.h
/var/tmp/build-root/Debian_Testing-x86_64/.mount/usr/include/zlib.h

[radarhere]

We do test Debian 11 in our CIs. I turned on debugging, and it is finding /usr/include/zlib.h correctly.

If you want, you could also turn on debugging to see what paths it checks -

Pillow/setup.py

Line 127 in 7d3a6aa

DEBUG = False

Do you have thoughts as to how we could test your exact setup? Is this something you can recreate using Docker?

[hswong3i]

If you want, you could also turn on debugging to see what paths it checks -

I am now rolling back setup.py with 9.5.0, enable debug mode, with following log message (https://build.opensuse.org/package/live_build_log/home:alvistack/python-pillow-Pillow-9.5.0/Debian_10/x86_64):

[   75s] Looking for `zlib` using pkg-config.
[   75s] Looking for `freetype2` using pkg-config.
[   75s] Appending path /usr/include/freetype2
[   75s] Appending path /usr/include/libpng16
[   75s] Looking for `harfbuzz` using pkg-config.
[   75s] Appending path /usr/include/harfbuzz
[   75s] Appending path /usr/include/glib-2.0
[   75s] Appending path /usr/lib/x86_64-linux-gnu/glib-2.0/include
[   75s] Looking for `fribidi` using pkg-config.
[   75s] Appending path /usr/include/fribidi
[   75s] Looking for `lcms2` using pkg-config.
[   75s] Looking for `libimagequant` using pkg-config.
[   75s] Appending path /usr/lib
[   75s] Appending path /usr/include
[   75s] Appending path /usr/local/lib
[   75s] Appending path /usr/local/include
[   75s] Appending path /lib
[   75s] Looking for zlib
[   75s] Checking for include file zlib.h in /usr/include/openjpeg-2.3
[   75s] Checking for include file zlib.h in /usr/include/x86_64-linux-gnu
[   75s] Checking for include file zlib.h in /usr/include/freetype2
[   75s] Checking for include file zlib.h in /usr/include/libpng16
[   75s] Checking for include file zlib.h in /usr/include/harfbuzz
[   75s] Checking for include file zlib.h in /usr/include/glib-2.0
[   75s] Checking for include file zlib.h in /usr/lib/x86_64-linux-gnu/glib-2.0/include
[   75s] Checking for include file zlib.h in /usr/include/fribidi
[   75s] Checking for include file zlib.h in /usr/include
[   75s] Found zlib.h
[   75s] Couldn't find library z in ['/usr/lib', '/usr/local/lib', '/lib', '/usr/lib']
[   75s] Looking for jpeg
[   75s] Checking for include file jpeglib.h in /usr/include/openjpeg-2.3
[   75s] Checking for include file jpeglib.h in /usr/include/x86_64-linux-gnu
[   75s] Checking for include file jpeglib.h in /usr/include/freetype2
[   75s] Checking for include file jpeglib.h in /usr/include/libpng16
[   75s] Checking for include file jpeglib.h in /usr/include/harfbuzz
[   75s] Checking for include file jpeglib.h in /usr/include/glib-2.0
[   75s] Checking for include file jpeglib.h in /usr/lib/x86_64-linux-gnu/glib-2.0/include
[   75s] Checking for include file jpeglib.h in /usr/include/fribidi
[   75s] Checking for include file jpeglib.h in /usr/include
[   75s] Found jpeglib.h
[   75s] Couldn't find library jpeg in ['/usr/lib', '/usr/local/lib', '/lib', '/usr/lib']
[   75s] Looking for jpeg2000
[   75s] Checking for openjpeg-#.# in /usr/include/openjpeg-2.3
[   75s] Checking for openjpeg-#.# in /usr/include/x86_64-linux-gnu
[   75s] Checking for openjpeg-#.# in /usr/include/freetype2
[   75s] Checking for openjpeg-#.# in /usr/include/libpng16
[   75s] Checking for openjpeg-#.# in /usr/include/harfbuzz
[   75s] Checking for openjpeg-#.# in /usr/include/glib-2.0
[   75s] Checking for openjpeg-#.# in /usr/lib/x86_64-linux-gnu/glib-2.0/include
[   75s] Checking for openjpeg-#.# in /usr/include/fribidi
[   75s] Checking for openjpeg-#.# in /usr/include
[   75s] Found openjpeg.h in /usr/include/openjpeg-2.1
[   75s] Best openjpeg version (2, 1) so far in /usr/include/openjpeg-2.1
[   75s] Found openjpeg.h in /usr/include/openjpeg-2.3
[   75s] Best openjpeg version (2, 3) so far in /usr/include/openjpeg-2.3
[   75s] Checking for openjpeg-#.# in /usr/local/include
[   75s] Checking for openjpeg-#.# in /usr/include/python3.7m
[   75s] Couldn't find library openjp2 in ['/usr/lib', '/usr/local/lib', '/lib', '/usr/lib']
[   75s] Looking for imagequant
[   75s] Checking for include file libimagequant.h in /usr/include/openjpeg-2.3
[   75s] Checking for include file libimagequant.h in /usr/include/x86_64-linux-gnu
[   75s] Checking for include file libimagequant.h in /usr/include/freetype2
[   75s] Checking for include file libimagequant.h in /usr/include/libpng16
[   75s] Checking for include file libimagequant.h in /usr/include/harfbuzz
[   75s] Checking for include file libimagequant.h in /usr/include/glib-2.0
[   75s] Checking for include file libimagequant.h in /usr/lib/x86_64-linux-gnu/glib-2.0/include
[   75s] Checking for include file libimagequant.h in /usr/include/fribidi
[   75s] Checking for include file libimagequant.h in /usr/include
[   75s] Checking for include file libimagequant.h in /usr/local/include
[   75s] Checking for include file libimagequant.h in /usr/include/python3.7m
[   75s] Looking for tiff
[   75s] Checking for include file tiff.h in /usr/include/openjpeg-2.3
[   75s] Checking for include file tiff.h in /usr/include/x86_64-linux-gnu
[   75s] Found tiff.h
[   75s] Couldn't find library tiff in ['/usr/lib', '/usr/local/lib', '/lib', '/usr/lib']
[   75s] Looking for freetype
[   75s] Couldn't find library freetype in ['/usr/lib', '/usr/local/lib', '/lib', '/usr/lib']
[   75s] Looking for lcms
[   75s] Checking for include file lcms2.h in /usr/include/openjpeg-2.3
[   75s] Checking for include file lcms2.h in /usr/include/x86_64-linux-gnu
[   75s] Checking for include file lcms2.h in /usr/include/freetype2
[   75s] Checking for include file lcms2.h in /usr/include/libpng16
[   75s] Checking for include file lcms2.h in /usr/include/harfbuzz
[   75s] Checking for include file lcms2.h in /usr/include/glib-2.0
[   75s] Checking for include file lcms2.h in /usr/lib/x86_64-linux-gnu/glib-2.0/include
[   75s] Checking for include file lcms2.h in /usr/include/fribidi
[   75s] Checking for include file lcms2.h in /usr/include
[   75s] Found lcms2.h
[   75s] Couldn't find library lcms2 in ['/usr/lib', '/usr/local/lib', '/lib', '/usr/lib']
[   75s] Couldn't find library lcms2_static in ['/usr/lib', '/usr/local/lib', '/lib', '/usr/lib']
[   75s] Looking for webp
[   75s] Checking for include file webp/encode.h in /usr/include/openjpeg-2.3
[   75s] Checking for include file webp/encode.h in /usr/include/x86_64-linux-gnu
[   75s] Checking for include file webp/encode.h in /usr/include/freetype2
[   75s] Checking for include file webp/encode.h in /usr/include/libpng16
[   75s] Checking for include file webp/encode.h in /usr/include/harfbuzz
[   75s] Checking for include file webp/encode.h in /usr/include/glib-2.0
[   75s] Checking for include file webp/encode.h in /usr/lib/x86_64-linux-gnu/glib-2.0/include
[   75s] Checking for include file webp/encode.h in /usr/include/fribidi
[   75s] Checking for include file webp/encode.h in /usr/include
[   75s] Found webp/encode.h
[   75s] Checking for include file webp/decode.h in /usr/include/openjpeg-2.3
[   75s] Checking for include file webp/decode.h in /usr/include/x86_64-linux-gnu
[   75s] Checking for include file webp/decode.h in /usr/include/freetype2
[   75s] Checking for include file webp/decode.h in /usr/include/libpng16
[   75s] Checking for include file webp/decode.h in /usr/include/harfbuzz
[   75s] Checking for include file webp/decode.h in /usr/include/glib-2.0
[   75s] Checking for include file webp/decode.h in /usr/lib/x86_64-linux-gnu/glib-2.0/include
[   75s] Checking for include file webp/decode.h in /usr/include/fribidi
[   75s] Checking for include file webp/decode.h in /usr/include
[   75s] Found webp/decode.h
[   75s] Couldn't find library webp in ['/usr/lib', '/usr/local/lib', '/lib', '/usr/lib']
[   75s] Couldn't find library libwebp in ['/usr/lib', '/usr/local/lib', '/lib', '/usr/lib']
[   75s] Looking for webpmux
[   75s] Checking for include file webp/mux.h in /usr/include/openjpeg-2.3
[   75s] Checking for include file webp/mux.h in /usr/include/x86_64-linux-gnu
[   75s] Checking for include file webp/mux.h in /usr/include/freetype2
[   75s] Checking for include file webp/mux.h in /usr/include/libpng16
[   75s] Checking for include file webp/mux.h in /usr/include/harfbuzz
[   75s] Checking for include file webp/mux.h in /usr/include/glib-2.0
[   75s] Checking for include file webp/mux.h in /usr/lib/x86_64-linux-gnu/glib-2.0/include
[   75s] Checking for include file webp/mux.h in /usr/include/fribidi
[   75s] Checking for include file webp/mux.h in /usr/include
[   75s] Found webp/mux.h
[   75s] Checking for include file webp/demux.h in /usr/include/openjpeg-2.3
[   75s] Checking for include file webp/demux.h in /usr/include/x86_64-linux-gnu
[   75s] Checking for include file webp/demux.h in /usr/include/freetype2
[   75s] Checking for include file webp/demux.h in /usr/include/libpng16
[   75s] Checking for include file webp/demux.h in /usr/include/harfbuzz
[   75s] Checking for include file webp/demux.h in /usr/include/glib-2.0
[   75s] Checking for include file webp/demux.h in /usr/lib/x86_64-linux-gnu/glib-2.0/include
[   75s] Checking for include file webp/demux.h in /usr/include/fribidi
[   75s] Checking for include file webp/demux.h in /usr/include
[   75s] Found webp/demux.h
[   75s] Couldn't find library webpmux in ['/usr/lib', '/usr/local/lib', '/lib', '/usr/lib']
[   75s] Couldn't find library libwebpmux in ['/usr/lib', '/usr/local/lib', '/lib', '/usr/lib']
[   75s] Looking for xcb
[   75s] Checking for include file xcb/xcb.h in /usr/include/openjpeg-2.3
[   75s] Checking for include file xcb/xcb.h in /usr/include/x86_64-linux-gnu
[   75s] Checking for include file xcb/xcb.h in /usr/include/freetype2
[   75s] Checking for include file xcb/xcb.h in /usr/include/libpng16
[   75s] Checking for include file xcb/xcb.h in /usr/include/harfbuzz
[   75s] 
[   75s] The headers or library files could not be found for zlib,
[   75s] a required dependency when compiling Pillow from source.
[   75s] 
[   75s] Please see the install instructions at:
[   75s]    https://pillow.readthedocs.io/en/latest/installation.html

In case of Debian Testing (https://build.opensuse.org/package/live_build_log/home:alvistack/python-pillow-Pillow-9.5.0/Debian_Testing/x86_64):

[  180s] Looking for `zlib` using pkg-config.
[  180s] Looking for `freetype2` using pkg-config.
[  180s] Appending path /usr/include/freetype2
[  180s] Appending path /usr/include/libpng16
[  180s] Looking for `harfbuzz` using pkg-config.
[  180s] Appending path /usr/include/harfbuzz
[  180s] Appending path /usr/include/glib-2.0
[  180s] Appending path /usr/lib/x86_64-linux-gnu/glib-2.0/include
[  180s] Looking for `fribidi` using pkg-config.
[  180s] Appending path /usr/include/fribidi
[  180s] Looking for `lcms2` using pkg-config.
[  180s] Looking for `libimagequant` using pkg-config.
[  180s] Appending path /usr/lib
[  180s] Appending path /usr/local/lib
[  180s] Appending path /usr/local/include
[  180s] Looking for zlib
[  180s] Checking for include file zlib.h in /usr/include
[  180s] Found zlib.h
[  180s] Found library z at /usr/lib/x86_64-linux-gnu/libz.so

[hswong3i]

After line-by-line diff and revert setup.py from 9.5.0 to 9.4.0, this issue could be fixed by revert #7044:

curl -skL https://github.com/python-pillow/Pillow/pull/7044.patch | patch -p1 -R

From my point of view, when building package with obs build Debian_10, the PATH environment variable may not setup correctly, and so couldn't initialize the use of ldconfig correctly, too.

[Yay295]

Is there a reason you can't set up your path correctly?

[radarhere]

Does #7068 fix the problem?

[adisbladis]

We ran into the same issue in https://github.com/nix-community/poetry2nix.
I bisected Pillow and found 04cf5e2 to be the commit with the regression.

cc @sebastic

[sebastic]

Cannot reproduce with the 9.5.0 tag in a bullseye chroot:

[...]
Looking for zlib
Checking for include file zlib.h in /usr/include/x86_64-linux-gnu
Checking for include file zlib.h in /usr/include/freetype2
Checking for include file zlib.h in /usr/include/libpng16
Checking for include file zlib.h in /usr/include/harfbuzz
Checking for include file zlib.h in /usr/include/glib-2.0
Checking for include file zlib.h in /usr/lib/x86_64-linux-gnu/glib-2.0/include
Checking for include file zlib.h in /usr/include/fribidi
Checking for include file zlib.h in /usr/include
Found zlib.h
Found library z at /usr/lib/x86_64-linux-gnu/libz.so
[...]
--------------------------------------------------------------------
PIL SETUP SUMMARY
--------------------------------------------------------------------
version      Pillow 9.5.0
platform     linux 3.9.2 (default, Feb 28 2021, 17:03:44)
             [GCC 10.2.1 20210110]
--------------------------------------------------------------------
--- JPEG support available
--- OPENJPEG (JPEG2000) support available (2.4)
--- ZLIB (PNG/ZIP) support available
--- LIBIMAGEQUANT support available
--- LIBTIFF support available
--- FREETYPE2 support available
--- RAQM (Text shaping) support available
--- LITTLECMS2 support available
--- WEBP support available
--- WEBPMUX support available
--- XCB (X protocol) support available
--------------------------------------------------------------------
To check the build, run the selftest.py script.

# PYTHONPATH=$(pwd)/build/lib.linux-x86_64-3.9 python3 selftest.py
--------------------------------------------------------------------
Pillow 9.5.0
Python 3.9.2 (default, Feb 28 2021, 17:03:44)
       [GCC 10.2.1 20210110]
--------------------------------------------------------------------
Python modules loaded from /tmp/buildd/Pillow/build/lib.linux-x86_64-3.9/PIL
Binary modules loaded from /tmp/buildd/Pillow/build/lib.linux-x86_64-3.9/PIL
--------------------------------------------------------------------
--- PIL CORE support ok, compiled for 9.5.0
--- TKINTER support ok, loaded 8.6
--- FREETYPE2 support ok, loaded 2.10.4
--- LITTLECMS2 support ok, loaded 2.12
--- WEBP support ok, loaded 0.6.1
--- WEBP Transparency support ok
--- WEBPMUX support ok
--- WEBP Animation support ok
--- JPEG support ok, compiled for libjpeg-turbo 2.0.6
--- OPENJPEG (JPEG2000) support ok, loaded 2.4.0
--- ZLIB (PNG/ZIP) support ok, loaded 1.2.11
--- LIBTIFF support ok, loaded 4.2.0
--- RAQM (Bidirectional Text) support ok, loaded 0.7.0
--- LIBIMAGEQUANT (Quantization method) support ok, loaded 2.12.0
--- XCB (X protocol) support ok
--------------------------------------------------------------------
Running selftest:
--- 59 tests passed.

[adisbladis]

I've narrowed this down to a type confusion issue in #7069.

[radarhere]

@adisbladis I'm not convinced that your problem is the same as the original problem in this issue, considering that the original poster found that reverting #7044 fixed their situation.

Unless you tell me that reverting #7044 fixes your situation as well, would you mind opening a new issue?