Saving an image with mode "P" (palette) as TIFF saves a corrupted palette

[vashek]

Pillow (or the built-in libtiff) corrupts the palette when saving a mode P image as TIFF.
I have Pillow-2.7.0-py2.7.egg and Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on win32.
Simply opening an existing mode P image and saving it as TIFF will exhibit the problem. The first band (R) of the palette seems OK and the G and B bands are garbage. When I use compression="raw" (and therefore libtiff is not used), the palette is OK.
May be related to #718 but I don't get any exception, the original image in memory is OK (only the saved one is corrupted) and the length doesn't change.

[aclark4life]

Some code to demonstrate the issue would help (even if it's just "Simply opening an existing mode P image and saving it as TIFF will exhibit the problem") Thanks

[vashek]

Yup, sorry.

This makes a correct TIFF:
PIL.Image.open("gif.gif").save("tif.tif")

This makes a corrupted TIFF:
PIL.Image.open("gif.gif").save("tif.tif", compression="deflate")

As the input file, any ordinary paletted file will do - I just made a screenshot and saved it in Microsoft Paint as GIF. (Can/should I upload a sample as attachment somehow?)

[aclark4life]

Thanks, yes you can select or drag images, see below the comment box.

[vashek]

In fact, this can easily be demonstrated even without an input file.

This creates a black image:
PIL.Image.new("P", (100,100)).save("empty.tif")

This creates a (in my case) blue image:
PIL.Image.new("P", (100,100)).save("empty.tif", compression="lzw")

[vashek]

Sample gif.gif:
TIFs not supported here, apparently.

[vashek]

Here is the corrupted tif.tif: http://ibin.co/1w9kL7bUficR

[cgohlke]

For me Image.new("P", (100,100)).save("empty.tif") segfaults at https://github.com/python-pillow/Pillow/blob/master/libImaging/TiffDecode.c#L341:

    msvcr90.dll!000000006dc4e480()  
    [Frames below may be incorrect and/or missing, no symbols loaded for msvcr90.dll]   
    _imaging.pyd!_TIFFsetShortArray()  + 0x63 bytes C
    _imaging.pyd!_TIFFsetDoubleArray()  + 0x5fa bytes   C
>   _imaging.pyd!ImagingLibTiffSetField(ImagingCodecStateInstance * state=0x0000000002b38aa0, unsigned int tag=320, ...)  Line 341 + 0x17 bytes C
    _imaging.pyd!PyImaging_LibTiffEncoderNew(_object * self=0x0000000000000000, _object * args=0x0000000002b6dee8)  Line 773 + 0x26 bytes   C
    python27.dll!PyCFunction_Call(_object * func=0x0000000002b6dee8, _object * arg=0x0000000000000000, _object * kw=0x0000000002b750f8)  Line 81 + 0x8 bytes    C
    python27.dll!ext_do_call(_object * func=0x0000000001f7f108, _object * * * pp_stack=0x0000000000000001, int flags=44198792, int na=1, int nk=0)  Line 4343 + 0xaf bytes  C
    python27.dll!PyEval_EvalFrameEx(_frame * f=0x0000000002a269d0, int throwflag=-12252264)  Line 2721  C
    python27.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x0000000001ef3bb0, _object * globals=0x0000000000000004, _object * locals=0x0000000002b36db8, _object * * args=0x0000000000708a00, int argcount=4, _object * * kws=0x0000000001e77700, int kwcount=0, _object * * defs=0x0000000002b36db8, int defcount=1, _object * closure=0x0000000000000000)  Line 3265 + 0xa bytes   C
    python27.dll!fast_function(_object * func=0x0000000000000004, _object * * * pp_stack=0x0000000000000000, int n=45333752, int na=504125158, int nk=0)  Line 4133 C
    python27.dll!call_function(_object * * * pp_stack=0x0000000000000000, int oparg=131)  Line 4054 + 0x16 bytes    C
    python27.dll!PyEval_EvalFrameEx(_frame * f=0x0000000001e77458, int throwflag=1067064)  Line 2683    C
    python27.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x0000000002b638b0, _object * globals=0x0000000000000003, _object * locals=0x0000000000000000, _object * * args=0x000000001e280360, int argcount=3, _object * * kws=0x0000000001f7be20, int kwcount=0, _object * * defs=0x0000000000000000, int defcount=0, _object * closure=0x0000000000000000)  Line 3265 + 0xa bytes   C
    python27.dll!fast_function(_object * func=0x0000000000000003, _object * * * pp_stack=0x0000000000000000, int n=45526488, int na=43076360, int nk=0)  Line 4133  C
    python27.dll!call_function(_object * * * pp_stack=0x0000000000000000, int oparg=131)  Line 4054 + 0x16 bytes    C
    python27.dll!PyEval_EvalFrameEx(_frame * f=0x0000000001f7bc48, int throwflag=45351064)  Line 2683   C
    python27.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x0000000001f489b0, _object * globals=0x0000000000000002, _object * locals=0x0000000002b40098, _object * * args=0x0000000001f522b8, int argcount=2, _object * * kws=0x0000000001e40c20, int kwcount=1, _object * * defs=0x0000000002b40098, int defcount=1, _object * closure=0x0000000000000000)  Line 3265 + 0xa bytes   C
    python27.dll!fast_function(_object * func=0x0000000000000002, _object * * * pp_stack=0x0000000002b31e08, int n=45348120, int na=504125158, int nk=1)  Line 4133 C
    python27.dll!call_function(_object * * * pp_stack=0x0000000000000000, int oparg=131)  Line 4054 + 0x16 bytes    C
    python27.dll!PyEval_EvalFrameEx(_frame * f=0x0000000001e40a98, int throwflag=7375219)  Line 2683    C
    python27.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x0000000001cf8e30, _object * globals=0x000000006dc3f904, _object * locals=0x0000000001cf8e30, _object * * args=0x0000000001d36eb0, int argcount=0, _object * * kws=0x0000000000000000, int kwcount=0, _object * * defs=0x0000000000000000, int defcount=0, _object * closure=0x0000000000000000)  Line 3265 + 0xa bytes   C
    python27.dll!PyEval_EvalCode(PyCodeObject * co=0x000000006dcbc660, _object * globals=0x0000000000708973, _object * locals=0x0000000001d49268)  Line 673 C
    python27.dll!run_mod(_mod * mod=0x000000006dcbc660, const char * filename=0x0000000001ed13d8, _object * globals=0x0000000000000001, _object * locals=0x000000001e14562e, PyCompilerFlags * flags=0x000000000022ff38, _arena * arena=0x0000000000707080)  Line 1372  C
    python27.dll!PyRun_FileExFlags(_iobuf * fp=0x0000000000708973, const char * filename=0x0000000001d481f8, int start=30708328, _object * globals=0x0000000001d48330, _object * locals=0x0000000001d49268, int closeit=1, PyCompilerFlags * flags=0x000000000022ff38)  Line 1358   C
    python27.dll!PyRun_SimpleFileExFlags(_iobuf * fp=0x0000000000000001, const char * filename=0x0000000000708973, int closeit=1842071136, PyCompilerFlags * flags=0x000000000022ff38)  Line 950 + 0x2a bytes   C
    python27.dll!PyRun_AnyFileExFlags(_iobuf * fp=0x0000000000708940, const char * filename=0x0000000000000001, int closeit=0, PyCompilerFlags * flags=0x0000000000708973)  Line 754    C
    python27.dll!Py_Main(int argc=0, char * * argv=0x0000027a00000000)  Line 643 + 0x24 bytes   C
    python.exe!__tmainCRTStartup()  Line 586 + 0x19 bytes   C
    kernel32.dll!00007ffee2e013d2()     
    ntdll.dll!00007ffee36de954()    

The colormap is passed to libtiff as int * while it should be short * (https://github.com/python-pillow/Pillow/blob/master/encode.c#L760)

[wiredfool]

Right -- it's related to the mess that is tiff metadata.

There's a partial fix to that general issue in a PR, but it breaks the existing API. I'm not sure if this particular issue would be fixed, but it should be added as a test case when we disentangle the tiff stuff.

[hugovk]

Is this now fixed?

With macOS High Sierra / Python 3.6.5 / Pillow 5.0.0 / libtiff 4.0.9:

Python 3.6.4 (default, Mar  1 2018, 18:36:50)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from PIL import Image
>>> Image.open("gif.gif").save("tif1.tif")
>>> Image.open("gif.gif").save("tif2.tif", compression="deflate")

$ identify tif*
tif1.tif TIFF 98x121 98x121+0+0 8-bit sRGB 256c 13528B 0.000u 0:00.009
tif2.tif TIFF 98x121 98x121+0+0 8-bit Grayscale Gray 11980B 0.000u 0:00.000

Python 3.6.4 (default, Mar  1 2018, 18:36:50)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from PIL import Image
>>> Image.new("P", (100,100)).save("empty1.tif")
>>> Image.new("P", (100,100)).save("empty2.tif", compression="lzw")

$ identify empty*
empty1.tif TIFF 100x100 100x100+0+0 8-bit sRGB 256c 11670B 0.000u 0:00.000
empty2.tif TIFF 100x100 100x100+0+0 8-bit Grayscale Gray 10122B 0.000u 0:00.000

[radarhere]

I am also unable to replicate with current Pillow. Testing, I find that the segfault was resolved by #1620