mimetypes.guess_type could not find mimetypes added by mimetypes.add_type i.e. in PY3.10.4 // OR : different return values (PY2.7 to PY3.9.7) vs (PY3.9.11 to PY3.10.4)

[ifrh]

I think something is broken in mimetype.guess_type or documentation did not cover the change ...

A small working example and its outputs for different Python versions are listed below.
In this example the value from a concrete mimetypes.types_map entry should
match to the returnvalue of mimetypes.guess_type in all tested Python versions:
But it did not; the return value of mimetypes.guess_type is wrong from my point of view.

Operating system and architecture:
Windows 10/WSL

$ uname -s -v -i -m -p -o
Linux #1237-Microsoft Sat Sep 11 14:32:00 PST 2021 x86_64 x86_64 x86_64 GNU/Linux

Ubuntu

$ uname -s -v -i -m -p -o
Linux #220-Ubuntu SMP Thu May 9 12:40:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Small working example : mimetest.py

import mimetypes
mimetypes.init()
MIMETYPE_ADDITIONAL_EXTENSIONS = [("text/x-r-script", ".R"), ]
for (mimetype, extension) in MIMETYPE_ADDITIONAL_EXTENSIONS:
    mimetypes.add_type(mimetype, extension, strict=True)
print("mimetypes.types_map['.R'] = %s , mimetypes.guess_type('example.R')[0] = %s" % (mimetypes.types_map['.R'],mimetypes.guess_type('example.R')[0])) 

Running this via multiple Python versions is showing the problem:

Python 2.7.17
mimetypes.types_map['.R'] = text/x-r-script , mimetypes.guess_type('example.R')[0] = text/x-r-script
Python 3.5.6
mimetypes.types_map['.R'] = text/x-r-script , mimetypes.guess_type('example.R')[0] = text/x-r-script
Python 3.6.13
mimetypes.types_map['.R'] = text/x-r-script , mimetypes.guess_type('example.R')[0] = text/x-r-script
Python 3.9.7
mimetypes.types_map['.R'] = text/x-r-script , mimetypes.guess_type('example.R')[0] = text/x-r-script
Python 3.9.11
mimetypes.types_map['.R'] = text/x-r-script , mimetypes.guess_type('example.R')[0] = None
Python 3.9.12
mimetypes.types_map['.R'] = text/x-r-script , mimetypes.guess_type('example.R')[0] = None
Python 3.10.4
mimetypes.types_map['.R'] = text/x-r-script , mimetypes.guess_type('example.R')[0] = None

perhaps this different behavior is a result of issue #64591
and its related merged pull requests

• [python:main] bpo-20392: Fix inconsistency with uppercase file extensions in mimetypes.guess_type #30229
• [python:3.9] [3.9] bpo-20392: Fix inconsistency with uppercase file extensions in mimetypes.guess_type (GH-30229) #31903
• [python:3.10] [3.10] bpo-20392: Fix inconsistency with uppercase file extensions in mimetypes.guess_type (GH-30229) #31904

Linked PRs

• gh-92455: Respect case-sensitive mimetype suffixes #148782

[ifrh]

old implementation

cpython/Lib/mimetypes.py

Lines 146 to 149 in 64a68c3

	if ext in types_map:
	return types_map[ext], encoding
	elif ext.lower() in types_map:
	return types_map[ext.lower()], encoding

current implementation

cpython/Lib/mimetypes.py

Lines 146 to 149 in 4802344

	ext = ext.lower()
	types_map = self.types_map[True]
	if ext in types_map:
	return types_map[ext], encoding

changing my small example adding both extensions ".r" and ".R" via mimetypes.add_type gives

$ python ./mimetest.py
mimetypes.types_map['.R'] = text/x-r-script , mimetypes.guess_type('example.R')[0] = text/x-r2-script

import mimetypes
mimetypes.init()
MIMETYPE_ADDITIONAL_EXTENSIONS = [("text/x-r-script", ".R"), ("text/x-r2-script", ".r"),]
for (mimetype, extension) in MIMETYPE_ADDITIONAL_EXTENSIONS:
    mimetypes.add_type(mimetype, extension, strict=True)
print("mimetypes.types_map['.R'] = %s , mimetypes.guess_type('example.R')[0] = %s" % (mimetypes.types_map['.R'],mimetypes.guess_type('example.R')[0])) 

With the current implementation mimetypes.guess_type cannot guess "own" mimetypes which had been added using uppercase letters.

In the docs https://docs.python.org/3/library/mimetypes.html#mimetypes.add_type there is not written, that mimetypes must be written in lowercase letters.

As you can see the implementation of add_type can add "file endings" with capital letters, but they cannot find via mimetypes.guess_type anymore:

cpython/Lib/mimetypes.py

Lines 80 to 95 in 4802344

	def add_type(self, type, ext, strict=True):
	"""Add a mapping between a type and an extension.
	When the extension is already known, the new
	type will replace the old one. When the type
	is already known the extension will be added
	to the list of known extensions.
	If strict is true, information will be added to
	list of standard types, else to the list of non-standard
	types.
	"""
	self.types_map[strict][ext] = type
	exts = self.types_map_inv[strict].setdefault(type, [])
	if ext not in exts:
	exts.append(ext)

should change to become compatible with the changed implementation of guessing method mimetypes.guess_type.

[iritkatriel]

CC @kumaraditya303

[ifrh]

just a ping. behavior is buggy in same way in newer Python versions than testet before (i.e. Python 3.13.11)

[yuanx749]

Friendly ping. PR #148782 is available for this issue.

[serhiy-storchaka]

There are also other issues. If we register an upper-case suffix, the lower-case suffix will not match. It is difficult to make it matching while keep distinction between different types for lower-case and upper-case suffixes, so we perhaps should leave this. But all this is undocumented. It should be documented.

There is question about the order. If we cannot find the exact match for an upper-case suffix in the strict mapping, should we try next the exact match in the non-strict mapping or the case-insensitive match in the strict mapping?