os.path.exists should not throw "Embedded NUL character" exception

[dolda2000]

BPO	29042
Nosy	@vstinner, @lazka

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-12-18.14:36:01.518>
created_at = <Date 2016-12-22.04:36:28.809>
labels = ['invalid', 'type-bug', 'library']
title = 'os.path.exists should not throw "Embedded NUL character" exception'
updated_at = <Date 2017-12-18.14:36:01.516>
user = 'https://bugs.python.org/Dolda2000'

bugs.python.org fields:

activity = <Date 2017-12-18.14:36:01.516>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2017-12-18.14:36:01.518>
closer = 'vstinner'
components = ['Library (Lib)']
creation = <Date 2016-12-22.04:36:28.809>
creator = 'Dolda2000'
dependencies = []
files = []
hgrepos = []
issue_num = 29042
keywords = []
message_count = 3.0
messages = ['283807', '283821', '308564']
nosy_count = 3.0
nosy_names = ['vstinner', 'Dolda2000', 'lazka']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue29042'
versions = ['Python 3.5']

[dolda2000]

Currently, calling os.path.exists on a path which contains NUL characters behaves consistently with most file-system calls by throwing an exception:

>>> os.path.exists('\0')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.5/genericpath.py", line 19, in exists
    os.stat(path)
ValueError: embedded null byte

However, os.path.exists is supposed to be a predicate returning whether there exists a file named by the path; it does not specify any particular method or system call for doing the test, and so reflecting the behavior of the underlying syscall used is not obviously desirable. A path containing an embedded NUL character simply cannot name an existing file, and therefore os.path.exists should return False for such a path.

[lazka]

Raising in case no valid path is passed seems fine to me. There are other cases where it fails:

python3 -c "import os; os.path.exists('\ud83d')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.5/genericpath.py", line 19, in exists
    os.stat(path)
UnicodeEncodeError: 'utf-8' codec can't encode character '\ud83d' in position 0: surrogates not allowed

LANG=C python3 -c "import os; os.path.exists('\xff')"       ~
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.5/genericpath.py", line 19, in exists
    os.stat(path)
UnicodeEncodeError: 'ascii' codec can't encode character '\xff' in position 0: ordinal not in range(128)

[vstinner]

A path containing an embedded NUL character simply cannot name an existing file, and therefore os.path.exists should return False for such a path.

I disagree. Python doesn't call the syscall and so must raise a different exception.

You must not pass a path with embedded NULL character/byte. That's all.

Write your own wrapper to os.path.exists() if you want to a different behaviour.