doc: execution model - clear and complete example in documentation

[robwolfe]

BPO	4246
Nosy	@terryjreedy, @cben, @ezio-melotti
Files	scopes.py: Execution model example

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2008-11-01.14:59:30.657>
labels = ['type-feature', '3.9', '3.10', 'docs']
title = 'doc: execution model - clear and complete example in documentation'
updated_at = <Date 2020-10-25.22:53:29.725>
user = 'https://bugs.python.org/robwolfe'

bugs.python.org fields:

activity = <Date 2020-10-25.22:53:29.725>
actor = 'iritkatriel'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2008-11-01.14:59:30.657>
creator = 'robwolfe'
dependencies = []
files = ['11924']
hgrepos = []
issue_num = 4246
keywords = []
message_count = 4.0
messages = ['75441', '75614', '75635', '121783']
nosy_count = 5.0
nosy_names = ['terry.reedy', 'cben', 'ezio.melotti', 'robwolfe', 'docs@python']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue4246'
versions = ['Python 3.9', 'Python 3.10']

Linked PRs

• gh-48496: Added example and link to faq for UnboundLocalError in reference #93068
• [3.11] gh-48496: Added example and link to faq for UnboundLocalError in reference (GH-93068) #100442
• [3.10] gh-48496: Added example and link to faq for UnboundLocalError in reference (GH-93068) #100443

[robwolfe]

I'd like to propose adding some complete example regarding scopes and
bindings resolving to execution model description.
There is no week on pl.comp.lang.python without a question about
UnboundLocalError problem. I'm getting tired answering that. ;-)
It does not have to look so verbose as my (attached) example, but please
add some example, which will clarify this issue.

[terryjreedy]

Your example seem too verbose and diffuse. Perhaps something more
focused on what people do wrong would be more helpful. I presume you
mean something like this -- with or without x=2 before the def.

>>> def f():
	print (x)
	x = 1

>>> f()
Traceback (most recent call last):
  File "<pyshell#31>", line 1, in <module>
    f()
  File "<pyshell#30>", line 2, in f
    print (x)
UnboundLocalError: local variable 'x' referenced before assignment

What are the other ways people get the error.

[robwolfe]

People seem to understand that they can not use variable before
definition. But this dramatically change when they come across nested
functions. They don't understand when variable can be resolved from
outer scope and when can not, e.g:

def outer():
    x = 1
    def inner1():
        print(x)
    def inner2():
        print(x)
        # [... some instructions (maybe a lot) ...]
        x = 1

They are always confused why inner1 works but inner2 doesn't.

[cben]

The FAQ for this was much improved in 2009 (bpo-7290): http://docs.python.org/py3k/faq/programming.html#why-am-i-getting-an-unboundlocalerror-when-the-variable-has-a-value

To support the claim that this keeps biting people, at least the following bug reports all came from people misundestanding this:
bpo-10043
bpo-9049
bpo-7747
bpo-7276
bpo-6833
bpo-5763
bpo-4109 (understood effect of =, surprised by +=)
bpo-972467
bpo-850110
bpo-463640
These are just the people who were persistent enough to open a bug (and in most cases managed to produce a minimal example); many more ask on c.l.p, StackOverflow (>50 hits for UnboundLocalError, many of which are this exact issue) etc., or just give up.

[Interesting point: people mostly complain when the unbound reference occurs textually *before* the assignment (though there's selection bias here), and many of them complain that things happen "out of order". It seems half the misunderstanding is that people expect variables to *become* localized when first assigned - they don't understand it's a static decision affecting all occurences in a function.]

The central problem I believe is that when people try to modify a non-local var and get
UnboundLocalError: local variable foo referenced before assignment
their mental model of Python scopes is *wrong*, so the error message is *useless* for them (what 'local variable foo' is it talking about?), and have no idea where to look next.

Also, I'm afraid people just won't internalize this issue until it bites them at least once (I've seen a Python course where I had explained this, with a print-before-assignment example, and 2 days later a student was bitten by the exception and was at a loss. Therefore, I think providing a clear learning path from UnboundLocalError is crucial.

==>

I propose (i.e. am working on patch(s)) attacking it at many points:

(1) Expand UnboundLocalError message to point to help('NAMESPACES')
[Execution Model → Naming and Binding] and/or the FAQ.
(requirement IMHO: use a help() ref that can be followed the
from a Python prompt on an offline machine; not sure if FAQ can
work this way.)

(1B) Ideally, detect if the var is bound in outer/global scope 
     and suggest help('nonlocal') / help('global') approriately.

(1C) Ideally, improve UnboundLocalError to explain "foo is local 
     throughout the function because it's bound at line 42".

(2) Add an example to Naming and Binding section.
Currently, it's a bunch of legalese, defining 7 terms(!) before
you get to the meat. Mortal users won't survive that.
Being the language spec, the precise legalese must stay there;
but it'd be good to prepend it with a human-readable blurb and
example regarding this issue.

(3) Improve the tutorial. Sections 4.6 [Defining functions] and 9.2
[Scopes and Namespaces] are relevant. 4.6 mentions the issue of
assignment to global but neither section has a clear example.
And 9.2 is scary IMHO; I'll see if I can make it any better...

(4) Add examples to documentation of global & nonlocal?
Not clear if helpful, probably too much.