PythonTour/11-List-Comprehensions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# List Comprehensions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<!--BOOK_INFORMATION-->\n",
    "<img align=\"left\" style=\"padding-right:10px;\" src=\"fig/cover-small.jpg\">\n",
    "*This notebook contains an excerpt from the [Whirlwind Tour of Python](http://www.oreilly.com/programming/free/a-whirlwind-tour-of-python.csp) by Jake VanderPlas; the content is available [on GitHub](https://github.com/jakevdp/WhirlwindTourOfPython).*\n",
    "\n",
    "*The text and code are released under the [CC0](https://github.com/jakevdp/WhirlwindTourOfPython/blob/master/LICENSE) license; see also the companion project, the [Python Data Science Handbook](https://github.com/jakevdp/PythonDataScienceHandbook).*\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<!--NAVIGATION-->\n",
    "< [Iterators](10-Iterators.ipynb) | [Contents](Index.ipynb) | [Generators](12-Generators.ipynb) >"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "If you read enough Python code, you'll eventually come across the terse and efficient **construction** known as a *list comprehension*.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 4, 5, 7, 8, 10, 11, 13, 14, 16, 17, 19]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#5 % 3\n",
    "#[i**2 for i in range(10)]\n",
    "[i for i in range(20) if i % 3 > 0] "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The result of this is a list of numbers which excludes multiples of 3.\n",
    "\n",
    "While this example may seem a bit confusing at first, \n",
    "- as familiarity with Python grows, reading and writing list comprehensions will become second nature."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Basic List Comprehensions\n",
    "\n",
    "List comprehensions are simply a way \n",
    "- to compress a list-building for-loop into a single short, readable line.\n",
    "\n",
    "For example, here is a loop that constructs a list of the first 12 square integers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100, 121]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L = []\n",
    "for n in range(12):\n",
    "    L.append(n ** 2)\n",
    "L"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The list comprehension equivalent of this is the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100, 121]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[n ** 2 for n in range(12)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The meaning of this statement in plain English: \n",
    "- \"construct a list consisting of the square of ``n`` for each ``n`` up to 12\".\n",
    "\n",
    "This basic syntax is ``[``*``expr``* ``for`` *``var``* ``in`` *``iterable``*``]``, \n",
    "- *``expr``* is any valid expression, \n",
    "- *``var``* is a variable name, and \n",
    "- *``iterable``* is any iterable Python object."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Multiple Iteration\n",
    "\n",
    "Sometimes you want to build a list not just from one value, but from two. \n",
    "- To do this, simply add another ``for`` expression in the comprehension:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2)]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[(i, j) for i in range(2) for j in range(3)]# for k in range(4)]\n",
    "#[(i, j, k) for i in range(2) for j in range(3) for k in range(4)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### Notice\n",
    "- the second ``for`` expression acts as the interior index, varying the fastest in the resulting list.\n",
    "\n",
    "This type of construction can be extended to three, four, or more iterators within the comprehension, \n",
    "- though at some point code readibility will suffer!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Conditionals on the Iterator\n",
    "You can further control the iteration by adding a conditional to the end of the expression.\n",
    "\n",
    "In the first example of the section, we iterated over all numbers from 1 to 20, but left-out multiples of 3.\n",
    "\n",
    "Look at this again, and notice the construction:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 4, 5, 7, 8, 10, 11, 13, 14, 16, 17, 19]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[val for val in range(20) if val % 3 > 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- \"Construct a list of values for each value up to 20, but only if the value is not divisible by 3\".\n",
    "- The expression ``(i % 3 > 0)`` evaluates to ``True`` unless ``val`` is divisible by 3.\n",
    "\n",
    "Once you are comfortable with it, this is much easier to write – and to understand at a glance – than the equivalent loop syntax:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 4, 5, 7, 8, 10, 11, 13, 14, 16, 17, 19]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L = []\n",
    "for val in range(20):\n",
    "    if val % 3:\n",
    "        L.append(val)\n",
    "L"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Conditionals on the Value\n",
    "If you've programmed in C, you might be familiar with the single-line conditional enabled by the ``?`` operator:\n",
    "``` C\n",
    "int absval = (val < 0) ? -val : val\n",
    "```\n",
    "Python has something very similar to this, which is most often used within \n",
    "- list comprehensions, \n",
    "- ``lambda`` functions, and \n",
    "- other places where a simple expression is desired:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val = -10\n",
    "val if val >= 0 else -val"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "This simply duplicates the functionality of the built-in ``abs()`` function, \n",
    "- the construction lets you do some really interesting things within list comprehensions.\n",
    "\n",
    "This is getting pretty complicated now:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 4, 5, 7, 8, 10, 11, 13, 14, 16, 17, 19]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[i for i in range(20) if i % 3]\n",
    "# Constructing a list and leaving out multiples of 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "# Why leaves out multiples of 3?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "False\n"
     ]
    }
   ],
   "source": [
    "if 0 == True:\n",
    "    print(True)\n",
    "elif 0 == False:\n",
    "    print(False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, -2, -4, 5, 7, -8, -10, 11, 13, -14, -16, 17, 19]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[val if val % 2  else -val\n",
    " for val in range(20) if val % 3 ]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Note the line break within the list comprehension before the ``for`` expression: \n",
    "- this is valid in Python, and is often a nice way to break-up long list comprehensions for greater readibility.\n",
    "\n",
    "What we're doing is constructing a list, \n",
    "- leaving out multiples of 3, and \n",
    "- negating all mutliples of 2."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Once you understand the dynamics of list comprehensions, it's straightforward to move on to other types of comprehensions. The syntax is largely the same; the only difference is the type of bracket you use.\n",
    "\n",
    "For example, with curly braces you can create a ``set`` with a *set comprehension*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100, 121}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "{n**2 for n in range(12)}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Recall that a ``set`` is a collection that contains no duplicates.\n",
    "The set comprehension respects this rule, and eliminates any duplicate entries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{0, 1, 2}"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "{a % 3 for a in range(100000)}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "With a slight tweak, you can add a colon (``:``) to create a *dict comprehension*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "{n:n**2 for n in range(6)}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Finally, if you use parentheses rather than square brackets, you get what's called a *generator expression*:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "A generator expression is essentially a list comprehension \n",
    "- in which elements are generated as-needed rather than all at-once"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<generator object <genexpr> at 0x1027a5a50>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(n**2 for n in range(12))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 延伸阅读\n",
    "\n",
    "Python 中列表推导（list comprehension）相对于循环有什么优势？性能会更高吗？\n",
    "\n",
    "https://www.zhihu.com/question/34637934"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<!--NAVIGATION-->\n",
    "< [Iterators](10-Iterators.ipynb) | [Contents](Index.ipynb) | [Generators](12-Generators.ipynb) >"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}