- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Filtering words\n",
- "The challenge is to read a list of words from a dictionary, and keep only those words which contain only lower-case letters. Any \"word\" that contains an upper-case letter, punctuation, spaces, or similar should be rejected on the basis that it's a proper noun, and abbreviation, or something else that means it can't be a valid target word for Hangman."
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "# Import the libraries we'll need\n",
- "import re\n",
- "import random\n",
- "import string"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [],
- "prompt_number": 3
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Get the list of all words."
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "all_words = [w.strip() for w in open('/usr/share/dict/british-english').readlines()]\n",
- "len(all_words)"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "metadata": {},
- "output_type": "pyout",
- "prompt_number": 4,
- "text": [
- "99156"
- ]
- }
- ],
- "prompt_number": 4
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Checking a word\n",
- "\n",
- "## Explicit iteration over the word\n",
- "This function walks over the word, character by character, and checks if it's in the list of valid characters (as given in `string.ascii_lowercase`). If it's not, the `valid` flag is set to `False`. The final value is returned."
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "def check_word_explicit(word):\n",
- " valid = True\n",
- " for letter in word:\n",
- " if letter not in string.ascii_lowercase:\n",
- " valid = False\n",
- " return valid"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [],
- "prompt_number": 5
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Short-circuiting explicit iteration\n",
- "As above, but the function `return`s `False` as soon as it detects an invalid character. This should make it quicker to reject words."
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "def check_word_short_circuit(word):\n",
- " for letter in word:\n",
- " if letter not in string.ascii_lowercase:\n",
- " return False\n",
- " return True"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [],
- "prompt_number": 6
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Using comprehensions\n",
- "Use a comprehension function to convert the list of letters into a list of Booleans showing whether the character in that position is a valid letter. Use the built-in `all()` function to check that all the values in the list are `True`."
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "# Examples of the idea\n",
- "print('hello :', [letter in string.ascii_lowercase for letter in 'hello'])\n",
- "print('heLLo :', [letter in string.ascii_lowercase for letter in 'heLLo'])"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [
- {
- "output_type": "stream",
- "stream": "stdout",
- "text": [
- "hello : [True, True, True, True, True]\n",
- "heLLo : [True, True, False, False, True]\n"
- ]
- }
- ],
- "prompt_number": 7
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "def check_word_comprehension(word):\n",
- " return all(letter in string.ascii_lowercase for letter in word)"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [],
- "prompt_number": 8
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Short-circuited comprehensions\n",
- "An attempt to be clever. Can we stop the checking of letters as soon as we've found an invalid one?"
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "def check_word_comprehension_clever(word):\n",
- " return not any(letter not in string.ascii_lowercase for letter in word)"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [],
- "prompt_number": 9
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## A recursive definition\n",
- "A word if all lowercase if the first character is lowercase and the rest of the word is all lowercase. The base case is an empty word. This should evaluate to `True` because an empty list does not contain any invalid characters.\n",
- "\n",
- "Note the Pythonic use of \"truthiness\" values. If you try to take the Boolean value of a string, it evaluates as `False` if it's empty and `True` otherwise. Using \n",
- "\n",
- "` if word != '':` \n",
- "\n",
- "in the first line is just as correct, but not as Pythonic."
- ]
- },
- {
- "cell_type": "code",
- "collapsed": false,
- "input": [
- "def check_word_recursive(word):\n",
- " if word:\n",
- " if word[0] not in string.ascii_lowercase:\n",
- " return False\n",
- " else:\n",
- " return check_word_recursive(word[1:])\n",
- " else:\n",
- " return True"
- ],
- "language": "python",
- "metadata": {},
- "outputs": [],
- "prompt_number": 10
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Regular expressions\n",
- "A regular expression is a way of defining a finite state machine (FSM) that accepts some sequences of characters. They're used a lot whenever you want to process something text-based. In this case, the regex consists of:\n",
- "* `^` : match the start of the string\n",
- "* `[a-z]` : match a single character in the range `a` to `z`\n",
- "* `[a-z]+` : match a sequence of one one or more characters in the range `a` to `z`\n",
- "* `$` : match the end of the string\n",
- "This means you have a regular expression that matches strings containing just lower-case letters with nothing else between the matched letters and the start and end of the string. \n",
- "\n",
- "Python has the `re.compile` feature to build the specialised FSM that does the matching. This is faster if you want to use the same regular expression a lot. If you only want to use it a few times, it's often easier to just give the regex directly. See below for examples.\n",
- "\n",
- "Regular expresions are incredibly powerful, but take time to learn. See the [regular expression tutorial](http://www.regular-expressions.info/tutorial.html) for a guide."
- ]
- },