output.json

[
    {
        "section": "Page 1",
        "content": [
            {
                "type": "text",
                "text": "Introduction to Theory of Computation\nAnil Maheshwari Michiel Smid\nSchool of Computer Science\nCarleton University\nOttawa\nCanada\nfanil,michielg@scs.carleton.ca\nAugust 29, 2024\n"
            }
        ]
    },
    {
        "section": "Page 2",
        "content": [
            {
                "type": "text",
                "text": "ii Contents"
            }
        ]
    },
    {
        "section": "Page 3",
        "content": [
            {
                "type": "text",
                "text": "Contents\nPreface vi\n1 Introduction 1\n1.1 Purpose and motivation . . . . . . . . . . . . . . . . . . . . . 1\n1.1.1 Complexity theory . . . . . . . . . . . . . . . . . . . . 2\n1.1.2 Computability theory . . . . . . . . . . . . . . . . . . . 2\n1.1.3 Automata theory . . . . . . . . . . . . . . . . . . . . . 3\n1.1.4 This course . . . . . . . . . . . . . . . . . . . . . . . . 3\n1.2 Mathematical preliminaries . . . . . . . . . . . . . . . . . . . 4\n1.3 Proof techniques . . . . . . . . . . . . . . . . . . . . . . . . . 7\n1.3.1 Direct proofs . . . . . . . . . . . . . . . . . . . . . . . 8\n1.3.2 Constructive proofs . . . . . . . . . . . . . . . . . . . . 9\n1.3.3 Nonconstructive proofs . . . . . . . . . . . . . . . . . . 10\n1.3.4 Proofs by contradiction . . . . . . . . . . . . . . . . . . 11\n1.3.5 The pigeon hole principle . . . . . . . . . . . . . . . . . 12\n1.3.6 Proofs by induction . . . . . . . . . . . . . . . . . . . . 13\n1.3.7 More examples of proofs . . . . . . . . . . . . . . . . . 15\nExercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18\n2 Finite Automata and Regular Languages 21\n2.1 An example: Controling a toll gate . . . . . . . . . . . . . . . 21\n2.2 Deterministic \fnite automata . . . . . . . . . . . . . . . . . . 23\n2.2.1 A \frst example of a \fnite automaton . . . . . . . . . . 26\n2.2.2 A second example of a \fnite automaton . . . . . . . . 28\n2.2.3 A third example of a \fnite automaton . . . . . . . . . 29\n2.3 Regular operations . . . . . . . . . . . . . . . . . . . . . . . . 31\n2.4 Nondeterministic \fnite automata . . . . . . . . . . . . . . . . 35\n2.4.1 A \frst example . . . . . . . . . . . . . . . . . . . . . . 35"
            }
        ]
    },
    {
        "section": "Page 4",
        "content": [
            {
                "type": "text",
                "text": "iv Contents\n2.4.2 A second example . . . . . . . . . . . . . . . . . . . . . 37\n2.4.3 A third example . . . . . . . . . . . . . . . . . . . . . . 38\n2.4.4 De\fnition of nondeterministic \fnite automaton . . . . 39\n2.5 Equivalence of DFAs and NFAs . . . . . . . . . . . . . . . . . 41\n2.5.1 An example . . . . . . . . . . . . . . . . . . . . . . . . 44\n2.6 Closure under the regular operations . . . . . . . . . . . . . . 48\n2.7 Regular expressions . . . . . . . . . . . . . . . . . . . . . . . . 52\n2.8 Equivalence of regular expressions and regular languages . . . 57\n2.8.1 Every regular expression describes a regular language . 58\n2.8.2 Converting a DFA to a regular expression . . . . . . . 61\n2.9 The pumping lemma and nonregular languages . . . . . . . . . 68\n2.9.1 Applications of the pumping lemma . . . . . . . . . . . 70\n2.10 Higman's Theorem . . . . . . . . . . . . . . . . . . . . . . . . 77\n2.10.1 Dickson's Theorem . . . . . . . . . . . . . . . . . . . . 77\n2.10.2 Proof of Higman's Theorem . . . . . . . . . . . . . . . 78\nExercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81\n3 Context-Free Languages 91\n3.1 Context-free grammars . . . . . . . . . . . . . . . . . . . . . . 91\n3.2 Examples of context-free grammars . . . . . . . . . . . . . . . 94\n3.2.1 Properly nested parentheses . . . . . . . . . . . . . . . 94\n3.2.2 A context-free grammar for a nonregular language . . . 95\n3.2.3 A context-free grammar for the complement of a non-\nregular language . . . . . . . . . . . . . . . . . . . . . 97\n3.2.4 A context-free grammar that veri\fes addition . . . . . 98\n3.3 Regular languages are context-free . . . . . . . . . . . . . . . . 100\n3.3.1 An example . . . . . . . . . . . . . . . . . . . . . . . . 102\n3.4 Chomsky normal form . . . . . . . . . . . . . . . . . . . . . . 104\n3.4.1 An example . . . . . . . . . . . . . . . . . . . . . . . . 109\n3.5 Pushdown automata . . . . . . . . . . . . . . . . . . . . . . . 112\n3.6 Examples of pushdown automata . . . . . . . . . . . . . . . . 116\n3.6.1 Properly nested parentheses . . . . . . . . . . . . . . . 116\n3.6.2 Strings of the form 0n1n. . . . . . . . . . . . . . . . . 117\n3.6.3 Strings with bin the middle . . . . . . . . . . . . . . . 118\n3.7 Equivalence of pushdown automata and context-free grammars 120\n3.8 The pumping lemma for context-free languages . . . . . . . . 124\n3.8.1 Proof of the pumping lemma . . . . . . . . . . . . . . . 125\n3.8.2 Applications of the pumping lemma . . . . . . . . . . . 128"
            }
        ]
    },
    {
        "section": "Page 5",
        "content": [
            {
                "type": "text",
                "text": "Contents v\nExercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132\n4 Turing Machines and the Church-Turing Thesis 137\n4.1 De\fnition of a Turing machine . . . . . . . . . . . . . . . . . . 137\n4.2 Examples of Turing machines . . . . . . . . . . . . . . . . . . 141\n4.2.1 Accepting palindromes using one tape . . . . . . . . . 141\n4.2.2 Accepting palindromes using two tapes . . . . . . . . . 142\n4.2.3 Accepting anbncnusing one tape . . . . . . . . . . . . . 143\n4.2.4 Accepting anbncnusing tape alphabet fa;b;c;2g. . . . 145\n4.2.5 Accepting ambncmnusing one tape . . . . . . . . . . . . 147\n4.3 Multi-tape Turing machines . . . . . . . . . . . . . . . . . . . 148\n4.4 The Church-Turing Thesis . . . . . . . . . . . . . . . . . . . . 151\nExercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152\n5 Decidable and Undecidable Languages 157\n5.1 Decidability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157\n5.1.1 The language ADFA. . . . . . . . . . . . . . . . . . . . 158\n5.1.2 The language ANFA. . . . . . . . . . . . . . . . . . . . 159\n5.1.3 The language ACFG. . . . . . . . . . . . . . . . . . . . 160\n5.1.4 The language ATM. . . . . . . . . . . . . . . . . . . . 161\n5.1.5 The Halting Problem . . . . . . . . . . . . . . . . . . . 163\n5.2 Countable sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 164\n5.2.1 The Halting Problem revisited . . . . . . . . . . . . . . 168\n5.3 Rice's Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 170\n5.3.1 Proof of Rice's Theorem . . . . . . . . . . . . . . . . . 171\n5.4 Enumerability . . . . . . . . . . . . . . . . . . . . . . . . . . . 173\n5.4.1 Hilbert's problem . . . . . . . . . . . . . . . . . . . . . 174\n5.4.2 The language ATM. . . . . . . . . . . . . . . . . . . . 176\n5.5 Where does the term \\enumerable\" come from? . . . . . . . . 177\n5.6 Most languages are not enumerable . . . . . . . . . . . . . . . 180\n5.6.1 The set of enumerable languages is countable . . . . . 180\n5.6.2 The set of all languages is not countable . . . . . . . . 181\n5.6.3 There are languages that are not enumerable . . . . . . 183\n5.7 The relationship between decidable and enumerable languages 184\n5.8 A language Asuch that both AandAare not enumerable . . 186\n5.8.1 EQTMis not enumerable . . . . . . . . . . . . . . . . . 186\n5.8.2 EQTMis not enumerable . . . . . . . . . . . . . . . . . 188\nExercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189"
            }
        ]
    },
    {
        "section": "Page 6",
        "content": [
            {
                "type": "text",
                "text": "vi Contents\n6 Complexity Theory 197\n6.1 The running time of algorithms . . . . . . . . . . . . . . . . . 197\n6.2 The complexity class P. . . . . . . . . . . . . . . . . . . . . . 199\n6.2.1 Some examples . . . . . . . . . . . . . . . . . . . . . . 199\n6.3 The complexity class NP. . . . . . . . . . . . . . . . . . . . . 202\n6.3.1 Pis contained in NP. . . . . . . . . . . . . . . . . . . 208\n6.3.2 Deciding NP-languages in exponential time . . . . . . 208\n6.3.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 211\n6.4 Non-deterministic algorithms . . . . . . . . . . . . . . . . . . 211\n6.5NP-complete languages . . . . . . . . . . . . . . . . . . . . . 213\n6.5.1 Two examples of reductions . . . . . . . . . . . . . . . 215\n6.5.2 De\fnition of NP-completeness . . . . . . . . . . . . . . 220\n6.5.3 An NP-complete domino game . . . . . . . . . . . . . 222\n6.5.4 Examples of NP-complete languages . . . . . . . . . . 231\nExercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235\n7 Summary 239"
            }
        ]
    },
    {
        "section": "Page 7",
        "content": [
            {
                "type": "text",
                "text": "Preface\nThis is a free textbook for an undergraduate course on the Theory of Com-\nputation, which we have been teaching at Carleton University since 2002.\nUntil the 2011/2012 academic year, this course was o\u000bered as a second-year\ncourse (COMP 2805) and was compulsory for all Computer Science students.\nStarting with the 2012/2013 academic year, the course has been downgraded\nto a third-year optional course (COMP 3803).\nWe have been developing this book since we started teaching this course.\nCurrently, we cover most of the material from Chapters 2{5 during a 12-week\nterm with three hours of classes per week.\nThe material from Chapter 6, on Complexity Theory, is taught in the\nthird-year course COMP 3804 (Design and Analysis of Algorithms). In the\nearly years of COMP 2805, we gave a two-lecture overview of Complexity\nTheory at the end of the term. Even though this overview has disappeared\nfrom the course, we decided to keep Chapter 6. This chapter has not been\nrevised/modi\fed for a long time.\nThe course as we teach it today has been in\ruenced by the following two\ntextbooks:\n\u000fIntroduction to the Theory of Computation (second edition), by Michael\nSipser, Thomson Course Technnology, Boston, 2006.\n\u000fEinf uhrung in die Theoretische Informatik, by Klaus Wagner, Springer-\nVerlag, Berlin, 1994.\nBesides reading this text, we recommend that you also take a look at\nthese excellent textbooks, as well as one or more of the following ones:\n\u000fElements of the Theory of Computation (second edition), by Harry\nLewis and Christos Papadimitriou, Prentice-Hall, 1998."
            }
        ]
    },
    {
        "section": "Page 8",
        "content": [
            {
                "type": "text",
                "text": "viii\n\u000fIntroduction to Languages and the Theory of Computation (third edi-\ntion), by John Martin, McGraw-Hill, 2003.\n\u000fIntroduction to Automata Theory, Languages, and Computation (third\nedition), by John Hopcroft, Rajeev Motwani, Je\u000brey Ullman, Addison\nWesley, 2007.\nPlease let us know if you \fnd errors, typos, simpler proofs, comments,\nomissions, or if you think that some parts of the book \\need improvement\"."
            }
        ]
    },
    {
        "section": "Page 9",
        "content": [
            {
                "type": "text",
                "text": "Chapter 1\nIntroduction\n1.1 Purpose and motivation\nThis course is on the Theory of Computation , which tries to answer the\nfollowing questions:\n\u000fWhat are the mathematical properties of computer hardware and soft-\nware?\n\u000fWhat is a computation and what is an algorithm ? Can we give rigorous\nmathematical de\fnitions of these notions?\n\u000fWhat are the limitations of computers? Can \\everything\" be com-\nputed? (As we will see, the answer to this question is \\no\".)\nPurpose of the Theory of Computation: Develop formal math-\nematical models of computation that re\rect real-world computers.\nThis \feld of research was started by mathematicians and logicians in the\n1930's, when they were trying to understand the meaning of a \\computation\".\nA central question asked was whether all mathematical problems can be\nsolved in a systematic way. The research that started in those days led to\ncomputers as we know them today.\nNowadays, the Theory of Computation can be divided into the follow-\ning three areas: Complexity Theory, Computability Theory, and Automata\nTheory."
            }
        ]
    },
    {
        "section": "Page 10",
        "content": [
            {
                "type": "text",
                "text": "2 Chapter 1. Introduction\n1.1.1 Complexity theory\nThe main question asked in this area is \\What makes some problems com-\nputationally hard and other problems easy?\"\nInformally, a problem is called \\easy\", if it is e\u000eciently solvable. Exam-\nples of \\easy\" problems are (i) sorting a sequence of, say, 1,000,000 numbers,\n(ii) searching for a name in a telephone directory, and (iii) computing the\nfastest way to drive from Ottawa to Miami. On the other hand, a problem is\ncalled \\hard\", if it cannot be solved e\u000eciently, or if we don't know whether\nit can be solved e\u000eciently. Examples of \\hard\" problems are (i) time table\nscheduling for all courses at Carleton, (ii) factoring a 300-digit integer into\nits prime factors, and (iii) computing a layout for chips in VLSI.\nCentral Question in Complexity Theory: Classify problems ac-\ncording to their degree of \\di\u000eculty\". Give a rigorous proof that\nproblems that seem to be \\hard\" are really \\hard\".\n1.1.2 Computability theory\nIn the 1930's, G odel, Turing, and Church discovered that some of the fun-\ndamental mathematical problems cannot be solved by a \\computer\". (This\nmay sound strange, because computers were invented only in the 1940's).\nAn example of such a problem is \\Is an arbitrary mathematical statement\ntrue or false?\" To attack such a problem, we need formal de\fnitions of the\nnotions of\n\u000fcomputer,\n\u000falgorithm, and\n\u000fcomputation.\nThe theoretical models that were proposed in order to understand solvable\nand unsolvable problems led to the development of real computers.\nCentral Question in Computability Theory: Classify problems\nas being solvable or unsolvable."
            }
        ]
    },
    {
        "section": "Page 11",
        "content": [
            {
                "type": "text",
                "text": "1.1. Purpose and motivation 3\n1.1.3 Automata theory\nAutomata Theory deals with de\fnitions and properties of di\u000berent types of\n\\computation models\". Examples of such models are:\n\u000fFinite Automata. These are used in text processing, compilers, and\nhardware design.\n\u000fContext-Free Grammars. These are used to de\fne programming lan-\nguages and in Arti\fcial Intelligence.\n\u000fTuring Machines. These form a simple abstract model of a \\real\"\ncomputer, such as your PC at home.\nCentral Question in Automata Theory: Do these models have\nthe same power, or can one model solve more problems than the\nother?\n1.1.4 This course\nIn this course, we will study the last two areas in reverse order: We will start\nwith Automata Theory, followed by Computability Theory. The \frst area,\nComplexity Theory, will be covered in COMP 3804.\nActually, before we start, we will review some mathematical proof tech-\nniques. As you may guess, this is a fairly theoretical course, with lots of\nde\fnitions, theorems, and proofs. You may guess this course is fun stu\u000b for\nmath lovers, but boring and irrelevant for others. You guessed it wrong , and\nhere are the reasons:\n1. This course is about the fundamental capabilities and limitations of\ncomputers. These topics form the core of computer science.\n2. It is about mathematical properties of computer hardware and software.\n3. This theory is very much relevant to practice, for example, in the design\nof new programming languages, compilers, string searching, pattern\nmatching, computer security, arti\fcial intelligence, etc., etc.\n4. This course helps you to learn problem solving skills. Theory teaches\nyou how to think, prove, argue, solve problems, express, and abstract."
            }
        ]
    },
    {
        "section": "Page 12",
        "content": [
            {
                "type": "text",
                "text": "4 Chapter 1. Introduction\n5. This theory simpli\fes the complex computers to an abstract and simple\nmathematical model, and helps you to understand them better.\n6. This course is about rigorously analyzing capabilities and limitations\nof systems.\nWhere does this course \ft in the Computer Science Curriculum at Car-\nleton University? It is a theory course that is the third part in the series\nCOMP 1805, COMP 2804, COMP 3803, COMP 3804, and COMP 4804.\nThis course also widens your understanding of computers and will in\ruence\nother courses including Compilers, Programming Languages, and Arti\fcial\nIntelligence.\n1.2 Mathematical preliminaries\nThroughout this course, we will assume that you know the following mathe-\nmatical concepts:\n1. A setis a collection of well-de\fned objects. Examples are (i) the set of\nall Dutch Olympic Gold Medallists, (ii) the set of all pubs in Ottawa,\nand (iii) the set of all even natural numbers.\n2. The set of natural numbers isN=f1;2;3;:::g.\n3. The set of integers isZ=f:::;\u00003;\u00002;\u00001;0;1;2;3;:::g.\n4. The set of rational numbers isQ=fm=n :m2Z;n2Z;n6= 0g.\n5. The set of real numbers is denoted by R.\n6. IfAandBare sets, then Ais asubset ofB, written as A\u0012B, if every\nelement of Ais also an element of B. For example, the set of even\nnatural numbers is a subset of the set of all natural numbers. Every\nsetAis a subset of itself, i.e., A\u0012A. The empty set is a subset of\nevery setA, i.e.,;\u0012A.\n7. IfBis a set, then the power setP(B) ofBis de\fned to be the set of\nall subsets of B:\nP(B) =fA:A\u0012Bg:\nObserve that;2P (B) andB2P(B)."
            }
        ]
    },
    {
        "section": "Page 13",
        "content": [
            {
                "type": "text",
                "text": "1.2. Mathematical preliminaries 5\n8. IfAandBare two sets, then\n(a) their union is de\fned as\nA[B=fx:x2Aorx2Bg;\n(b) their intersection is de\fned as\nA\\B=fx:x2Aandx2Bg;\n(c) their di\u000berence is de\fned as\nAnB=fx:x2Aandx62Bg;\n(d) the Cartesian product ofAandBis de\fned as\nA\u0002B=f(x;y) :x2Aandy2Bg;\n(e) the complement ofAis de\fned as\nA=fx:x62Ag:\n9. A binary relation on two sets AandBis a subset of A\u0002B.\n10. A functionffromAtoB, denoted by f:A!B, is a binary relation\nR, having the property that for each element a2A, there is exactly\none ordered pair in R, whose \frst component is a. We will also say\nthatf(a) =b, orfmapsatob, or the image of aunderfisb. The\nsetAis called the domain off, and the set\nfb2B: there is an a2Awithf(a) =bg\nis called the range off.\n11. A function f:A!Bisone-to-one (orinjective ), if for any two distinct\nelementsaanda0inA, we havef(a)6=f(a0). The function fisonto\n(orsurjective ), if for each element b2B, there exists an element a2A,\nsuch thatf(a) =b; in other words, the range of fis equal to the set\nB. A function fis abijection , iffis both injective and surjective.\n12. A binary relation R\u0012A\u0002Ais an equivalence relation , if it satis\fes\nthe following three conditions:"
            }
        ]
    },
    {
        "section": "Page 14",
        "content": [
            {
                "type": "text",
                "text": "6 Chapter 1. Introduction\n(a)Risre\rexive : For every element in a2A, we have (a;a)2R.\n(b)Rissymmetric : For allaandbinA, if (a;b)2R, then also\n(b;a)2R.\n(c)Ristransitive : For alla,b, andcinA, if (a;b)2Rand (b;c)2R,\nthen also (a;c)2R.\n13. A graphG= (V;E) is a pair consisting of a set V, whose elements are\ncalled vertices , and a setE, where each element of Eis a pair of distinct\nvertices. The elements of Eare called edges . The \fgure below shows\nsome well-known graphs: K5(the complete graph on \fve vertices), K3;3\n(the complete bipartite graph on 2 \u00023 = 6 vertices), and the Peterson\ngraph.\nK5K3,3\nPeterson graph\nThe degree of a vertexv, denoted by deg(v), is de\fned to be the number\nof edges that are incident on v.\nApath in a graph is a sequence of vertices that are connected by edges.\nA path is a cycle , if it starts and ends at the same vertex. A simple\npath is a path without any repeated vertices. A graph is connected , if\nthere is a path between every pair of vertices.\n14. In the context of strings, an alphabet is a \fnite set, whose elements\nare called symbols . Examples of alphabets are \u0006 = f0;1gand \u0006 =\nfa;b;c;:::;zg.\n15. A string over an alphabet \u0006 is a \fnite sequence of symbols, where each\nsymbol is an element of \u0006. The length of a stringw, denoted byjwj, is\nthe number of symbols contained in w. The empty string , denoted by"
            }
        ]
    },
    {
        "section": "Page 15",
        "content": [
            {
                "type": "text",
                "text": "1.3. Proof techniques 7\n\u000f, is the string having length zero. For example, if the alphabet \u0006 is\nequal tof0;1g, then 10, 1000, 0, 101, and \u000fare strings over \u0006, having\nlengths 2, 4, 1, 3, and 0, respectively.\n16. A language is a set of strings.\n17. The Boolean values are 1 and 0, that represent true and false, respec-\ntively. The basic Boolean operations include\n(a) negation (or NOT ), represented by :,\n(b) conjunction (or AND ), represented by ^,\n(c) disjunction (or OR), represented by _,\n(d) exclusive-or (or XOR ), represented by \b,\n(e) equivalence, represented by $or,,\n(f) implication, represented by !or).\nThe following table explains the meanings of these operations.\nNOT AND OR XOR equivalence implication\n:0 = 1 0^0 = 0 0_0 = 0 0\b0 = 0 0$0 = 1 0!0 = 1\n:1 = 0 0^1 = 0 0_1 = 1 0\b1 = 1 0$1 = 0 0!1 = 1\n1^0 = 0 1_0 = 1 1\b0 = 1 1$0 = 0 1!0 = 0\n1^1 = 1 1_1 = 1 1\b1 = 0 1$1 = 1 1!1 = 1\n1.3 Proof techniques\nIn mathematics, a theorem is a statement that is true. A proof is a sequence\nof mathematical statements that form an argument to show that a theorem is\ntrue. The statements in the proof of a theorem include axioms (assumptions\nabout the underlying mathematical structures), hypotheses of the theorem\nto be proved, and previously proved theorems. The main question is \\How\ndo we go about proving theorems?\" This question is similar to the question\nof how to solve a given problem. Of course, the answer is that \fnding proofs,\nor solving problems, is not easy; otherwise life would be dull! There is no\nspeci\fed way of coming up with a proof, but there are some generic strategies\nthat could be of help. In this section, we review some of these strategies,\nthat will be su\u000ecient for this course. The best way to get a feeling of how\nto come up with a proof is by solving a large number of problems. Here are"
            }
        ]
    },
    {
        "section": "Page 16",
        "content": [
            {
                "type": "text",
                "text": "8 Chapter 1. Introduction\nsome useful tips. (You may take a look at the book How to Solve It , by G.\nP\u0013 olya).\n1. Read and completely understand the statement of the theorem to be\nproved. Most often this is the hardest part.\n2. Sometimes, theorems contain theorems inside them. For example,\n\\PropertyAif and only if property B\", requires showing two state-\nments:\n(a) If property Ais true, then property Bis true (A)B).\n(b) If property Bis true, then property Ais true (B)A).\nAnother example is the theorem \\Set Aequals setB.\" To prove this,\nwe need to prove that A\u0012BandB\u0012A. That is, we need to show\nthat each element of set Ais in setB, and that each element of set B\nis in setA.\n3. Try to work out a few simple cases of the theorem just to get a grip on\nit (i.e., crack a few simple cases \frst).\n4. Try to write down the proof once you have it. This is to ensure the\ncorrectness of your proof. Often, mistakes are found at the time of\nwriting.\n5. Finding proofs takes time, we do not come prewired to produce proofs.\nBe patient, think, express and write clearly and try to be precise as\nmuch as possible.\nIn the next sections, we will go through some of the proof strategies.\n1.3.1 Direct proofs\nAs the name suggests, in a direct proof of a theorem, we just approach the\ntheorem directly.\nTheorem 1.3.1 Ifnis an odd positive integer, then n2is odd as well."
            }
        ]
    },
    {
        "section": "Page 17",
        "content": [
            {
                "type": "text",
                "text": "1.3. Proof techniques 9\nProof. An odd positive integer ncan be written as n= 2k+ 1, for some\nintegerk\u00150. Then\nn2= (2k+ 1)2= 4k2+ 4k+ 1 = 2(2k2+ 2k) + 1:\nSince 2(2k2+ 2k) is even, and \\even plus one is odd\", we can conclude that\nn2is odd.\nTheorem 1.3.2 LetG= (V;E)be a graph. Then the sum of the degrees of\nall vertices is an even integer, i.e.,\nX\nv2Vdeg(v)\nis even.\nProof. If you do not see the meaning of this statement, then \frst try it out\nfor a few graphs. The reason why the statement holds is very simple: Each\nedge contributes 2 to the summation (because an edge is incident on exactly\ntwo distinct vertices).\nActually, the proof above proves the following theorem.\nTheorem 1.3.3 LetG= (V;E)be a graph. Then the sum of the degrees of\nall vertices is equal to twice the number of edges, i.e.,\nX\nv2Vdeg(v) = 2jEj:\n1.3.2 Constructive proofs\nThis technique not only shows the existence of a certain object, it actually\ngives a method of creating it. Here is how a constructive proof looks like:\nTheorem 1.3.4 There exists an object with property P.\nProof. Here is the object: [ :::]\nAnd here is the proof that the object satis\fes property P: [:::]\nHere is an example of a constructive proof. A graph is called 3- regular , if\neach vertex has degree three."
            }
        ]
    },
    {
        "section": "Page 18",
        "content": [
            {
                "type": "text",
                "text": "10 Chapter 1. Introduction\nTheorem 1.3.5 For every even integer n\u00154, there exists a 3-regular graph\nwithnvertices.\nProof. De\fne\nV=f0;1;2;:::;n\u00001g;\nand\nE=ffi;i+1g: 0\u0014i\u0014n\u00002g[ffn\u00001;0gg[ffi;i+n=2g: 0\u0014i\u0014n=2\u00001g:\nThen the graph G= (V;E) is 3-regular.\nConvince yourself that this graph is indeed 3-regular. It may help to draw\nthe graph for, say, n= 8.\n1.3.3 Nonconstructive proofs\nIn a nonconstructive proof, we show that a certain object exists, without\nactually creating it. Here is an example of such a proof:\nTheorem 1.3.6 There exist irrational numbers xandysuch thatxyis ra-\ntional.\nProof. There are two possible cases.\nCase 1:p\n2p\n22Q.\nIn this case, we take x=y=p\n2. In Theorem 1.3.9 below, we will prove\nthatp\n2 is irrational.\nCase 2:p\n2p\n262Q.\nIn this case, we take x=p\n2p\n2andy=p\n2. Since\nxy=\u0012p\n2p\n2\u0013p\n2\n=p\n22= 2;\nthe claim in the theorem follows.\nObserve that this proof indeed proves the theorem, but it does not give\nan example of a pair of irrational numbers xandysuch thatxyis rational."
            }
        ]
    },
    {
        "section": "Page 19",
        "content": [
            {
                "type": "text",
                "text": "1.3. Proof techniques 11\n1.3.4 Proofs by contradiction\nThis is how a proof by contradiction looks like:\nTheorem 1.3.7 StatementSis true.\nProof. Assume that statement Sis false. Then, derive a contradiction (such\nas 1 + 1 = 3).\nIn other words, show that the statement \\ :S) false\" is true. This is\nsu\u000ecient, because the contrapositive of the statement \\ :S) false\" is the\nstatement \\ true)S \". The latter logical formula is equivalent to S, and\nthat is what we wanted to show.\nBelow, we give two examples of proofs by contradiction.\nTheorem 1.3.8 Letnbe a positive integer. If n2is even, then nis even.\nProof. We will prove the theorem by contradiction. So we assume that n2\nis even, but nis odd. Since nis odd, we know from Theorem 1.3.1 that n2\nis odd. This is a contradiction, because we assumed that n2is even.\nTheorem 1.3.9p\n2is irrational, i.e.,p\n2cannot be written as a fraction of\ntwo integers mandn.\nProof. We will prove the theorem by contradiction. So we assume thatp\n2\nis rational. Thenp\n2 can be written as a fraction of two integers,p\n2 =m=n,\nwherem\u00151 andn\u00151. We may assume that mandndo not share any\ncommon factors, i.e., the greatest common divisor of mandnis equal to\none; if this is not the case, then we can get rid of the common factors. By\nsquaringp\n2 =m=n, we get 2n2=m2. This implies that m2is even. Then,\nby Theorem 1.3.8, mis even, which means that we can write masm= 2k,\nfor some positive integer k. It follows that 2 n2=m2= 4k2, which implies\nthatn2= 2k2. Hence,n2is even. Again by Theorem 1.3.8, it follows that n\nis even.\nWe have shown that mandnare both even. But we know that mand\nnarenotboth even. Hence, we have a contradiction. Our assumption thatp\n2 is rational is wrong. Thus, we can conclude thatp\n2 is irrational.\nThere is a nice discussion of this proof in the book My Brain is Open:\nThe Mathematical Journeys of Paul Erd} os by B. Schechter."
            }
        ]
    },
    {
        "section": "Page 20",
        "content": [
            {
                "type": "text",
                "text": "12 Chapter 1. Introduction\n1.3.5 The pigeon hole principle\nThis is a simple principle with surprising consequences.\nPigeon Hole Principle: Ifn+ 1 or more objects are placed into n\nboxes, then there is at least one box containing two or more objects.\nIn other words, if AandBare two sets such that jAj>jBj, then\nthere is no one-to-one function from AtoB.\nTheorem 1.3.10 Letnbe a positive integer. Every sequence of n2+ 1dis-\ntinct real numbers contains a subsequence of length n+ 1 that is either in-\ncreasing or decreasing.\nProof. For example consider the sequence (20 ;10;9;7;11;2;21;1;20;31) of\n10 = 32+ 1 numbers. This sequence contains an increasing subsequence of\nlength 4 = 3 + 1, namely (10 ;11;21;31).\nThe proof of this theorem is by contradiction, and uses the pigeon hole\nprinciple.\nLet (a1;a2;:::;an2+1) be an arbitrary sequence of n2+ 1 distinct real\nnumbers. For each iwith 1\u0014i\u0014n2+ 1, let incidenote the length of\nthe longest increasing subsequence that starts at ai, and let decidenote the\nlength of the longest decreasing subsequence that starts at ai.\nUsing this notation, the claim in the theorem can be formulated as follows:\nThere is an index isuch that inci\u0015n+ 1 or deci\u0015n+ 1.\nWe will prove the claim by contradiction. So we assume that inci\u0014n\nand deci\u0014nfor alliwith 1\u0014i\u0014n2+ 1.\nConsider the set\nB=f(b;c) : 1\u0014b\u0014n;1\u0014c\u0014ng;\nand think of the elements of Bas being boxes. For each iwith 1\u0014i\u0014n2+1,\nthe pair ( inci;deci) is an element of B. So we have n2+1 elements ( inci;deci),\nwhich are placed in the n2boxes ofB. By the pigeon hole principle, there\nmust be a box that contains two (or more) elements. In other words, there\nexist two integers iandjsuch thati<j and\n(inci;deci) = ( incj;decj):\nRecall that the elements in the sequence are distinct. Hence, ai6=aj. We\nconsider two cases."
            }
        ]
    },
    {
        "section": "Page 21",
        "content": [
            {
                "type": "text",
                "text": "1.3. Proof techniques 13\nFirst assume that ai< aj. Then the length of the longest increasing\nsubsequence starting at aimust be at least 1+ incj, because we can append ai\nto the longest increasing subsequence starting at aj. Therefore, inci6=incj,\nwhich is a contradiction.\nThe second case is when ai>aj. Then the length of the longest decreasing\nsubsequence starting at aimust be at least 1+ decj, because we can append ai\nto the longest decreasing subsequence starting at aj. Therefore, deci6=decj,\nwhich is again a contradiction.\n1.3.6 Proofs by induction\nThis is a very powerful and important technique for proving theorems.\nFor each positive integer n, letP(n) be a mathematical statement that\ndepends on n. Assume we wish to prove that P(n) is true for all positive\nintegersn. A proof by induction of such a statement is carried out as follows:\nBasis: Prove that P(1) is true.\nInduction step: Prove that for all n\u00151, the following holds: If P(n) is\ntrue, thenP(n+ 1) is also true.\nIn the induction step, we choose an arbitrary integer n\u00151 and assume\nthatP(n) is true; this is called the induction hypothesis . Then we prove that\nP(n+ 1) is also true.\nTheorem 1.3.11 For all positive integers n, we have\n1 + 2 + 3 + :::+n=n(n+ 1)\n2:\nProof. We start with the basis of the induction. If n= 1, then the left-hand\nside is equal to 1, and so is the right-hand side. So the theorem is true for\nn= 1.\nFor the induction step, let n\u00151 and assume that the theorem is true for\nn, i.e., assume that\n1 + 2 + 3 + :::+n=n(n+ 1)\n2:"
            }
        ]
    },
    {
        "section": "Page 22",
        "content": [
            {
                "type": "text",
                "text": "14 Chapter 1. Introduction\nWe have to prove that the theorem is true for n+ 1, i.e., we have to prove\nthat\n1 + 2 + 3 + :::+ (n+ 1) =(n+ 1)(n+ 2)\n2:\nHere is the proof:\n1 + 2 + 3 + :::+ (n+ 1) = 1 + 2 + 3 + :::+n|{z}\n=n(n+1)\n2+(n+ 1)\n=n(n+ 1)\n2+ (n+ 1)\n=(n+ 1)(n+ 2)\n2:\nBy the way, here is an alternative proof of the theorem above: Let S=\n1 + 2 + 3 + :::+n. Then,\nS = 1 + 2 + 3 + : : : + ( n\u00002) + ( n\u00001) + n\nS = n + ( n\u00001) + ( n\u00002) + : : : + 3 + 2 + 1\n2S = ( n+ 1) + ( n+ 1) + ( n+ 1) + : : : + ( n+ 1) + ( n+ 1) + ( n+ 1)\nSince there are nterms on the right-hand side, we have 2 S=n(n+ 1). This\nimplies that S=n(n+ 1)=2.\nTheorem 1.3.12 For every positive integer n,a\u0000bis a factor of an\u0000bn.\nProof. A direct proof can be given by providing a factorization of an\u0000bn:\nan\u0000bn= (a\u0000b)(an\u00001+an\u00002b+an\u00003b2+:::+abn\u00002+bn\u00001):\nWe now prove the theorem by induction. For the basis, let n= 1. The claim\nin the theorem is \\ a\u0000bis a factor of a\u0000b\", which is obviously true.\nLetn\u00151 and assume that a\u0000bis a factor of an\u0000bn. We have to prove\nthata\u0000bis a factor of an+1\u0000bn+1. We have\nan+1\u0000bn+1=an+1\u0000anb+anb\u0000bn+1=an(a\u0000b) + (an\u0000bn)b:\nThe \frst term on the right-hand side is divisible by a\u0000b. By the induction\nhypothesis, the second term on the right-hand side is divisible by a\u0000bas\nwell. Therefore, the entire right-hand side is divisible by a\u0000b. Since the\nright-hand side is equal to an+1\u0000bn+1, it follows that a\u0000bis a factor of\nan+1\u0000bn+1.\nWe now give an alternative proof of Theorem 1.3.3:"
            }
        ]
    },
    {
        "section": "Page 23",
        "content": [
            {
                "type": "text",
                "text": "1.3. Proof techniques 15\nTheorem 1.3.13 LetG= (V;E)be a graph with medges. Then the sum\nof the degrees of all vertices is equal to twice the number of edges, i.e.,\nX\nv2Vdeg(v) = 2m:\nProof. The proof is by induction on the number mof edges. For the basis of\nthe induction, assume that m= 0. Then the graph Gdoes not contain any\nedges and, therefore,P\nv2Vdeg(v) = 0. Thus, the theorem is true if m= 0.\nLetm\u00150 and assume that the theorem is true for every graph with m\nedges. LetGbe an arbitrary graph with m+ 1 edges. We have to prove thatP\nv2Vdeg(v) = 2(m+ 1).\nLetfa;bgbe an arbitrary edge in G, and letG0be the graph obtained\nfromGby removing the edge fa;bg. SinceG0hasmedges, we know from\nthe induction hypothesis that the sum of the degrees of all vertices in G0is\nequal to 2m. Using this, we obtain\nX\nv2Gdeg(v) =X\nv2G0deg(v) + 2 = 2m+ 2 = 2(m+ 1):\n1.3.7 More examples of proofs\nRecall Theorem 1.3.5, which states that for every even integern\u00154, there\nexists a 3-regular graph with nvertices. The following theorem explains why\nwe stated this theorem for even values of n.\nTheorem 1.3.14 Letn\u00155be an odd integer. There is no 3-regular graph\nwithnvertices.\nProof. The proof is by contradiction. So we assume that there exists a\ngraphG= (V;E) withnvertices that is 3-regular. Let mbe the number of\nedges inG. Since deg(v) = 3 for every vertex, we have\nX\nv2Vdeg(v) = 3n:\nOn the other hand, by Theorem 1.3.3, we have\nX\nv2Vdeg(v) = 2m:"
            }
        ]
    },
    {
        "section": "Page 24",
        "content": [
            {
                "type": "text",
                "text": "16 Chapter 1. Introduction\nIt follows that 3 n= 2m, which can be rewritten as m= 3n=2. Sincemis an\ninteger, and since gcd(2;3) = 1,n=2 must be an integer. Hence, nis even,\nwhich is a contradiction.\nLetKnbe the complete graph onnvertices. This graph has a vertex set\nof sizen, and every pair of distinct vertices is joined by an edge.\nIfG= (V;E) is a graph with nvertices, then the complement GofGis\nthe graph with vertex set Vthat consists of those edges of Knthat are not\npresent inG.\nTheorem 1.3.15 Letn\u00152and letGbe a graph on nvertices. Then Gis\nconnected or Gis connected.\nProof. We prove the theorem by induction on the number nof vertices. For\nthe basis, assume that n= 2. There are two possibilities for the graph G:\n1.Gcontains one edge. In this case, Gis connected.\n2.Gdoes not contain an edge. In this case, the complement Gcontains\none edge and, therefore, Gis connected.\nSo forn= 2, the theorem is true.\nLetn\u00152 and assume that the theorem is true for every graph with n\nvertices. Let Gbe graph with n+ 1 vertices. We have to prove that Gis\nconnected or Gis connected. We consider three cases.\nCase 1: There is a vertex vwhose degree in Gis equal ton.\nSinceGhasn+1 vertices, vis connected by an edge to every other vertex\nofG. Therefore, Gis connected.\nCase 2: There is a vertex vwhose degree in Gis equal to 0.\nIn this case, the degree of vin the graph Gis equal ton. SinceGhasn+1\nvertices,vis connected by an edge to every other vertex of G. Therefore, G\nis connected.\nCase 3: For every vertex v, the degree of vinGis inf1;2;:::;n\u00001g.\nLetvbe an arbitrary vertex of G. LetG0be the graph obtained by\ndeleting from Gthe vertexv, together with all edges that are incident on v.\nSinceG0hasnvertices, we know from the induction hypothesis that G0is\nconnected or G0is connected."
            }
        ]
    },
    {
        "section": "Page 25",
        "content": [
            {
                "type": "text",
                "text": "1.3. Proof techniques 17\nLet us \frst assume that G0is connected. Then the graph Gis connected\nas well, because there is at least one edge in Gbetweenvand some vertex\nofG0.\nIfG0is not connected, then G0must be connected. Since we are in Case 3,\nwe know that the degree of vinGis in the setf1;2;:::;n\u00001g. It follows\nthat the degree of vin the graph Gis in this set as well. Hence, there is at\nleast one edge in Gbetweenvand some vertex in G0. This implies that Gis\nconnected.\nThe previous theorem can be rephrased as follows:\nTheorem 1.3.16 Letn\u00152and consider the complete graph Knonnver-\ntices. Color each edge of this graph as either red or blue. Let Rbe the graph\nconsisting of all the red edges, and let Bbe the graph consisting of all the\nblue edges. Then Ris connected or Bis connected.\nA graph is said to be planar , if it can be drawn (a better term is \\embed-\nded\") in the plane in such a way that no two edges intersect, except possibly\nat their endpoints. An embedding of a planar graph consists of vertices,\nedges, and faces. In the example below, there are 11 vertices, 18 edges, and\n9 faces (including the unbounded face).\nThe following theorem is known as Euler's theorem for planar graphs .\nApparently, this theorem was discovered by Euler around 1750. Legendre\ngave the \frst proof in 1794, see\nhttp://www.ics.uci.edu/~eppstein/junkyard/euler/\nTheorem 1.3.17 (Euler) Consider an embedding of a planar graph G. Let\nv,e, andfbe the number of vertices, edges, and faces (including the single"
            }
        ]
    },
    {
        "section": "Page 26",
        "content": [
            {
                "type": "text",
                "text": "18 Chapter 1. Introduction\nunbounded face) of this embedding, respectively. Moreover, let cbe the number\nof connected components of G. Then\nv\u0000e+f=c+ 1:\nProof. The proof is by induction on the number of edges of G. To be more\nprecise, we start with a graph having no edges, and prove that the theorem\nholds for this case. Then, we add the edges one by one, and show that the\nrelationv\u0000e+f=c+ 1 is maintained.\nSo we \frst assume that Ghas no edges, i.e., e= 0. Then the embedding\nconsists of a collection of vpoints. In this case, we have f= 1 andc=v.\nHence, the relation v\u0000e+f=c+ 1 holds.\nLete > 0 and assume that Euler's formula holds for a subgraph of G\nhavinge\u00001 edges. Letfu;vgbe an edge of Gthat is not in the subgraph,\nand add this edge to the subgraph. There are two cases depending on whether\nthis new edge joins two connected components or joins two vertices in the\nsame connected component.\nCase 1: The new edgefu;vgjoins two connected components.\nIn this case, the number of vertices and the number of faces do not change,\nthe number of connected components goes down by 1, and the number of\nedges increases by 1. It follows that the relation in the theorem is still valid.\nCase 2: The new edgefu;vgjoins two vertices in the same connected com-\nponent.\nIn this case, the number of vertices and the number of connected com-\nponents do not change, the number of edges increases by 1, and the number\nof faces increases by 1 (because the new edge splits one face into two faces).\nTherefore, the relation in the theorem is still valid.\nEuler's theorem is usually stated as follows:\nTheorem 1.3.18 (Euler) Consider an embedding of a connected planar\ngraphG. Letv,e, andfbe the number of vertices, edges, and faces (in-\ncluding the single unbounded face) of this embedding, respectively. Then\nv\u0000e+f= 2:\nIf you like surprising proofs of various mathematical results, you should\nread the book Proofs from THE BOOK by Aigner and Ziegler."
            }
        ]
    },
    {
        "section": "Page 27",
        "content": [
            {
                "type": "text",
                "text": "Exercises 19\nExercises\n1.1Use induction to prove that every integer n\u00152 can be written as a\nproduct of prime numbers.\n1.2For every prime number p, prove thatppis irrational.\n1.3Letnbe a positive integer that is not a perfect square. Prove thatpn\nis irrational.\n1.4Prove by induction that n4\u00004n2is divisible by 3, for all integers n\u00151.\n1.5Prove thatnX\ni=11\ni2<2\u00001=n;\nfor every integer n\u00152.\n1.6Prove that 9 divides n3+ (n+ 1)3+ (n+ 2)3, for every integer n\u00150.\n1.7Prove that in any set of n+ 1 numbers from f1;2;:::; 2ng, there are\nalways two numbers that are consecutive.\n1.8Prove that in any set of n+ 1 numbers from f1;2;:::; 2ng, there are\nalways two numbers such that one divides the other."
            }
        ]
    },
    {
        "section": "Page 28",
        "content": [
            {
                "type": "text",
                "text": "20 Chapter 1. Introduction"
            }
        ]
    },
    {
        "section": "Page 29",
        "content": [
            {
                "type": "text",
                "text": "Chapter 2\nFinite Automata and Regular\nLanguages\nIn this chapter, we introduce and analyze the class of languages that are\nknown as regular languages . Informally, these languages can be \\processed\"\nby computers having a very small amount of memory.\n2.1 An example: Controling a toll gate\nBefore we give a formal de\fnition of a \fnite automaton, we consider an\nexample in which such an automaton shows up in a natural way. We consider\nthe problem of designing a \\computer\" that controls a toll gate .\nWhen a car arrives at the toll gate, the gate is closed. The gate opens as\nsoon as the driver has payed 25 cents. We assume that we have only three\ncoin denominations: 5, 10, and 25 cents. We also assume that no excess\nchange is returned.\nAfter having arrived at the toll gate, the driver inserts a sequence of coins\ninto the machine. At any moment, the machine has to decide whether or not\nto open the gate, i.e., whether or not the driver has paid 25 cents (or more).\nIn order to decide this, the machine is in one of the following six states , at\nany moment during the process:\n\u000fThe machine is in state q0, if it has not collected any money yet.\n\u000fThe machine is in state q1, if it has collected exactly 5 cents.\n\u000fThe machine is in state q2, if it has collected exactly 10 cents."
            }
        ]
    },
    {
        "section": "Page 30",
        "content": [
            {
                "type": "text",
                "text": "22 Chapter 2. Finite Automata and Regular Languages\n\u000fThe machine is in state q3, if it has collected exactly 15 cents.\n\u000fThe machine is in state q4, if it has collected exactly 20 cents.\n\u000fThe machine is in state q5, if it has collected 25 cents or more.\nInitially (when a car arrives at the toll gate), the machine is in state q0.\nAssume, for example, that the driver presents the sequence (10,5,5,10) of\ncoins.\n\u000fAfter receiving the \frst 10 cents coin, the machine switches from state\nq0to stateq2.\n\u000fAfter receiving the \frst 5 cents coin, the machine switches from state\nq2to stateq3.\n\u000fAfter receiving the second 5 cents coin, the machine switches from state\nq3to stateq4.\n\u000fAfter receiving the second 10 cents coin, the machine switches from\nstateq4to stateq5. At this moment, the gate opens. (Remember that\nno change is given.)\nThe \fgure below represents the behavior of the machine for all possible\nsequences of coins. State q5is represented by two circles, because it is a\nspecial state: As soon as the machine reaches this state, the gate opens.\nq0 q1 q2 q3 q4 q55 5 5 510 10 10\n25\n252510,255,10,25\n5,10\n25start\nObserve that the machine (or computer) only has to remember which\nstate it is in at any given time. Thus, it needs only a very small amount\nof memory: It has to be able to distinguish between any one of six possible\ncases and, therefore, it only needs a memory of dlog 6e= 3 bits."
            }
        ]
    },
    {
        "section": "Page 31",
        "content": [
            {
                "type": "text",
                "text": "2.2. Deterministic \fnite automata 23\n2.2 Deterministic \fnite automata\nLet us look at another example. Consider the following state diagram :\nq1 q2 q30\n0\n11\n0,1\nWe say that q1is the start state and q2is an accept state. Consider the\ninput string 1101. This string is processed in the following way:\n\u000fInitially, the machine is in the start state q1.\n\u000fAfter having read the \frst 1, the machine switches from state q1to\nstateq2.\n\u000fAfter having read the second 1, the machine switches from state q2to\nstateq2. (So actually, it does not switch.)\n\u000fAfter having read the \frst 0, the machine switches from state q2to\nstateq3.\n\u000fAfter having read the third 1, the machine switches from state q3to\nstateq2.\nAfter the entire string 1101 has been processed, the machine is in state q2,\nwhich is an accept state. We say that the string 1101 is accepted by the\nmachine.\nConsider now the input string 0101010. After having read this string\nfrom left to right (starting in the start state q1), the machine is in state q3.\nSinceq3is not an accept state, we say that the machine rejects the string\n0101010.\nWe hope you are able to see that this machine accepts every binary string\nthat ends with a 1. In fact, the machine accepts more strings:\n\u000fEvery binary string having the property that there are an even number\nof 0s following the rightmost 1, is accepted by this machine."
            }
        ]
    },
    {
        "section": "Page 32",
        "content": [
            {
                "type": "text",
                "text": "24 Chapter 2. Finite Automata and Regular Languages\n\u000fEvery other binary string is rejected by the machine. Observe that each\nsuch string is either empty, consists of 0s only, or has an odd number\nof 0s following the rightmost 1.\nWe now come to the formal de\fnition of a \fnite automaton:\nDe\fnition 2.2.1 A\fnite automaton is a 5-tuple M= (Q;\u0006;\u000e;q;F ), where\n1.Qis a \fnite set, whose elements are called states ,\n2. \u0006 is a \fnite set, called the alphabet ; the elements of \u0006 are called symbols ,\n3.\u000e:Q\u0002\u0006!Qis a function, called the transition function ,\n4.qis an element of Q; it is called the start state ,\n5.Fis a subset of Q; the elements of Fare called accept states .\nYou can think of the transition function \u000eas being the \\program\" of the\n\fnite automaton M= (Q;\u0006;\u000e;q;F ). This function tells us what Mcan do\nin \\one step\":\n\u000fLetrbe a state of Qand letabe a symbol of the alphabet \u0006. If\nthe \fnite automaton Mis in staterand reads the symbol a, then it\nswitches from state rto state\u000e(r;a). (In fact, \u000e(r;a) may be equal to\nr.)\nThe \\computer\" that we designed in the toll gate example in Section 2.1\nis a \fnite automaton. For this example, we have Q=fq0;q1;q2;q3;q4;q5g,\n\u0006 =f5;10;25g, the start state is q0,F=fq5g, and\u000eis given by the following\ntable:\n5 10 25\nq0q1q2q5\nq1q2q3q5\nq2q3q4q5\nq3q4q5q5\nq4q5q5q5\nq5q5q5q5\nThe example given in the beginning of this section is also a \fnite automa-\nton. For this example, we have Q=fq1;q2;q3g, \u0006 =f0;1g, the start state\nisq1,F=fq2g, and\u000eis given by the following table:"
            }
        ]
    },
    {
        "section": "Page 33",
        "content": [
            {
                "type": "text",
                "text": "2.2. Deterministic \fnite automata 25\n0 1\nq1q1q2\nq2q3q2\nq3q2q2\nLet us denote this \fnite automaton by M. The language of M, denoted\nbyL(M), is the set of all binary strings that are accepted by M. As we have\nseen before, we have\nL(M) =fw:wcontains at least one 1 and ends with an even number of 0s g:\nWe now give a formal de\fnition of the language of a \fnite automaton:\nDe\fnition 2.2.2 LetM= (Q;\u0006;\u000e;q;F ) be a \fnite automaton and let w=\nw1w2:::wnbe a string over \u0006. De\fne the sequence r0;r1;:::;rnof states, in\nthe following way:\n\u000fr0=q,\n\u000fri+1=\u000e(ri;wi+1), fori= 0;1;:::;n\u00001.\n1. Ifrn2F, then we say that Macceptsw.\n2. Ifrn62F, then we say that Mrejectsw.\nIn this de\fnition, wmay be the empty string , which we denote by \u000f, and\nwhose length is zero; thus in the de\fnition above, n= 0. In this case, the\nsequencer0;r1;:::;rnof states has length one; it consists of just the state\nr0=q. The empty string is accepted by Mif and only if the start state q\nbelongs toF.\nDe\fnition 2.2.3 LetM= (Q;\u0006;\u000e;q;F ) be a \fnite automaton. The lan-\nguageL(M)accepted byMis de\fned to be the set of all strings that are\naccepted by M:\nL(M) =fw:wis a string over \u0006 and Macceptswg:\nDe\fnition 2.2.4 A language Ais called regular , if there exists a \fnite au-\ntomatonMsuch thatA=L(M)."
            }
        ]
    },
    {
        "section": "Page 34",
        "content": [
            {
                "type": "text",
                "text": "26 Chapter 2. Finite Automata and Regular Languages\nWe \fnish this section by presenting an equivalent way of de\fning the\nlanguage accepted by a \fnite automaton. Let M= (Q;\u0006;\u000e;q;F ) be a \fnite\nautomaton. The transition function \u000e:Q\u0002\u0006!Qtells us that, when M\nis in stater2Qand reads symbol a2\u0006, it switches from state rto state\n\u000e(r;a). Let \u0006\u0003denote the set of all strings over the alphabet \u0006. (\u0006\u0003includes\nthe empty string \u000f.) We extend the function \u000eto a function\n\u000e:Q\u0002\u0006\u0003!Q;\nthat is de\fned as follows. For any state r2Qand for any string wover the\nalphabet \u0006,\n\u000e(r;w) =\u001ar ifw=\u000f,\n\u000e(\u000e(r;v);a) ifw=va, wherevis a string and a2\u0006.\nWhat is the meaning of this function \u000e? Letrbe a state of Qand letwbe\na string over the alphabet \u0006. Then\n\u000f\u000e(r;w) is the state that Mreaches, when it starts in state r, reads the\nstringwfrom left to right, and uses \u000eto switch from state to state.\nUsing this notation, we have\nL(M) =fw:wis a string over \u0006 and \u000e(q;w)2Fg:\n2.2.1 A \frst example of a \fnite automaton\nLet\nA=fw:wis a binary string containing an odd number of 1s g:\nWe claim that this language Ais regular. In order to prove this, we have to\nconstruct a \fnite automaton Msuch thatA=L(M).\nHow to construct M? Here is a \frst idea: The \fnite automaton reads the\ninput string wfrom left to right and keeps track of the number of 1s it has\nseen. After having read the entire string w, it checks whether this number\nis odd (in which case wis accepted) or even (in which case wis rejected).\nUsing this approach, the \fnite automaton needs a state for every integer\ni\u00150, indicating that the number of 1s read so far is equal to i. Hence,\nto design a \fnite automaton that follows this approach, we need an in\fnite"
            }
        ]
    },
    {
        "section": "Page 35",
        "content": [
            {
                "type": "text",
                "text": "2.2. Deterministic \fnite automata 27\nnumber of states. But, the de\fnition of \fnite automaton requires the number\nof states to be \fnite .\nA better, and correct approach, is to keep track of whether the number\nof 1s read so far is even or odd. This leads to the following \fnite automaton:\n\u000fThe set of states is Q=fqe;qog. If the \fnite automaton is in state qe,\nthen it has read an even number of 1s; if it is in state qo, then it has\nread an odd number of 1s.\n\u000fThe alphabet is \u0006 = f0;1g.\n\u000fThe start state is qe, because at the start, the number of 1s read by the\nautomaton is equal to 0, and 0 is even.\n\u000fThe setFof accept states is F=fqog.\n\u000fThe transition function \u000eis given by the following table:\n0 1\nqeqeqo\nqoqoqe\nThis \fnite automaton M= (Q;\u0006;\u000e;qe;F) can also be described by its state\ndiagram , which is given in the \fgure below. The arrow that comes \\out of\nthe blue\" and enters the state qe, indicates that qeis the start state. The\nstate depicted with double circles indicates the accept state.\nqe qo0\n01\n1\nWe have constructed a \fnite automaton Mthat accepts the language A.\nTherefore,Ais a regular language."
            }
        ]
    },
    {
        "section": "Page 36",
        "content": [
            {
                "type": "text",
                "text": "28 Chapter 2. Finite Automata and Regular Languages\n2.2.2 A second example of a \fnite automaton\nDe\fne the language Aas\nA=fw:wis a binary string containing 101 as a substring g:\nAgain, we claim that Ais a regular language. In other words, we claim that\nthere exists a \fnite automaton Mthat accepts A, i.e.,A=L(M).\nThe \fnite automaton Mwill do the following, when reading an input\nstring from left to right:\n\u000fIt skips over all 0s, and stays in the start state.\n\u000fAt the \frst 1, it switches to the state \\maybe the next two symbols are\n01\".\n{If the next symbol is 1, then it stays in the state \\maybe the next\ntwo symbols are 01\".\n{On the other hand, if the next symbol is 0, then it switches to the\nstate \\maybe the next symbol is 1\".\n\u0003If the next symbol is indeed 1, then it switches to the accept\nstate (but keeps on reading until the end of the string).\n\u0003On the other hand, if the next symbol is 0, then it switches\nto the start state, and skips 0s until it reads 1 again.\nBy de\fning the following four states, this process will become clear:\n\u000fq1:Mis in this state if the last symbol read was 1, but the substring\n101 has not been read.\n\u000fq10:Mis in this state if the last two symbols read were 10, but the\nsubstring 101 has not been read.\n\u000fq101:Mis in this state if the substring 101 has been read in the input\nstring.\n\u000fq: In all other cases, Mis in this state.\nHere is the formal description of the \fnite automaton that accepts the\nlanguageA:\n\u000fQ=fq;q1;q10;q101g,"
            }
        ]
    },
    {
        "section": "Page 37",
        "content": [
            {
                "type": "text",
                "text": "2.2. Deterministic \fnite automata 29\n\u000f\u0006 =f0;1g,\n\u000fthe start state is q,\n\u000fthe setFof accept states is equal to F=fq101g, and\n\u000fthe transition function \u000eis given by the following table:\n0 1\nqq q 1\nq1q10q1\nq10q q 101\nq101q101q101\nThe \fgure below gives the state diagram of the \fnite automaton M=\n(Q;\u0006;\u000e;q;F ).\nq q 1\nq10 q1010\n11\n00\n10,1\nThis \fnite automaton accepts the language Aconsisting of all binary\nstrings that contain the substring 101. As an exercise, how would you obtain\na \fnite automaton that accepts the complement of A, i.e., the language\nconsisting of all binary strings that do not contain the substring 101?\n2.2.3 A third example of a \fnite automaton\nThe \fnite automata we have seen so far have exactly one accept state. In\nthis section, we will see an example of a \fnite automaton having more accept\nstates."
            }
        ]
    },
    {
        "section": "Page 38",
        "content": [
            {
                "type": "text",
                "text": "30 Chapter 2. Finite Automata and Regular Languages\nLetAbe the language\nA=fw2f0;1g\u0003:whas a 1 in the third position from the right g;\nwheref0;1g\u0003is the set of all binary strings, including the empty string \u000f. We\nclaim thatAis a regular language. To prove this, we have to construct a \fnite\nautomaton Msuch thatA=L(M). At \frst sight, it seems di\u000ecult (or even\nimpossible?) to construct such a \fnite automaton: How does the automaton\n\\know\" that it has reached the third symbol from the right? It is, however,\npossible to construct such an automaton. The main idea is to remember the\nlast three symbols that have been read. Thus, the \fnite automaton has eight\nstatesqijk, wherei,j, andkrange over the two elements of f0;1g. If the\nautomaton is in state qijk, then the following hold:\n\u000fIfMhas read at least three symbols, then the three most recently read\nsymbols are ijk.\n\u000fIfMhas read only two symbols, then these two symbols are jk; more-\nover,i= 0.\n\u000fIfMhas read only one symbol, then this symbol is k; moreover, i=\nj= 0.\n\u000fIfMhas not read any symbol, then i=j=k= 0.\nThe start state is q000and the set of accept states is fq100;q110;q101;q111g.\nThe transition function of Mis given by the following state diagram.\nq000 q100 q010 q110\nq001 q101 q011 q1110\n10\n10\n10\n1\n0\n110\n10\n10"
            }
        ]
    },
    {
        "section": "Page 39",
        "content": [
            {
                "type": "text",
                "text": "2.3. Regular operations 31\n2.3 Regular operations\nIn this section, we de\fne three operations on languages. Later, we will answer\nthe question whether the set of all regular languages is closed under these\noperations. Let AandBbe two languages over the same alphabet.\n1. The union ofAandBis de\fned as\nA[B=fw:w2Aorw2Bg:\n2. The concatenation ofAandBis de\fned as\nAB=fww0:w2Aandw02Bg:\nIn words,ABis the set of all strings obtained by taking an arbitrary\nstringwinAand an arbitrary string w0inB, and gluing them together\n(such thatwis to the left of w0).\n3. The star ofAis de\fned as\nA\u0003=fu1u2:::uk:k\u00150 andui2Afor alli= 1;2;:::;kg:\nIn words,A\u0003is obtained by taking any \fnite number of strings in A, and\ngluing them together. Observe that k= 0 is allowed; this corresponds\nto the empty string \u000f. Thus,\u000f2A\u0003.\nTo give an example, let A=f0;01gandB=f1;10g. Then\nA[B=f0;01;1;10g;\nAB=f01;010;011;0110g;\nand\nA\u0003=f\u000f;0;01;00;001;010;0101;000;0001;00101;:::g:\nAs another example, if \u0006 = f0;1g, then \u0006\u0003is the set of all binary strings\n(including the empty string). Observe that a string always has a \fnite length.\nBefore we proceed, we give an alternative (and equivalent) de\fnition of\nthe star of the language A: De\fne\nA0=f\u000fg"
            }
        ]
    },
    {
        "section": "Page 40",
        "content": [
            {
                "type": "text",
                "text": "32 Chapter 2. Finite Automata and Regular Languages\nand, fork\u00151,\nAk=AAk\u00001;\ni.e.,Akis the concatenation of the two languages AandAk\u00001. Then we have\nA\u0003=1[\nk=0Ak:\nTheorem 2.3.1 The set of regular languages is closed under the union op-\neration, i.e., if AandBare regular languages over the same alphabet \u0006, then\nA[Bis also a regular language.\nProof. SinceAandBare regular languages, there are \fnite automata\nM1= (Q1;\u0006;\u000e1;q1;F1) andM2= (Q2;\u0006;\u000e2;q2;F2) that accept AandB,\nrespectively. In order to prove that A[Bis regular, we have to construct a\n\fnite automaton Mthat accepts A[B. In other words, Mmust have the\nproperty that for every string w2\u0006\u0003,\nMacceptsw,M1acceptsworM2acceptsw.\nAs a \frst idea, we may think that Mcould do the following:\n\u000fStarting in the start state q1ofM1,M\\runs\"M1onw.\n\u000fIf, after having read w,M1is in a state of F1, thenw2A, thus\nw2A[Band, therefore, Macceptsw.\n\u000fOn the other hand, if, after having read w,M1is in a state that is not\ninF1, thenw62AandM\\runs\"M2onw, starting in the start state\nq2ofM2. If, after having read w,M2is in a state of F2, then we know\nthatw2B, thusw2A[Band, therefore, Macceptsw. Otherwise,\nwe know that w62A[B, andMrejectsw.\nThis idea does not work, because the \fnite automaton Mcan read the input\nstringwonly once . The correct approach is to runM1andM2simulta-\nneously . We de\fne the set Qof states of Mto be the Cartesian product\nQ1\u0002Q2. IfMis in state ( r1;r2), this means that\n\u000fifM1would have read the input string up to this point, then it would\nbe in state r1, and"
            }
        ]
    },
    {
        "section": "Page 41",
        "content": [
            {
                "type": "text",
                "text": "2.3. Regular operations 33\n\u000fifM2would have read the input string up to this point, then it would\nbe in state r2.\nThis leads to the \fnite automaton M= (Q;\u0006;\u000e;q;F ), where\n\u000fQ=Q1\u0002Q2=f(r1;r2) :r12Q1andr22Q2g. Observe that\njQj=jQ1j\u0002jQ2j, which is \fnite.\n\u000f\u0006 is the alphabet of AandB(recall that we assume that AandBare\nlanguages over the same alphabet).\n\u000fThe start state qofMis equal toq= (q1;q2).\n\u000fThe setFof accept states of Mis given by\nF=f(r1;r2) :r12F1orr22F2g= (F1\u0002Q2)[(Q1\u0002F2):\n\u000fThe transition function \u000e:Q\u0002\u0006!Qis given by\n\u000e((r1;r2);a) = (\u000e1(r1;a);\u000e2(r2;a));\nfor allr12Q1,r22Q2, anda2\u0006.\nTo \fnish the proof, we have to show that this \fnite automaton Mindeed\naccepts the language A[B. Intuitively, this should be clear from the discus-\nsion above. The easiest way to give a formal proof is by using the extended\ntransition functions \u000e1and\u000e2. (The extended transition function has been\nde\fned after De\fnition 2.2.4.) Here we go: Recall that we have to prove that\nMacceptsw,M1acceptsworM2acceptsw,\ni.e,\nMacceptsw,\u000e1(q1;w)2F1or\u000e2(q2;w)2F2.\nIn terms of the extended transition function \u000eof the transition function \u000eof\nM, this becomes\n\u000e((q1;q2);w)2F,\u000e1(q1;w)2F1or\u000e2(q2;w)2F2. (2.1)\nBy applying the de\fnition of the extended transition function, as given after\nDe\fnition 2.2.4, to \u000e, it can be seen that\n\u000e((q1;q2);w) = (\u000e1(q1;w);\u000e2(q2;w)):"
            }
        ]
    },
    {
        "section": "Page 42",
        "content": [
            {
                "type": "text",
                "text": "34 Chapter 2. Finite Automata and Regular Languages\nThe latter equality implies that (2.1) is true and, therefore, Mindeed accepts\nthe language A[B.\nWhat about the closure of the regular languages under the concatenation\nand star operations? It turns out that the regular languages are closed under\nthese operations. But how do we prove this?\nLetAandBbe two regular languages, and let M1andM2be \fnite\nautomata that accept AandB, respectively. How do we construct a \fnite\nautomaton Mthat accepts the concatenation AB? Given an input string\nu,Mhas to decide whether or not ucan be broken into two strings wand\nw0(i.e., write uasu=ww0), such that w2Aandw02B. In words, M\nhas to decide whether or not ucan be broken into two substrings, such that\nthe \frst substring is accepted by M1and the second substring is accepted by\nM2. The di\u000eculty is caused by the fact that Mhas to make this decision by\nscanning the string uonly once. If u2AB, thenMhas to decide, during\nthis single scan , where to break uinto two substrings. Similarly, if u62AB,\nthenMhas to decide, during this single scan , thatucannot be broken into\ntwo substrings such that the \frst substring is in Aand the second substring\nis inB.\nIt seems to be even more di\u000ecult to prove that A\u0003is a regular language,\nifAitself is regular. In order to prove this, we need a \fnite automaton that,\nwhen given an arbitrary input string u, decides whether or not ucan be\nbroken into substrings such that each substring is in A. The problem is that,\nifu2A\u0003, the \fnite automaton has to determine into how many substrings,\nand where, the string uhas to be broken; it has to do this during one single\nscan of the string u.\nAs we mentioned already, if AandBare regular languages, then both\nABandA\u0003are also regular. In order to prove these claims, we will introduce\na more general type of \fnite automaton.\nThe \fnite automata that we have seen so far are deterministic . This\nmeans the following:\n\u000fIf the \fnite automaton Mis in staterand if it reads the symbol a,\nthenMswitches from state rto the uniquely de\fned state \u000e(r;a).\nFrom now on, we will call such a \fnite automaton a deterministic \fnite\nautomaton (DFA) . In the next section, we will de\fne the notion of a nonde-\nterministic \fnite automaton (NFA) . For such an automaton, there are zero\nor more possible states to switch to. At \frst sight, nondeterministic \fnite"
            }
        ]
    },
    {
        "section": "Page 43",
        "content": [
            {
                "type": "text",
                "text": "2.4. Nondeterministic \fnite automata 35\nautomata seem to be more powerful than their deterministic counterparts.\nWe will prove, however, that DFAs have the same power as NFAs. As we will\nsee, using this fact, it will be easy to prove that the class of regular languages\nis closed under the concatenation and star operations.\n2.4 Nondeterministic \fnite automata\nWe start by giving three examples of nondeterministic \fnite automata. These\nexamples will show the di\u000berence between this type of automata and the\ndeterministic versions that we have considered in the previous sections. After\nthese examples, we will give a formal de\fnition of a nondeterministic \fnite\nautomaton.\n2.4.1 A \frst example\nConsider the following state diagram:\nq1 q2 q3 q40,1\n1 0,ε 10,1\nYou will notice three di\u000berences with the \fnite automata that we have\nseen until now. First, if the automaton is in state q1and reads the symbol 1,\nthen it has two options: Either it stays in state q1, or it switches to state q2.\nSecond, if the automaton is in state q2, then it can switch to state q3without\nreading a symbol ; this is indicated by the edge having the empty string \u000fas\nlabel. Third, if the automaton is in state q3and reads the symbol 0, then it\ncannot continue.\nLet us see what this automaton can do when it gets the string 010110 as\ninput. Initially, the automaton is in the start state q1.\n\u000fSince the \frst symbol in the input string is 0, the automaton stays in\nstateq1after having read this symbol.\n\u000fThe second symbol is 1, and the automaton can either stay in state q1\nor switch to state q2."
            }
        ]
    },
    {
        "section": "Page 44",
        "content": [
            {
                "type": "text",
                "text": "36 Chapter 2. Finite Automata and Regular Languages\n{If the automaton stays in state q1, then it is still in this state after\nhaving read the third symbol.\n{If the automaton switches to state q2, then it again has two op-\ntions:\n\u0003Either read the third symbol in the input string, which is 0,\nand switch to state q3,\n\u0003or switch to state q3, without reading the third symbol.\nIf we continue in this way, then we see that, for the input string 010110,\nthere are seven possible computations. All these computations are given in\nthe \fgure below.\nq1 q101q1 q101\n1q1\nq21\n1q1\nq2q10\n0\nεq3\nq3 hang\nhang\nε\nq3 q41 0q4\n1\nq20\nεq3\nq3 hang1q41q4 q40\nConsider the lowest path in the \fgure above:\n\u000fWhen reading the \frst symbol, the automaton stays in state q1.\n\u000fWhen reading the second symbol, the automaton switches to state q2.\n\u000fThe automaton does not read the third symbol (equivalently, it \\reads\"\nthe empty string \u000f), and switches to state q3. At this moment, the"
            }
        ]
    },
    {
        "section": "Page 45",
        "content": [
            {
                "type": "text",
                "text": "2.4. Nondeterministic \fnite automata 37\nautomaton cannot continue: The third symbol is 0, but there is no\nedge leaving q3that is labeled 0, and there is no edge leaving q3that\nis labeled\u000f. Therefore, the computation hangs at this point.\nFrom the \fgure, you can see that, out of the seven possible computations,\nexactly two end in the accept state q4(after the entire input string 010110 has\nbeen read). We say that the automaton accepts the string 010110, because\nthere is at least one computation that ends in the accept state.\nNow consider the input string 010. In this case, there are three possible\ncomputations:\n1.q10!q11!q10!q1\n2.q10!q11!q20!q3\n3.q10!q11!q2\u000f!q3!hang\nNone of these computations ends in the accept state (after the entire input\nstring 010 has been read). Therefore, we say that the automaton rejects the\ninput string 010.\nThe state diagram given above is an example of a nondeterministic \fnite\nautomaton (NFA). Informally, an NFA accepts a string, if there exists at least\none path in the state diagram that (i) starts in the start state, (ii) does not\nhang before the entire string has been read, and (iii) ends in an accept state.\nA string for which (i), (ii), and (iii) does not hold is rejected by the NFA.\nThe NFA given above accepts all binary strings that contain 101 or 11 as\na substring. All other binary strings are rejected.\n2.4.2 A second example\nLetAbe the language\nA=fw2f0;1g\u0003:whas a 1 in the third position from the right g:\nThe following state diagram de\fnes an NFA that accepts all strings that are\ninA, and rejects all strings that are not in A.\nq1 q2 q3 q40,1\n1 0,1 0 ,1"
            }
        ]
    },
    {
        "section": "Page 46",
        "content": [
            {
                "type": "text",
                "text": "38 Chapter 2. Finite Automata and Regular Languages\nThis NFA does the following. If it is in the start state q1and reads the\nsymbol 1, then it either stays in state q1or it \\guesses\" that this symbol\nis the third symbol from the right in the input string. In the latter case,\nthe NFA switches to state q2, and then it \\veri\fes\" that there are indeed\nexactly two remaining symbols in the input string. If there are more than\ntwo remaining symbols, then the NFA hangs (in state q4) after having read\nthe next two symbols.\nObserve how this guessing mechanism is used: The automaton can only\nread the input string once, from left to right. Hence, it does not know when\nit reaches the third symbol from the right. When the NFA reads a 1, it can\nguess that this is the third symbol from the right; after having made this\nguess, it veri\fes whether or not the guess was correct.\nIn Section 2.2.3, we have seen a DFA for the same language A. Observe\nthat the NFA has a much simpler structure than the DFA.\n2.4.3 A third example\nConsider the following state diagram, which de\fnes an NFA whose alphabet\nisf0g.\nε\nε0\n0\n0\n00\nThis NFA accepts the language\nA=f0k:k\u00110 mod 2 or k\u00110 mod 3g;\nwhere 0kis the string consisting of kmany 0s. (If k= 0, then 0k=\u000f.)\nObserve that Ais the union of the two languages\nA1=f0k:k\u00110 mod 2g"
            }
        ]
    },
    {
        "section": "Page 47",
        "content": [
            {
                "type": "text",
                "text": "2.4. Nondeterministic \fnite automata 39\nand\nA2=f0k:k\u00110 mod 3g:\nThe NFA basically consists of two DFAs: one of these accepts A1, whereas the\nother accepts A2. Given an input string w, the NFA has to decide whether\nor notw2A, which is equivalent to deciding whether or not w2A1or\nw2A2. The NFA makes this decision in the following way: At the start, it\n\\guesses\" whether (i) it is going to check whether or not w2A1(i.e., the\nlength ofwis even), or (ii) it is going to check whether or not w2A2(i.e.,\nthe length of wis a multiple of 3). After having made the guess, it veri\fes\nwhether or not the guess was correct. If w2A, then there exists a way of\nmaking the correct guess and verifying that wis indeed an element of A(by\nending in an accept state). If w62A, then no matter which guess is made,\nthe NFA will never end in an accept state.\n2.4.4 De\fnition of nondeterministic \fnite automaton\nThe previous examples give you an idea what nondeterministic \fnite au-\ntomata are and how they work. In this section, we give a formal de\fnition\nof these automata.\nFor any alphabet \u0006, we de\fne \u0006 \u000fto be the set\n\u0006\u000f= \u0006[f\u000fg:\nRecall the notion of a power set : For any set Q, the power set of Q, denoted\nbyP(Q), is the set of all subsets of Q, i.e.,\nP(Q) =fR:R\u0012Qg:\nDe\fnition 2.4.1 Anondeterministic \fnite automaton (NFA) is a 5-tuple\nM= (Q;\u0006;\u000e;q;F ), where\n1.Qis a \fnite set, whose elements are called states ,\n2. \u0006 is a \fnite set, called the alphabet ; the elements of \u0006 are called symbols ,\n3.\u000e:Q\u0002\u0006\u000f!P (Q) is a function, called the transition function ,\n4.qis an element of Q; it is called the start state ,\n5.Fis a subset of Q; the elements of Fare called accept states ."
            }
        ]
    },
    {
        "section": "Page 48",
        "content": [
            {
                "type": "text",
                "text": "40 Chapter 2. Finite Automata and Regular Languages\nAs for DFAs, the transition function \u000ecan be thought of as the \\program\"\nof the \fnite automaton M= (Q;\u0006;\u000e;q;F ):\n\u000fLetr2Q, and leta2\u0006\u000f. Then\u000e(r;a) is a (possibly empty) subset of\nQ. If the NFA Mis in stater, and if it reads a(whereamay be the\nempty string \u000f), thenMcan switch from state rtoanystate in\u000e(r;a).\nIf\u000e(r;a) =;, thenMcannot continue and the computation hangs.\nThe example given in Section 2.4.1 is an NFA, where Q=fq1;q2;q3;q4g,\n\u0006 =f0;1g, the start state is q1, the set of accept states is F=fq4g, and the\ntransition function \u000eis given by the following table:\n0 1 \u000f\nq1fq1g fq1;q2g ;\nq2fq3g ; f q3g\nq3; fq4g ;\nq4fq4g fq4g ;\nDe\fnition 2.4.2 LetM= (Q;\u0006;\u000e;q;F ) be an NFA, and let w2\u0006\u0003. We\nsay thatMacceptsw, if1\n\u000fw=\u000fand the start state qis an accept state, or\n\u000fthere exists an integer m\u00151, such that wcan be written as w=\ny1y2:::ym, whereyi2\u0006\u000ffor alliwith 1\u0014i\u0014m, and there exists a\nsequencer0;r1;:::;rmof states in Q, such that\n{r0=q,\n{ri+12\u000e(ri;yi+1), fori= 0;1;:::;m\u00001, and\n{rm2F.\nOtherwise, we say that Mrejects the stringw.\nThe NFA in the example in Section 2.4.1 accepts the string 01100. This\ncan be seen by taking\n\u000fm= 6,\n1Thanks to Antoine Vigneron for pointing out an error in a previous version of this\nde\fnition."
            }
        ]
    },
    {
        "section": "Page 49",
        "content": [
            {
                "type": "text",
                "text": "2.5. Equivalence of DFAs and NFAs 41\n\u000fw= 01\u000f100 =y1y2y3y4y5y6, and\n\u000fr0=q1,r1=q1,r2=q2,r3=q3,r4=q4,r5=q4, andr6=q4.\nDe\fnition 2.4.3 LetM= (Q;\u0006;\u000e;q;F ) be an NFA. The languageL(M)\naccepted byMis de\fned as\nL(M) =fw2\u0006\u0003:Macceptswg:\n2.5 Equivalence of DFAs and NFAs\nYou may have the impression that nondeterministic \fnite automata are more\npowerful than deterministic \fnite automata. In this section, we will show\nthat this is not the case. That is, we will prove that a language can be\naccepted by a DFA if and only if it can be accepted by an NFA. In order\nto prove this, we will show how to convert an arbitrary NFA to a DFA that\naccepts the same language.\nWhat about converting a DFA to an NFA? Well, there is (almost) nothing\nto do, because a DFA is also an NFA. This is not quite true, because\n\u000fthe transition function of a DFA maps a state and a symbol to a state,\nwhereas\n\u000fthe transition function of an NFA maps a state and a symbol to a set\nof zero or more states.\nThe formal conversion of a DFA to an NFA is done as follows: Let M=\n(Q;\u0006;\u000e;q;F ) be a DFA. Recall that \u000eis a function \u000e:Q\u0002\u0006!Q. We\nde\fne the function \u000e0:Q\u0002\u0006\u000f!P (Q) as follows. For any r2Qand for\nanya2\u0006\u000f,\n\u000e0(r;a) =\u001af\u000e(r;a)gifa6=\u000f,\n; ifa=\u000f.\nThenN= (Q;\u0006;\u000e0;q;F ) is an NFA, whose behavior is exactly the same as\nthat of the DFA M; the easiest way to see this is by observing that the state\ndiagrams of MandNare equal. Therefore, we have L(M) =L(N).\nIn the rest of this section, we will show how to convert an NFA to a DFA:\nTheorem 2.5.1 LetN= (Q;\u0006;\u000e;q;F )be a nondeterministic \fnite automa-\nton. There exists a deterministic \fnite automaton M, such that L(M) =\nL(N)."
            }
        ]
    },
    {
        "section": "Page 50",
        "content": [
            {
                "type": "text",
                "text": "42 Chapter 2. Finite Automata and Regular Languages\nProof. Recall that the NFA Ncan (in general) perform more than one\ncomputation on a given input string. The idea of the proof is to construct a\nDFAMthat runs all these di\u000berent computations simultaneously . (We have\nseen this idea already in the proof of Theorem 2.3.1.) To be more precise,\nthe DFAMwill have the following property:\n\u000fthe state that Mis in after having read an initial part of the input\nstring corresponds exactly to the set of all states that Ncan reach\nafter having read the same part of the input string.\nWe start by presenting the conversion for the case when Ndoes not\ncontain\u000f-transitions. In other words, the state diagram of Ndoes not contain\nany edge that has \u000fas a label. (Later, we will extend the conversion to the\ngeneral case.) Let the DFA Mbe de\fned as M= (Q0;\u0006;\u000e0;q0;F0), where\n\u000fthe setQ0of states is equal to Q0=P(Q); observe thatjQ0j= 2jQj,\n\u000fthe start state q0is equal toq0=fqg; soMhas the \\same\" start state\nasN,\n\u000fthe setF0of accept states is equal to the set of all elements RofQ0\nhaving the property that Rcontains at least one accept state of N, i.e.,\nF0=fR2Q0:R\\F6=;g;\n\u000fthe transition function \u000e0:Q0\u0002\u0006!Q0is de\fned as follows: For each\nR2Q0and for each a2\u0006,\n\u000e0(R;a) =[\nr2R\u000e(r;a):\nLet us see what the transition function \u000e0ofMdoes. First observe that,\nsinceNis an NFA, \u000e(r;a) is a subset of Q. This implies that \u000e0(R;a) is the\nunion of subsets of Qand, therefore, also a subset of Q. Hence,\u000e0(R;a) is\nan element of Q0.\nThe set\u000e(r;a) is equal to the set of all states of the NFA Nthat can be\nreached from state rby reading the symbol a. We take the union of these\nsets\u000e(r;a), whererranges over all elements of R, to obtain the new set\n\u000e0(R;a). This new set is the state that the DFA Mreaches from state R, by\nreading the symbol a."
            }
        ]
    },
    {
        "section": "Page 51",
        "content": [
            {
                "type": "text",
                "text": "2.5. Equivalence of DFAs and NFAs 43\nIn this way, we obtain the correspondence that was given in the beginning\nof this proof.\nAfter this warming-up, we can consider the general case. In other words,\nfrom now on, we allow \u000f-transitions in the NFA N. The DFA Mis de\fned as\nabove, except that the start state q0and the transition function \u000e0have to be\nmodi\fed. Recall that a computation of the NFA Nconsists of the following:\n1. Start in the start state qand make zero or more \u000f-transitions.\n2. Read one \\real\" symbol of \u0006 and move to a new state (or stay in the\ncurrent state).\n3. Make zero or more \u000f-transitions.\n4. Read one \\real\" symbol of \u0006 and move to a new state (or stay in the\ncurrent state).\n5. Make zero or more \u000f-transitions.\n6. Etc.\nThe DFAMwill simulate this computation in the following way:\n\u000fSimulate 1. in one single step. As we will see below, this simulation is\nimplicitly encoded in the de\fnition of the start state q0ofM.\n\u000fSimulate 2. and 3. in one single step.\n\u000fSimulate 4. and 5. in one single step.\n\u000fEtc.\nThus, in onestep, the DFA Msimulates the reading of one \\real\" symbol of\n\u0006, followed by making zero or more \u000f-transitions.\nTo formalize this, we need the notion of \u000f-closure . For any state rof the\nNFAN, the\u000f-closure of r, denoted by C\u000f(r), is de\fned to be the set of all\nstates ofNthat can be reached from r, by making zero or more \u000f-transitions.\nFor any state Rof the DFA M(hence,R\u0012Q), we de\fne\nC\u000f(R) =[\nr2RC\u000f(r):"
            }
        ]
    },
    {
        "section": "Page 52",
        "content": [
            {
                "type": "text",
                "text": "44 Chapter 2. Finite Automata and Regular Languages\nHow do we de\fne the start state q0of the DFA M? Before the NFA N\nreads its \frst \\real\" symbol of \u0006, it makes zero or more \u000f-transitions. In\nother words, at the moment when Nreads the \frst symbol of \u0006, it can be\nin any state of C\u000f(q). Therefore, we de\fne q0to be\nq0=C\u000f(q) =C\u000f(fqg):\nHow do we de\fne the transition function \u000e0of the DFA M? Assume that\nMis in stateR, and reads the symbol a. At this moment, the NFA Nwould\nhave been in any state rofR. By reading the symbol a,Ncan switch to\nany state in \u000e(r;a), and then make zero or more \u000f-transitions. Hence, the\nNFA can switch to any state in the set C\u000f(\u000e(r;a)). Based on this, we de\fne\n\u000e0(R;a) to be\n\u000e0(R;a) =[\nr2RC\u000f(\u000e(r;a)):\nTo summarize, the NFA N= (Q;\u0006;\u000e;q;F ) is converted to the DFA\nM= (Q0;\u0006;\u000e0;q0;F0), where\n\u000fQ0=P(Q),\n\u000fq0=C\u000f(fqg),\n\u000fF0=fR2Q0:R\\F6=;g,\n\u000f\u000e0:Q0\u0002\u0006!Q0is de\fned as follows: For each R2Q0and for each\na2\u0006,\n\u000e0(R;a) =[\nr2RC\u000f(\u000e(r;a)):\nThe results proved until now can be summarized in the following theorem.\nTheorem 2.5.2 LetAbe a language. Then Ais regular if and only if there\nexists a nondeterministic \fnite automaton that accepts A.\n2.5.1 An example\nConsider the NFA N= (Q;\u0006;\u000e;q;F ), whereQ=f1;2;3g, \u0006 =fa;bg,q= 1,\nF=f2g, and\u000eis given by the following table:"
            }
        ]
    },
    {
        "section": "Page 53",
        "content": [
            {
                "type": "text",
                "text": "2.5. Equivalence of DFAs and NFAs 45\na b \u000f\n1f3g ; f 2g\n2f1g ; ;\n3f2g f2;3g ;\nThe state diagram of Nis as follows:\n1 2\n3aaǫ\nba, b\nWe will show how to convert this NFA Nto a DFAMthat accepts the\nsame language. Following the proof of Theorem 2.5.1, the DFA Mis speci\fed\nbyM= (Q0;\u0006;\u000e0;q0;F0), where each of the components is de\fned below.\n\u000fQ0=P(Q). Hence,\nQ0=f;;f1g;f2g;f3g;f1;2g;f1;3g;f2;3g;f1;2;3gg:\n\u000fq0=C\u000f(fqg). Hence, the start state q0ofMis the set of all states of\nNthat can be reached from N's start state q= 1, by making zero or\nmore\u000f-transitions. We obtain\nq0=C\u000f(fqg) =C\u000f(f1g) =f1;2g:\n\u000fF0=fR2Q0:R\\F6=;g. Hence, the accept states of Mare those\nstates that contain the accept state 2 of N. We obtain\nF0=ff2g;f1;2g;f2;3g;f1;2;3gg:"
            }
        ]
    },
    {
        "section": "Page 54",
        "content": [
            {
                "type": "text",
                "text": "46 Chapter 2. Finite Automata and Regular Languages\n\u000f\u000e0:Q0\u0002\u0006!Q0is de\fned as follows: For each R2Q0and for each\na2\u0006,\n\u000e0(R;a) =[\nr2RC\u000f(\u000e(r;a)):\nIn this example \u000e0is given by\n\u000e0(;;a) =;\u000e0(;;b) =;\n\u000e0(f1g;a) =f3g\u000e0(f1g;b) =;\n\u000e0(f2g;a) =f1;2g\u000e0(f2g;b) =;\n\u000e0(f3g;a) =f2g\u000e0(f3g;b) =f2;3g\n\u000e0(f1;2g;a) =f1;2;3g\u000e0(f1;2g;b) =;\n\u000e0(f1;3g;a) =f2;3g\u000e0(f1;3g;b) =f2;3g\n\u000e0(f2;3g;a) =f1;2g\u000e0(f2;3g;b) =f2;3g\n\u000e0(f1;2;3g;a) =f1;2;3g\u000e0(f1;2;3g;b) =f2;3g\nThe state diagram of the DFA Mis as follows:"
            }
        ]
    },
    {
        "section": "Page 55",
        "content": [
            {
                "type": "text",
                "text": "2.5. Equivalence of DFAs and NFAs 47\n/0 {1}\n{2}\n{3} {1,2}\n{2,3} {1,3}\n{1,2,3}a,b\nb\nab\naa\nb\na,b ab\nba\nb\na\nWe make the following observations:\n\u000fThe statesf1gandf1;3gdo not have incoming edges. Therefore, these\ntwo states cannot be reached from the start state f1;2g.\n\u000fThe statef3ghas only one incoming edge; it comes from the state\nf1g. Sincef1gcannot be reached from the start state, f3gcannot be\nreached from the start state.\n\u000fThe statef2ghas only one incoming edge; it comes from the state\nf3g. Sincef3gcannot be reached from the start state, f2gcannot be\nreached from the start state.\nHence, we can remove the four states f1g,f2g,f3g, andf1;3g. The\nresulting DFA accepts the same language as the DFA above. This leads\nto the following state diagram, which depicts a DFA that accepts the same\nlanguage as the NFA N:"
            }
        ]
    },
    {
        "section": "Page 56",
        "content": [
            {
                "type": "text",
                "text": "48 Chapter 2. Finite Automata and Regular Languages\n/0\n{1,2}\n{2,3}\n{1,2,3}a,b\nab\nba\nb\na\n2.6 Closure under the regular operations\nIn Section 2.3, we have de\fned the regular operations union, concatenation,\nand star. We proved in Theorem 2.3.1 that the union of two regular lan-\nguages is a regular language. We also explained why it is not clear that the\nconcatenation of two regular languages is regular, and that the star of a reg-\nular language is regular. In this section, we will see that the concept of NFA,\ntogether with Theorem 2.5.2, can be used to give a simple proof of the fact\nthat the regular languages are indeed closed under the regular operations.\nWe start by giving an alternative proof of Theorem 2.3.1:\nTheorem 2.6.1 The set of regular languages is closed under the union op-\neration, i.e., if A1andA2are regular languages over the same alphabet \u0006,\nthenA1[A2is also a regular language."
            }
        ]
    },
    {
        "section": "Page 57",
        "content": [
            {
                "type": "text",
                "text": "2.6. Closure under the regular operations 49\nq1\nM1\nM2q2q0q1\nq2ε\nε\nM\nFigure 2.1: The NFAMacceptsL(M1)[L(M2).\nProof. SinceA1is regular, there is, by Theorem 2.5.2, an NFA M1=\n(Q1;\u0006;\u000e1;q1;F1), such that A1=L(M1). Similarly, there is an NFA M2=\n(Q2;\u0006;\u000e2;q2;F2), such that A2=L(M2). We may assume that Q1\\Q2=;,\nbecause otherwise, we can give new \\names\" to the states of Q1andQ2.\nFrom these two NFAs, we will construct an NFA M= (Q;\u0006;\u000e;q 0;F), such\nthatL(M) =A1[A2. The construction is illustrated in Figure 2.1. The\nNFAMis de\fned as follows:\n1.Q=fq0g[Q1[Q2, whereq0is a new state.\n2.q0is the start state of M.\n3.F=F1[F2.\n4.\u000e:Q\u0002\u0006\u000f!P (Q) is de\fned as follows: For any r2Qand for any"
            }
        ]
    },
    {
        "section": "Page 58",
        "content": [
            {
                "type": "text",
                "text": "50 Chapter 2. Finite Automata and Regular Languages\na2\u0006\u000f,\n\u000e(r;a) =8\n>><\n>>:\u000e1(r;a) ifr2Q1,\n\u000e2(r;a) ifr2Q2,\nfq1;q2gifr=q0anda=\u000f,\n; ifr=q0anda6=\u000f.\nTheorem 2.6.2 The set of regular languages is closed under the concatena-\ntion operation, i.e., if A1andA2are regular languages over the same alphabet\n\u0006, thenA1A2is also a regular language.\nProof. LetM1= (Q1;\u0006;\u000e1;q1;F1) be an NFA, such that A1=L(M1).\nSimilarly, let M2= (Q2;\u0006;\u000e2;q2;F2) be an NFA, such that A2=L(M2).\nAs in the proof of Theorem 2.6.1, we may assume that Q1\\Q2=;. We\nwill construct an NFA M= (Q;\u0006;\u000e;q 0;F), such that L(M) =A1A2. The\nconstruction is illustrated in Figure 2.2. The NFA Mis de\fned as follows:\n1.Q=Q1[Q2.\n2.q0=q1.\n3.F=F2.\n4.\u000e:Q\u0002\u0006\u000f!P (Q) is de\fned as follows: For any r2Qand for any\na2\u0006\u000f,\n\u000e(r;a) =8\n>><\n>>:\u000e1(r;a) if r2Q1andr62F1,\n\u000e1(r;a) if r2F1anda6=\u000f,\n\u000e1(r;a)[fq2gifr2F1anda=\u000f,\n\u000e2(r;a) if r2Q2.\nTheorem 2.6.3 The set of regular languages is closed under the star oper-\nation, i.e., if Ais a regular language, then A\u0003is also a regular language."
            }
        ]
    },
    {
        "section": "Page 59",
        "content": [
            {
                "type": "text",
                "text": "2.6. Closure under the regular operations 51\nq1\nM1 M2q2\nq2ε\nε\nεq0\nM\nFigure 2.2: The NFAMacceptsL(M1)L(M2).\nq1\nNq1\nq0εε\nε\nε\nM\nFigure 2.3: The NFAMaccepts (L(N))\u0003.\nProof. Let \u0006 be the alphabet of Aand letN= (Q1;\u0006;\u000e1;q1;F1) be an\nNFA, such that A=L(N). We will construct an NFA M= (Q;\u0006;\u000e;q 0;F),\nsuch thatL(M) =A\u0003. The construction is illustrated in Figure 2.3. The\nNFAMis de\fned as follows:"
            }
        ]
    },
    {
        "section": "Page 60",
        "content": [
            {
                "type": "text",
                "text": "52 Chapter 2. Finite Automata and Regular Languages\n1.Q=fq0g[Q1, whereq0is a new state.\n2.q0is the start state of M.\n3.F=fq0g[F1. (Since\u000f2A\u0003,q0has to be an accept state.)\n4.\u000e:Q\u0002\u0006\u000f!P (Q) is de\fned as follows: For any r2Qand for any\na2\u0006\u000f,\n\u000e(r;a) =8\n>>>><\n>>>>:\u000e1(r;a) if r2Q1andr62F1,\n\u000e1(r;a) if r2F1anda6=\u000f,\n\u000e1(r;a)[fq1gifr2F1anda=\u000f,\nfq1g ifr=q0anda=\u000f,\n; ifr=q0anda6=\u000f.\nIn the \fnal theorem of this section, we mention (without proof) two more\nclosure properties of the regular languages:\nTheorem 2.6.4 The set of regular languages is closed under the complement\nand intersection operations:\n1. IfAis a regular language over the alphabet \u0006, then the complement\nA=fw2\u0006\u0003:w62Ag\nis also a regular language.\n2. IfA1andA2are regular languages over the same alphabet \u0006, then the\nintersection\nA1\\A2=fw2\u0006\u0003:w2A1andw2A2g\nis also a regular language.\n2.7 Regular expressions\nIn this section, we present regular expressions, which are a means to describe\nlanguages. As we will see, the class of languages that can be described by\nregular expressions coincides with the class of regular languages."
            }
        ]
    },
    {
        "section": "Page 61",
        "content": [
            {
                "type": "text",
                "text": "2.7. Regular expressions 53\nBefore formally de\fning the notion of a regular expression, we give some\nexamples. Consider the expression\n(0[1)01\u0003:\nThe language described by this expression is the set of all binary strings\n1. that start with either 0 or 1 (this is indicated by (0 [1)),\n2. for which the second symbol is 0 (this is indicated by 0), and\n3. that end with zero or more 1s (this is indicated by 1\u0003).\nThat is, the language described by this expression is\nf00;001;0011;00111;:::; 10;101;1011;10111;:::g:\nHere are some more examples (in all cases, the alphabet is f0;1g):\n\u000fThe languagefw:wcontains exactly two 0s gis described by the ex-\npression\n1\u000301\u000301\u0003:\n\u000fThe languagefw:wcontains at least two 0s gis described by the ex-\npression\n(0[1)\u00030(0[1)\u00030(0[1)\u0003:\n\u000fThe languagefw: 1011 is a substring of wgis described by the ex-\npression\n(0[1)\u00031011(0[1)\u0003:\n\u000fThe languagefw: the length of wis evengis described by the expres-\nsion\n((0[1)(0[1))\u0003:\n\u000fThe languagefw: the length of wis oddgis described by the expres-\nsion\n(0[1) ((0[1)(0[1))\u0003:\n\u000fThe languagef1011;0gis described by the expression\n1011[0:"
            }
        ]
    },
    {
        "section": "Page 62",
        "content": [
            {
                "type": "text",
                "text": "54 Chapter 2. Finite Automata and Regular Languages\n\u000fThe languagefw: the \frst and last symbols of ware equalgis de-\nscribed by the expression\n0(0[1)\u00030[1(0[1)\u00031[0[1:\nAfter these examples, we give a formal (and inductive) de\fnition of regular\nexpressions :\nDe\fnition 2.7.1 Let \u0006 be a non-empty alphabet.\n1.\u000fis a regular expression.\n2.;is a regular expression.\n3. For each a2\u0006,ais a regular expression.\n4. IfR1andR2are regular expressions, then R1[R2is a regular expres-\nsion.\n5. IfR1andR2are regular expressions, then R1R2is a regular expression.\n6. IfRis a regular expression, then R\u0003is a regular expression.\nYou can regard 1., 2., and 3. as being the \\building blocks\" of regular\nexpressions. Items 4., 5., and 6. give rules that can be used to combine\nregular expressions into new (and \\larger\") regular expressions. To give an\nexample, we claim that\n(0[1)\u0003101(0[1)\u0003\nis a regular expression (where the alphabet \u0006 is equal to f0;1g). In order\nto prove this, we have to show that this expression can be \\built\" using the\n\\rules\" given in De\fnition 2.7.1. Here we go:\n\u000fBy 3., 0 is a regular expression.\n\u000fBy 3., 1 is a regular expression.\n\u000fSince 0 and 1 are regular expressions, by 4., 0 [1 is a regular expression.\n\u000fSince 0[1 is a regular expression, by 6., (0 [1)\u0003is a regular expression.\n\u000fSince 1 and 0 are regular expressions, by 5., 10 is a regular expression."
            }
        ]
    },
    {
        "section": "Page 63",
        "content": [
            {
                "type": "text",
                "text": "2.7. Regular expressions 55\n\u000fSince 10 and 1 are regular expressions, by 5., 101 is a regular expression.\n\u000fSince (0[1)\u0003and 101 are regular expressions, by 5., (0 [1)\u0003101 is a\nregular expression.\n\u000fSince (0[1)\u0003101 and (0[1)\u0003are regular expressions, by 5., (0 [\n1)\u0003101(0[1)\u0003is a regular expression.\nNext we de\fne the language that is described by a regular expression:\nDe\fnition 2.7.2 Let \u0006 be a non-empty alphabet.\n1. The regular expression \u000fdescribes the language f\u000fg.\n2. The regular expression ;describes the language ;.\n3. For each a2\u0006, the regular expression adescribes the language fag.\n4. LetR1andR2be regular expressions and let L1andL2be the lan-\nguages described by them, respectively. The regular expression R1[R2\ndescribes the language L1[L2.\n5. LetR1andR2be regular expressions and let L1andL2be the languages\ndescribed by them, respectively. The regular expression R1R2describes\nthe language L1L2.\n6. LetRbe a regular expression and let Lbe the language described by\nit. The regular expression R\u0003describes the language L\u0003.\nWe consider some examples:\n\u000fThe regular expression (0 [\u000f)(1[\u000f) describes the language f01;0;1;\u000fg.\n\u000fThe regular expression 0 [\u000fdescribes the language f0;\u000fg, whereas the\nregular expression 1\u0003describes the language f\u000f;1;11;111;:::g. There-\nfore, the regular expression (0 [\u000f)1\u0003describes the language\nf0;01;011;0111;:::;\u000f; 1;11;111;:::g:\nObserve that this language is also described by the regular expression\n01\u0003[1\u0003."
            }
        ]
    },
    {
        "section": "Page 64",
        "content": [
            {
                "type": "text",
                "text": "56 Chapter 2. Finite Automata and Regular Languages\n\u000fThe regular expression 1\u0003;describes the empty language, i.e., the lan-\nguage;. (You should convince yourself that this is correct.)\n\u000fThe regular expression ;\u0003describes the language f\u000fg.\nDe\fnition 2.7.3 LetR1andR2be regular expressions and let L1andL2\nbe the languages described by them, respectively. If L1=L2(i.e.,R1and\nR2describe the same language), then we will write R1=R2.\nHence, even though (0 [\u000f)1\u0003and 01\u0003[1\u0003are di\u000berent regular expressions,\nwe write\n(0[\u000f)1\u0003= 01\u0003[1\u0003;\nbecause they describe the same language.\nIn Section 2.8.2, we will show that every regular language can be described\nby a regular expression. The proof of this fact is purely algebraic and uses\nthe following algebraic identities involving regular expressions.\nTheorem 2.7.4 LetR1,R2, andR3be regular expressions. The following\nidentities hold:\n1.R1;=;R1=;.\n2.R1\u000f=\u000fR1=R1.\n3.R1[;=;[R1=R1.\n4.R1[R1=R1.\n5.R1[R2=R2[R1.\n6.R1(R2[R3) =R1R2[R1R3.\n7.(R1[R2)R3=R1R3[R2R3.\n8.R1(R2R3) = (R1R2)R3.\n9.;\u0003=\u000f.\n10.\u000f\u0003=\u000f.\n11.(\u000f[R1)\u0003=R\u0003\n1."
            }
        ]
    },
    {
        "section": "Page 65",
        "content": [
            {
                "type": "text",
                "text": "2.8. Equivalence of regular expressions and regular languages 57\n12.(\u000f[R1)(\u000f[R1)\u0003=R\u0003\n1.\n13.R\u0003\n1(\u000f[R1) = (\u000f[R1)R\u0003\n1=R\u0003\n1.\n14.R\u0003\n1R2[R2=R\u0003\n1R2.\n15.R1(R2R1)\u0003= (R1R2)\u0003R1.\n16.(R1[R2)\u0003= (R\u0003\n1R2)\u0003R\u0003\n1= (R\u0003\n2R1)\u0003R\u0003\n2.\nWe will not present the (boring) proofs of these identities, but urge you\nto convince yourself informally that they make perfect sense. To give an\nexample, we mentioned above that\n(0[\u000f)1\u0003= 01\u0003[1\u0003:\nWe can verify this identity in the following way:\n(0[\u000f)1\u0003= 01\u0003[\u000f1\u0003(by identity 7)\n= 01\u0003[1\u0003(by identity 2)\n2.8 Equivalence of regular expressions and reg-\nular languages\nIn the beginning of Section 2.7, we mentioned the following result:\nTheorem 2.8.1 LetLbe a language. Then Lis regular if and only if there\nexists a regular expression that describes L.\nThe proof of this theorem consists of two parts:\n\u000fIn Section 2.8.1, we will prove that every regular expression describes\na regular language.\n\u000fIn Section 2.8.2, we will prove that every DFA Mcan be converted to\na regular expression that describes the language L(M).\nThese two results will prove Theorem 2.8.1."
            }
        ]
    },
    {
        "section": "Page 66",
        "content": [
            {
                "type": "text",
                "text": "58 Chapter 2. Finite Automata and Regular Languages\n2.8.1 Every regular expression describes a regular lan-\nguage\nLetRbe an arbitrary regular expression over the alphabet \u0006. We will prove\nthat the language described by Ris a regular language. The proof is by\ninduction on the structure of R(i.e., by induction on the way Ris \\built\"\nusing the \\rules\" given in De\fnition 2.7.1).\nThe \frst base case: Assume that R=\u000f. ThenRdescribes the lan-\nguagef\u000fg. In order to prove that this language is regular, it su\u000eces, by\nTheorem 2.5.2, to construct an NFA M= (Q;\u0006;\u000e;q;F ) that accepts this\nlanguage. This NFA is obtained by de\fning Q=fqg,qis the start state,\nF=fqg, and\u000e(q;a) =;for alla2\u0006\u000f. The \fgure below gives the state\ndiagram of M:\nq\nThe second base case: Assume that R=;. ThenRdescribes the language\n;. In order to prove that this language is regular, it su\u000eces, by Theorem 2.5.2,\nto construct an NFA M= (Q;\u0006;\u000e;q;F ) that accepts this language. This\nNFA is obtained by de\fning Q=fqg,qis the start state, F=;, and\n\u000e(q;a) =;for alla2\u0006\u000f. The \fgure below gives the state diagram of M:\nq\nThe third base case: Leta2\u0006 and assume that R=a. ThenRdescribes\nthe languagefag. In order to prove that this language is regular, it su\u000eces,\nby Theorem 2.5.2, to construct an NFA M= (Q;\u0006;\u000e;q 1;F) that accepts\nthis language. This NFA is obtained by de\fning Q=fq1;q2g,q1is the start\nstate,F=fq2g, and\n\u000e(q1;a) =fq2g;\n\u000e(q1;b) =;for allb2\u0006\u000fnfag,\n\u000e(q2;b) =;for allb2\u0006\u000f.\nThe \fgure below gives the state diagram of M:"
            }
        ]
    },
    {
        "section": "Page 67",
        "content": [
            {
                "type": "text",
                "text": "2.8. Equivalence of regular expressions and regular languages 59\nq1 q2a\nThe \frst case of the induction step: Assume that R=R1[R2, where\nR1andR2are regular expressions. Let L1andL2be the languages described\nbyR1andR2, respectively, and assume that L1andL2are regular. Then R\ndescribes the language L1[L2, which, by Theorem 2.6.1, is regular.\nThe second case of the induction step: Assume that R=R1R2, where\nR1andR2are regular expressions. Let L1andL2be the languages described\nbyR1andR2, respectively, and assume that L1andL2are regular. Then R\ndescribes the language L1L2, which, by Theorem 2.6.2, is regular.\nThe third case of the induction step: Assume that R= (R1)\u0003, where\nR1is a regular expression. Let L1be the language described by R1and\nassume that L1is regular. Then Rdescribes the language ( L1)\u0003, which, by\nTheorem 2.6.3, is regular.\nThis concludes the proof of the claim that every regular expression de-\nscribes a regular language.\nTo give an example, consider the regular expression\n(ab[a)\u0003;\nwhere the alphabet is fa;bg. We will prove that this regular expression de-\nscribes a regular language, by constructing an NFA that accepts the language\ndescribed by this regular expression. Observe how the regular expression is\n\\built\":\n\u000fTake the regular expressions aandb, and combine them into the regular\nexpressionab.\n\u000fTake the regular expressions abanda, and combine them into the\nregular expression ab[a.\n\u000fTake the regular expression ab[a, and transform it into the regular\nexpression ( ab[a)\u0003.\nFirst, we construct an NFA M1that accepts the language described by\nthe regular expression a:"
            }
        ]
    },
    {
        "section": "Page 68",
        "content": [
            {
                "type": "text",
                "text": "60 Chapter 2. Finite Automata and Regular Languages\naM1\nNext, we construct an NFA M2that accepts the language described by\nthe regular expression b:\nM2b\nNext, we apply the construction given in the proof of Theorem 2.6.2 to\nM1andM2. This gives an NFA M3that accepts the language described by\nthe regular expression ab:\nM3a ε b\nNext, we apply the construction given in the proof of Theorem 2.6.1 to\nM3andM1. This gives an NFA M4that accepts the language described by\nthe regular expression ab[a:\na ε b\naε\nεM4\nFinally, we apply the construction given in the proof of Theorem 2.6.3\ntoM4. This gives an NFA M5that accepts the language described by the\nregular expression ( ab[a)\u0003:"
            }
        ]
    },
    {
        "section": "Page 69",
        "content": [
            {
                "type": "text",
                "text": "2.8. Equivalence of regular expressions and regular languages 61\na ε b\naε\nεεε\nεM5\n2.8.2 Converting a DFA to a regular expression\nIn this section, we will prove that every DFA Mcan be converted to a regular\nexpression that describes the language L(M). In order to prove this result,\nwe need to solve recurrence relations involving languages.\nSolving recurrence relations\nLet \u0006 be an alphabet, let BandCbe \\known\" languages in \u0006\u0003such that\n\u000f62B, and letLbe an \\unknown\" language such that\nL=BL[C:\nCan we \\solve\" this equation for L? That is, can we express Lin terms of\nBandC?\nConsider an arbitrary string uinL. We are going to determine how u\nlooks like. Since u2LandL=BL[C, we know that uis a string in\nBL[C. Hence, there are two possibilities for u.\n1.uis an element of C.\n2.uis an element of BL. In this case, there are strings b2Bandv2L\nsuch thatu=bv. Since\u000f62B, we haveb6=\u000fand, therefore,jvj<juj.\n(Recall thatjvjdenotes the length, i.e., the number of symbols, of the\nstringv.) Sincevis a string in L, which is equal to BL[C,vis a\nstring inBL[C. Hence, there are two possibilities for v."
            }
        ]
    },
    {
        "section": "Page 70",
        "content": [
            {
                "type": "text",
                "text": "62 Chapter 2. Finite Automata and Regular Languages\n(a)vis an element of C. In this case,\nu=bv;whereb2Bandv2C; thus,u2BC.\n(b)vis an element of BL. In this case, there are strings b02Band\nw2Lsuch thatv=b0w. Since\u000f62B, we haveb06=\u000fand,\ntherefore,jwj<jvj. Sincewis a string in L, which is equal to\nBL[C,wis a string in BL[C. Hence, there are two possibilities\nforw.\ni.wis an element of C. In this case,\nu=bb0w;whereb;b02Bandw2C; thus,u2BBC .\nii.wis an element of BL. In this case, there are strings b002B\nandx2Lsuch thatw=b00x. Since\u000f62B, we haveb006=\u000f\nand, therefore,jxj<jwj. Sincexis a string in L, which is\nequal toBL[C,xis a string in BL[C. Hence, there are\ntwo possibilities for x.\nA.xis an element of C. In this case,\nu=bb0b00x;whereb;b0;b002Bandx2C; thus,u2BBBC .\nB.xis an element of BL. Etc., etc.\nThis process hopefully convinces you that any string uinLcan be written\nas the concatenation of zero or more strings in B, followed by one string in\nC. In fact,Lconsists of exactly those strings having this property:\nLemma 2.8.2 Let\u0006be an alphabet, and let B,C, andLbe languages in\n\u0006\u0003such that\u000f62Band\nL=BL[C:\nThen\nL=B\u0003C:\nProof. First, we show that B\u0003C\u0012L. Letube an arbitrary string in B\u0003C.\nThenuis the concatenation of kstrings ofB, for somek\u00150, followed by\none string of C. We proceed by induction on k.\nThe base case is when k= 0. In this case, uis a string in C. Hence,uis\na string inBL[C. SinceBL[C=L, it follows that uis a string in L."
            }
        ]
    },
    {
        "section": "Page 71",
        "content": [
            {
                "type": "text",
                "text": "2.8. Equivalence of regular expressions and regular languages 63\nNow letk\u00151. Then we can write u=vwc, wherevis a string in B,\nwis the concatenation of k\u00001 strings of B, andcis a string of C. De\fne\ny=wc. Observe that yis the concatenation of k\u00001 strings of Bfollowed\nby one string of C. Therefore, by induction, the string yis an element of L.\nHence,u=vy, wherevis a string in Bandyis a string in L. This shows\nthatuis a string in BL. Hence,uis a string in BL[C. SinceBL[C=L,\nit follows that uis a string in L. This completes the proof that B\u0003C\u0012L.\nIt remains to show that L\u0012B\u0003C. Letube an arbitrary string in L,\nand let`be its length (i.e., `is the number of symbols in u). We prove by\ninduction on `thatuis a string in B\u0003C.\nThe base case is when `= 0. Then u=\u000f. Sinceu2LandL=BL[C,\nuis a string in BL[C. Since\u000f62B,ucannot be a string in BL. Hence,u\nmust be a string in C. SinceC\u0012B\u0003C, it follows that uis a string in B\u0003C.\nLet`\u00151. Ifuis a string in C, thenuis a string in B\u0003Cand we are done.\nSo assume that uis not a string in C. Sinceu2LandL=BL[C,uis a\nstring inBL. Hence, there are strings b2Bandv2Lsuch thatu=bv.\nSince\u000f62B, the length of bis at least one; hence, the length of vis less than\nthe length of u. By induction, vis a string in B\u0003C. Hence,u=bv, where\nb2Bandv2B\u0003C. This shows that u2B(B\u0003C). SinceB(B\u0003C)\u0012B\u0003C,\nit follows that u2B\u0003C.\nNote that Lemma 2.8.2 holds for anylanguageBthat does not contain\nthe empty string \u000f. As an example, assume that B=;. Then the language\nLsatis\fes the equation\nL=BL[C=;L[C:\nUsing Theorem 2.7.4, this equation becomes\nL=;[C=C:\nWe now show that Lemma 2.8.2 also implies that L=C: Since\u000f62B,\nLemma 2.8.2 implies that L=B\u0003C, which, using Theorem 2.7.4, becomes\nL=B\u0003C=;\u0003C=\u000fC=C:\nThe conversion\nWe will now use Lemma 2.8.2 to prove that every DFA can be converted to\na regular expression."
            }
        ]
    },
    {
        "section": "Page 72",
        "content": [
            {
                "type": "text",
                "text": "64 Chapter 2. Finite Automata and Regular Languages\nLetM= (Q;\u0006;\u000e;q;F ) be an arbitrary deterministic \fnite automaton.\nWe will show that there exists a regular expression that describes the lan-\nguageL(M).\nFor each state r2Q, we de\fne\nLr=fw2\u0006\u0003: the path in the state diagram of Mthat starts\nin staterand that corresponds to wends in a\nstate ofFg.\nIn words,Lris the language accepted by M,ifrwere the start state .\nWe will show that each such language Lrcan be described by a regular\nexpression. Since L(M) =Lq, this will prove that L(M) can be described by\na regular expression.\nThe basic idea is to set up equations for the languages Lr, which we then\nsolve using Lemma 2.8.2. We claim that\nLr=[\na2\u0006a\u0001L\u000e(r;a) ifr62F: (2.2)\nWhy is this true? Let wbe a string in Lr. Then the path Pin the state\ndiagram of Mthat starts in state rand that corresponds to wends in a\nstate ofF. Sincer62F, this path contains at least one edge. Let r0be the\nstate that follows the \frst state (i.e., r) ofP. Thenr0=\u000e(r;b) for some\nsymbolb2\u0006. Hence,bis the \frst symbol of w. Writew=bv, wherevis\nthe remaining part of w. Then the path P0=Pnfrgin the state diagram\nofMthat starts in state r0and that corresponds to vends in a state of F.\nTherefore,v2Lr0=L\u000e(r;b). Hence,\nw2b\u0001L\u000e(r;b)\u0012[\na2\u0006a\u0001L\u000e(r;a):\nConversely, let wbe a string inS\na2\u0006a\u0001L\u000e(r;a). Then there is a symbol b2\u0006\nand a string v2L\u000e(r;b)such thatw=bv. LetP0be the path in the state\ndiagram of Mthat starts in state \u000e(r;b) and that corresponds to v. Since\nv2L\u000e(r;b), this path ends in a state of F. LetPbe the path in the state\ndiagram of Mthat starts in r, follows the edge to \u000e(r;b), and then follows P0.\nThis pathPcorresponds to wand ends in a state of F. Therefore, w2Lr.\nThis proves the correctness of (2.2)."
            }
        ]
    },
    {
        "section": "Page 73",
        "content": [
            {
                "type": "text",
                "text": "2.8. Equivalence of regular expressions and regular languages 65\nSimilarly, we can prove that\nLr=\u000f[ [\na2\u0006a\u0001L\u000e(r;a)!\nifr2F: (2.3)\nSo we now have a set of equations in the \\unknowns\" Lr, forr2Q. The\nnumber of equations is equal to the size of Q. In other words, the number\nof equations is equal to the number of unknowns. The regular expression for\nL(M) =Lqis obtained by solving these equations using Lemma 2.8.2.\nOf course, we have to convince ourselves that these equations have a so-\nlution for any given DFA. Before we deal with this issue, we give an example.\nAn example\nConsider the deterministic \fnite automaton M= (Q;\u0006;\u000e;q 0;F), whereQ=\nfq0;q1;q2g, \u0006 =fa;bg,q0is the start state, F=fq2g, and\u000eis given in the\nstate diagram below. We show how to obtain the regular expression that\ndescribes the language accepted by M.\nq0\nq1q2a\na a\nbb\nb\nFor this case, (2.2) and (2.3) give the following equations:\n8\n<\n:Lq0=a\u0001Lq0[b\u0001Lq2\nLq1=a\u0001Lq0[b\u0001Lq1\nLq2=\u000f[a\u0001Lq1[b\u0001Lq0"
            }
        ]
    },
    {
        "section": "Page 74",
        "content": [
            {
                "type": "text",
                "text": "66 Chapter 2. Finite Automata and Regular Languages\nIn the third equation, Lq2is expressed in terms of Lq0andLq1. Hence, if we\nsubstitute the third equation into the \frst one, and use Theorem 2.7.4, then\nwe get\nLq0=a\u0001Lq0[b\u0001(\u000f[a\u0001Lq1[b\u0001Lq0)\n= (a[bb)\u0001Lq0[ba\u0001Lq1[b:\nWe obtain the following set of equations.\n\u001aLq0= (a[bb)\u0001Lq0[ba\u0001Lq1[b\nLq1=b\u0001Lq1[a\u0001Lq0\nLetL=Lq1,B=b, andC=a\u0001Lq0. Then\u000f62Band the second equation\nreadsL=BL[C. Hence, by Lemma 2.8.2,\nLq1=L=B\u0003C=b\u0003a\u0001Lq0:\nIf we substitute Lq1into the \frst equation, then we get (again using Theo-\nrem 2.7.4)\nLq0= (a[bb)\u0001Lq0[ba\u0001b\u0003a\u0001Lq0[b\n= (a[bb[bab\u0003a)Lq0[b:\nAgain applying Lemma 2.8.2, this time with L=Lq0,B=a[bb[bab\u0003aand\nC=b, gives\nLq0= (a[bb[bab\u0003a)\u0003b:\nThus, the regular expression that describes the language accepted by Mis\n(a[bb[bab\u0003a)\u0003b:\nCompleting the correctness of the conversion\nIt remains to prove that, for any DFA, the system of equations (2.2) and (2.3)\ncan be solved. This will follow from the following (more general) lemma.\n(You should verify that the equations (2.2) and (2.3) are in the form as\nspeci\fed in this lemma.)"
            }
        ]
    },
    {
        "section": "Page 75",
        "content": [
            {
                "type": "text",
                "text": "2.8. Equivalence of regular expressions and regular languages 67\nLemma 2.8.3 Letn\u00151be an integer and, for 1\u0014i\u0014nand1\u0014j\u0014n,\nletBijandCibe regular expressions such that \u000f62Bij. LetL1;L2;:::;Lnbe\nlanguages that satisfy\nLi= n[\nj=1BijLj!\n[Cifor1\u0014i\u0014n.\nThenL1can be expressed as a regular expression only involving the regular\nexpressions BijandCi.\nProof. The proof is by induction on n. The base case is when n= 1. In\nthis case, we have\nL1=B11L1[C1:\nSince\u000f62B11, it follows from Lemma 2.8.2 that L1=B\u0003\n11C1. This proves\nthe base case.\nLetn\u00152 and assume the lemma is true for n\u00001. We have\nLn= n[\nj=1BnjLj!\n[Cn\n=BnnLn[ n\u00001[\nj=1BnjLj!\n[Cn:\nSince\u000f62Bnn, it follows from Lemma 2.8.2 that\nLn=B\u0003\nnn  n\u00001[\nj=1BnjLj!\n[Cn!\n=B\u0003\nnn n\u00001[\nj=1BnjLj!\n[B\u0003\nnnCn\n= n\u00001[\nj=1B\u0003\nnnBnjLj!\n[B\u0003\nnnCn\nBy substituting this equation for Lninto the equations for Li, 1\u0014i\u0014n\u00001,"
            }
        ]
    },
    {
        "section": "Page 76",
        "content": [
            {
                "type": "text",
                "text": "68 Chapter 2. Finite Automata and Regular Languages\nwe obtain\nLi= n[\nj=1BijLj!\n[Ci\n=BinLn[ n\u00001[\nj=1BijLj!\n[Ci\n= n\u00001[\nj=1(BinB\u0003\nnnBnj[Bij)Lj!\n[BinB\u0003\nnnCn[Ci:\nThus, we have obtained n\u00001 equations in L1;L2;:::;Ln\u00001. Since\u000f62\nBinB\u0003\nnnBnj[Bij, it follows from the induction hypothesis that L1can be\nexpressed as a regular expression only involving the regular expressions Bij\nandCi.\n2.9 The pumping lemma and nonregular lan-\nguages\nIn the previous sections, we have seen that the class of regular languages is\nclosed under various operations, and that these languages can be described by\n(deterministic or nondeterministic) \fnite automata and regular expressions.\nThese properties helped in developing techniques for showing that a language\nis regular. In this section, we will present a tool that can be used to prove\nthat certain languages are notregular. Observe that for a regular language,\n1. the amount of memory that is needed to determine whether or not a\ngiven string is in the language is \fnite and independent of the length\nof the string, and\n2. if the language consists of an in\fnite number of strings, then this lan-\nguage should contain in\fnite subsets having a fairly repetitive struc-\nture.\nIntuitively, languages that do not follow 1. or 2. should be nonregular. For\nexample, consider the language\nf0n1n:n\u00150g:"
            }
        ]
    },
    {
        "section": "Page 77",
        "content": [
            {
                "type": "text",
                "text": "2.9. The pumping lemma and nonregular languages 69\nThis language should be nonregular, because it seems unlikely that a DFA can\nremember how many 0s it has seen when it has reached the border between\nthe 0s and the 1s. Similarly the language\nf0n:nis a prime number g\nshould be nonregular, because the prime numbers do not seem to have any\nrepetitive structure that can be used by a DFA. To be more rigorous about\nthis, we will establish a property that all regular languages must possess.\nThis property is called the pumping lemma . If a language does not have this\nproperty, then it must be nonregular.\nThe pumping lemma states that any su\u000eciently long string in a regular\nlanguage can be pumped , i.e., there is a section in that string that can be\nrepeated any number of times, so that the resulting strings are all in the\nlanguage.\nTheorem 2.9.1 (Pumping Lemma for Regular Languages) LetAbe\na regular language. Then there exists an integer p\u00151, called the pumping\nlength , such that the following holds: Every string sinA, withjsj\u0015p, can\nbe written as s=xyz, such that\n1.y6=\u000f(i.e.,jyj\u00151),\n2.jxyj\u0014p, and\n3. for alli\u00150,xyiz2A.\nIn words, the pumping lemma states that by replacing the portion yins\nby zero or more copies of it, the resulting string is still in the language A.\nProof. Let \u0006 be the alphabet of A. SinceAis a regular language, there\nexists a DFA M= (Q;\u0006;\u000e;q;F ) that accepts A. We de\fne pto be the\nnumber of states in Q.\nLets=s1s2:::snbe an arbitrary string in Asuch thatn\u0015p. De\fne\nr1=q,r2=\u000e(r1;s1),r3=\u000e(r2;s2),:::,rn+1=\u000e(rn;sn). Thus, when the\nDFAMreads the string sfrom left to right, it visits the states r1;r2;:::;rn+1.\nSincesis a string in A, we know that rn+1belongs toF.\nConsider the \frst p+ 1 statesr1;r2;:::;rp+1in this sequence. Since the\nnumber of states of Mis equal to p, the pigeonhole principle implies that\nthere must be a state that occurs twice in this sequence. That is, there are\nindicesjand`such that 1\u0014j <`\u0014p+ 1 andrj=r`."
            }
        ]
    },
    {
        "section": "Page 78",
        "content": [
            {
                "type": "text",
                "text": "70 Chapter 2. Finite Automata and Regular Languages\nq=r1\nrn+1rj=rℓread x\nread y\nread z\nWe de\fnex=s1s2:::sj\u00001,y=sj:::s`\u00001, andz=s`:::sn. Sincej <` ,\nwe havey6=\u000f, proving the \frst claim in the theorem. Since `\u0014p+ 1, we\nhavejxyj=`\u00001\u0014p, proving the second claim in the theorem. To see that\nthe third claim also holds, recall that the string s=xyzis accepted by M.\nWhile reading x,Mmoves from the start state qto staterj. While reading\ny, it moves from state rjto stater`=rj, i.e., after having read y,Mis again\nin staterj. While reading z,Mmoves from state rjto the accept state rn+1.\nTherefore, the substring ycan be repeated any number i\u00150 of times, and\nthe corresponding string xyizwill still be accepted by M. It follows that\nxyiz2Afor alli\u00150.\n2.9.1 Applications of the pumping lemma\nFirst example\nConsider the language\nA=f0n1n:n\u00150g:\nWe will prove by contradiction that Ais not a regular language.\nAssume that Ais a regular language. Let p\u00151 be the pumping length,\nas given by the pumping lemma. Consider the string s= 0p1p. It is clear\nthats2Aandjsj= 2p\u0015p. Hence, by the pumping lemma, scan be\nwritten ass=xyz, wherey6=\u000f,jxyj\u0014p, andxyiz2Afor alli\u00150.\nObserve that, since jxyj\u0014p, the string ycontains only 0s. Moreover,\nsincey6=\u000f,ycontains at least one 0. But now we are in trouble: None of\nthe strings xy0z=xz,xy2z=xyyz ,xy3z=xyyyz , . . . , is contained in A.\nHowever, by the pumping lemma, all these strings must be in A. Hence, we\nhave a contradiction and we conclude that Ais not a regular language."
            }
        ]
    },
    {
        "section": "Page 79",
        "content": [
            {
                "type": "text",
                "text": "2.9. The pumping lemma and nonregular languages 71\nSecond example\nConsider the language\nA=fw2f0;1g\u0003: the number of 0s in wequals the number of 1s in wg:\nAgain, we prove by contradiction that Ais not a regular language.\nAssume that Ais a regular language. Let p\u00151 be the pumping length,\nas given by the pumping lemma. Consider the string s= 0p1p. Thens2A\nandjsj= 2p\u0015p. By the pumping lemma, scan be written as s=xyz,\nwherey6=\u000f,jxyj\u0014p, andxyiz2Afor alli\u00150.\nSincejxyj\u0014p, the string ycontains only 0s. Since y6=\u000f,ycontains at\nleast one 0. Therefore, the string xy2z=xyyz contains more 0s than 1s,\nwhich implies that this string is not contained in A. But, by the pumping\nlemma, this string is contained in A. This is a contradiction and, therefore,\nAis not a regular language.\nThird example\nConsider the language\nA=fww:w2f0;1g\u0003g:\nWe prove by contradiction that Ais not a regular language.\nAssume that Ais a regular language. Let p\u00151 be the pumping length,\nas given by the pumping lemma. Consider the string s= 0p10p1. Thens2A\nandjsj= 2p+ 2\u0015p. By the pumping lemma, scan be written as s=xyz,\nwherey6=\u000f,jxyj\u0014p, andxyiz2Afor alli\u00150.\nSincejxyj\u0014p, the string ycontains only 0s. Since y6=\u000f,ycontains at\nleast one 0. Therefore, the string xy2z=xyyz is not contained in A. But,\nby the pumping lemma, this string is contained in A. This is a contradiction\nand, therefore, Ais not a regular language.\nYou should convince yourself that by choosing s= 02p(which is a string\ninAwhose length is at least p), we do not obtain a contradiction. The reason\nis that the string ymay have an even length. Thus, 02pis the \\wrong\" string\nfor showing that Ais not regular. By choosing s= 0p10p1, we do obtain\na contradiction; thus, this is the \\correct\" string for showing that Ais not\nregular."
            }
        ]
    },
    {
        "section": "Page 80",
        "content": [
            {
                "type": "text",
                "text": "72 Chapter 2. Finite Automata and Regular Languages\nFourth example\nConsider the language\nA=f0m1n:m>n\u00150g:\nWe prove by contradiction that Ais not a regular language.\nAssume that Ais a regular language. Let p\u00151 be the pumping length,\nas given by the pumping lemma. Consider the string s= 0p+11p. Thens2A\nandjsj= 2p+ 1\u0015p. By the pumping lemma, scan be written as s=xyz,\nwherey6=\u000f,jxyj\u0014p, andxyiz2Afor alli\u00150.\nSincejxyj\u0014p, the string ycontains only 0s. Since y6=\u000f,ycontains at\nleast one 0. Consider the string xy0z=xz. The number of 1s in this string\nis equal top, whereas the number of 0s is at most equal to p. Therefore, the\nstringxy0zis not contained in A. But, by the pumping lemma, this string\nis contained in A. This is a contradiction and, therefore, Ais not a regular\nlanguage.\nFifth example\nConsider the language\nA=f1n2:n\u00150g:\nWe prove by contradiction that Ais not a regular language.\nAssume that Ais a regular language. Let p\u00151 be the pumping length,\nas given by the pumping lemma. Consider the string s= 1p2. Thens2A\nandjsj=p2\u0015p. By the pumping lemma, scan be written as s=xyz,\nwherey6=\u000f,jxyj\u0014p, andxyiz2Afor alli\u00150.\nObserve that\njsj=jxyzj=p2\nand\njxy2zj=jxyyzj=jxyzj+jyj=p2+jyj:\nSincejxyj\u0014p, we havejyj\u0014p. Sincey6=\u000f, we havejyj\u00151. It follows that\np2<jxy2zj\u0014p2+p<(p+ 1)2:\nHence, the length of the string xy2zis strictly between two consecutive\nsquares. It follows that this length is not a square and, therefore, xy2z\nis not contained in A. But, by the pumping lemma, this string is contained\ninA. This is a contradiction and, therefore, Ais not a regular language."
            }
        ]
    },
    {
        "section": "Page 81",
        "content": [
            {
                "type": "text",
                "text": "2.9. The pumping lemma and nonregular languages 73\nSixth example\nConsider the language\nA=f1n:nis a prime number g:\nWe prove by contradiction that Ais not a regular language.\nAssume that Ais a regular language. Let p\u00151 be the pumping length,\nas given by the pumping lemma. Let n\u0015pbe a prime number, and consider\nthe strings= 1n. Thens2Aandjsj=n\u0015p. By the pumping lemma, s\ncan be written as s=xyz, wherey6=\u000f,jxyj\u0014p, andxyiz2Afor alli\u00150.\nLetkbe the integer such that y= 1k. Sincey6=\u000f, we havek\u00151. For\neachi\u00150,n+ (i\u00001)kis a prime number, because xyiz= 1n+(i\u00001)k2A.\nFori=n+ 1, however, we have\nn+ (i\u00001)k=n+nk=n(1 +k);\nwhich is not a prime number, because n\u00152 and 1 + k\u00152. This is a\ncontradiction and, therefore, Ais not a regular language.\nSeventh example\nConsider the language\nA=fw2f0;1g\u0003: the number of occurrences of 01 in wis equal to\nthe number of occurrences of 10 in wg.\nSince this language has the same \ravor as the one in the second example,\nwe may suspect that Ais not a regular language. This is, however, not true:\nAs we will show, Ais a regular language.\nThe key property is the following one: Let wbe an arbitrary string in\nf0;1g\u0003. Then\nthe absolute value of the number of occurrences of 01 in wminus\nthe number of occurrences of 10 in wis at most one.\nThis property holds, because between any two consecutive occurrences of\n01, there must be exactly one occurrence of 10. Similarly, between any two\nconsecutive occurrences of 10, there must be exactly one occurrence of 01.\nWe will construct a DFA that accepts A. This DFA uses the following\n\fve states:"
            }
        ]
    },
    {
        "section": "Page 82",
        "content": [
            {
                "type": "text",
                "text": "74 Chapter 2. Finite Automata and Regular Languages\n\u000fq: start state; no symbol has been read.\n\u000fq01: the last symbol read was 1; in the part of the string read so far, the\nnumber of occurrences of 01 is one more than the number of occurrences\nof 10.\n\u000fq10: the last symbol read was 0; in the part of the string read so far, the\nnumber of occurrences of 10 is one more than the number of occurrences\nof 01.\n\u000fq0\nequal: the last symbol read was 0; in the part of the string read so far,\nthe number of occurrences of 01 is equal to the number of occurrences\nof 10.\n\u000fq1\nequal: the last symbol read was 1; in the part of the string read so far,\nthe number of occurrences of 01 is equal to the number of occurrences\nof 10.\nThe set of accept states is equal to fq;q0\nequal;q1\nequalg. The state diagram of\nthe DFA is given below.\nq0\nequal\nq1\nequalq01\nq10q00\n1\n1\n0\n1\n0\n0\n1\n1\nIn fact, the key property mentioned above implies that the language A\nconsists of the empty string \u000fand all non-empty binary strings that start"
            }
        ]
    },
    {
        "section": "Page 83",
        "content": [
            {
                "type": "text",
                "text": "2.9. The pumping lemma and nonregular languages 75\nand end with the same symbol. As a result, Ais the language described by\nthe regular expression\n\u000f[0[1[0(0[1)\u00030[1(0[1)\u00031:\nThis gives an alternative proof for the fact that Ais a regular language.\nEighth example\nConsider the language\nL=fw2f0;1g\u0003:wis the binary representation of a prime number g:\nWe assume that for any positive integer, the leftmost bit in its binary repre-\nsentation is 1. In other words, we assume that there are no 0's added to the\nleft of such a binary representation. Thus,\nL=f10;11;101;111;1011;1101;10001;:::g:\nWe will prove that Lis not a regular language.\nAssume that Lis a regular language. Let p\u00151 be the pumping length.\nLetN > 2pbe a prime number and let s2f0;1g\u0003be the binary representa-\ntion ofN. Observe thatjsj\u0015p+ 1. Also, the leftmost and rightmost bits of\nsare 1.\nSinces2Landjsj\u0015p+ 1\u0015p, the Pumping Lemma implies that we\ncan writes=xyz, such that\n1.jyj\u00151,\n2.jxyj\u0014p(and, thus,jzj\u00151), and\n3. for alli\u00150,xyiz2L, i.e.,xyizis the binary representation of a prime\nnumber.\nDe\fneA,B, andCto be the integers whose binary representations are\nx,y, andz, respectively. Note that both yandzmay have leading 0's. In\nfact,ymay be a string consisting of 0's only, in which case B= 0. However,\nsince the rightmost bit of zis 1, we have C\u00151. Observe that\nN=C+B\u00012jzj+A\u00012jzj+jyj: (2.4)"
            }
        ]
    },
    {
        "section": "Page 84",
        "content": [
            {
                "type": "text",
                "text": "76 Chapter 2. Finite Automata and Regular Languages\nLeti=N, consider the bitstring xyiz=xyNz, and letMbe the prime\nnumber whose binary representation is given by this bitstring. Then,\nM=C+N\u00001X\nk=0B\u00012jzj+kjyj+A\u00012jzj+Njyj\n=C+B\u00012jzjN\u00001X\nk=02kjyj+A\u00012jzj+Njyj:\nLet\nT=N\u00001X\nk=02kjyj:\nThen\u0000\n2jyj\u00001\u0001\nT= 2Njyj\u00001: (2.5)\nBy Fermat's Little Theorem, we have\n2N\u00112 (modN);\nimplying that\n2Njyj\u00001 =\u0000\n2N\u0001jyj\u00001\u00112jyj\u00001 (modN):\nThus, (2.5) implies that\n\u0000\n2jyj\u00001\u0001\nT\u00112jyj\u00001 (modN): (2.6)\nObserve that 2jyj\u00142p<N, becausejyj\u0014jxyj\u0014p. Also, 2jyj\u00152, because\ny6=\u000f. It follows that\n1\u00142jyj\u00001<N;\nimplying that\n2jyj\u000016\u00110 (modN):\nThis, together with (2.6), implies that\nT\u00111 (modN):\nSince\nM=C+B\u00012jzj\u0001T+A\u00012jzj+Njyj;"
            }
        ]
    },
    {
        "section": "Page 85",
        "content": [
            {
                "type": "text",
                "text": "2.10. Higman's Theorem 77\nit follows that\nM\u0011C+B\u00012jzj+A\u00012jzj+jyj(modN):\nThis, together with (2.4), implies that\nM\u00110 (modN);\ni.e.,NdividesM. SinceM >N , we conclude that Mis not a prime number,\nwhich is a contradiction. Thus, the language Lis not regular.\n2.10 Higman's Theorem\nLet \u0006 be a \fnite alphabet. For any two strings xandyin \u0006\u0003, we say that x\nis asubsequence ofy, ifxcan be obtained by deleting zero or more symbols\nfromy. For example, 10110 is a subsequence of 0010010101010001. For any\nlanguageL\u0012\u0006\u0003, we de\fne\nSUBSEQ (L) :=fx: there exists a y2Lsuch thatxis a subsequence of yg:\nThat is, SUBSEQ (L) is the language consisting of the subsequences of all\nstrings inL. In 1952, Higman proved the following result:\nTheorem 2.10.1 (Higman) For any \fnite alphabet \u0006and for any lan-\nguageL\u0012\u0006\u0003, the language SUBSEQ (L)is regular.\n2.10.1 Dickson's Theorem\nOur proof of Higman's Theorem will use a theorem that was proved in 1913\nby Dickson.\nRecall that Ndenotes the set of positive integers. Let n2N. For any\ntwo pointsp= (p1;p2;:::;pn) andq= (q1;q2;:::;qn) inNn, we say that pis\ndominated byq, ifpi\u0014qifor alliwith 1\u0014i\u0014n.\nTheorem 2.10.2 (Dickson) LetS\u0012Nn, and letMbe the set consisting of\nall elements of Sthat are minimal in the relation \\is dominated by\". Thus,\nM=fq2S:there is no pinSnfqgsuch thatpis dominated by qg:\nThen, the set Mis \fnite."
            }
        ]
    },
    {
        "section": "Page 86",
        "content": [
            {
                "type": "text",
                "text": "78 Chapter 2. Finite Automata and Regular Languages\nWe will prove this theorem by induction on the dimension n. Ifn= 1,\nthen either M=;(ifS=;) orMconsists of exactly one element (if S6=;).\nTherefore, the theorem holds if n= 1. Letn\u00152 and assume the theorem\nholds for all subsets of Nn\u00001. LetSbe a subset of Nnand consider the set\nMof minimal elements in S. IfS=;, thenM=;and, thus,Mis \fnite.\nAssume that S6=;. We \fx an arbitrary element qinM. Ifp2Mnfqg,\nthenqis not dominated by p. Therefore, there exists an index isuch that\npi\u0014qi\u00001. It follows that\nMnfqg\u0012n[\ni=1\u0000\nNi\u00001\u0002[1;qi\u00001]\u0002Nn\u0000i\u0001\n:\nFor alliandkwith 1\u0014i\u0014nand 1\u0014k\u0014qi\u00001, we de\fne\nSik=fp2S:pi=kg\nand\nMik=fp2M:pi=kg:\nThen,\nMnfqg=n[\ni=1qi\u00001[\nk=1Mik: (2.7)\nLemma 2.10.3 Mikis a subset of the set of all elements of Sikthat are\nminimal in the relation \\is dominated by\".\nProof. Letpbe an element of Mik, and assume that pis not minimal in\nSik. Then there is an element rinSik, such that r6=pandris dominated\nbyp. Sincepandrare both elements of S, it follows that p62M. This is a\ncontradiction.\nSince the set Sikis basically a subset of Nn\u00001, it follows from the induction\nhypothesis that Sikcontains \fnitely many minimal elements. This, combined\nwith Lemma 2.10.3, implies that Mikis a \fnite set. Thus, by (2.7), Mnfqg\nis the union of \fnitely many \fnite sets. Therefore, the set Mis \fnite.\n2.10.2 Proof of Higman's Theorem\nWe give the proof of Theorem 2.10.1 for the case when \u0006 = f0;1g. IfL=;\norSUBSEQ (L) =f0;1g\u0003, then SUBSEQ (L) is obviously a regular language."
            }
        ]
    },
    {
        "section": "Page 87",
        "content": [
            {
                "type": "text",
                "text": "2.10. Higman's Theorem 79\nHence, we may assume that Lis non-empty and SUBSEQ (L) is a proper\nsubset off0;1g\u0003.\nWe \fx a string zof length at least two in the complement SUBSEQ (L) of\nthe language SUBSEQ (L). Observe that this is possible, because SUBSEQ (L)\nis an in\fnite language. We insert 0s and 1s into z, such that, in the result-\ning stringz0, 0s and 1s alternate. For example, if z= 0011101011, then\nz0= 01010101010101. Let n=jz0j\u00001, wherejz0jdenotes the length of z0.\nThen,n\u0015jzj\u00001\u00151.\nA (0;1)-alternation in a binary string xis any occurrence of 01 or 10 in x.\nFor example, the string 1101001 contains four (0 ;1)-alternations. We de\fne\nA=fx2f0;1g\u0003:xhas at most nmany (0;1)-alternationsg:\nLemma 2.10.4 SUBSEQ (L)\u0012A.\nProof. Letx2SUBSEQ (L) and assume that x62A. Then,xhas at least\nn+ 1 =jz0jmany (0;1)-alternations and, therefore, z0is a subsequence of x.\nIn particular, zis a subsequence of x. Sincex2SUBSEQ (L), it follows that\nz2SUBSEQ (L), which is a contradiction.\nLemma 2.10.5 SUBSEQ (L) =\u0010\nA\\SUBSEQ (L)\u0011\n[A.\nProof. Follows from Lemma 2.10.4.\nLemma 2.10.6 The language Ais regular.\nProof. The complement AofAis the language consisting of all binary\nstrings with at least n+ 1 many (0 ;1)-alternations. If, for example, n= 3,\nthenAis described by the regular expression\n(00\u000311\u000300\u000311\u00030(0[1)\u0003)[(11\u000300\u000311\u000300\u00031(0[1)\u0003):\nThis should convince you that the claim is true for any value of n.\nFor anyb2f0;1gand for any k\u00150, we de\fne Abkto be the language\nconsisting of all binary strings that start with a band have exactly kmany\n(0;1)-alternations. Then, we have\nA=f\u000fg[ 1[\nb=0n[\nk=0Abk!\n:"
            }
        ]
    },
    {
        "section": "Page 88",
        "content": [
            {
                "type": "text",
                "text": "80 Chapter 2. Finite Automata and Regular Languages\nThus, if we de\fne\nFbk=Abk\\SUBSEQ (L);\nand use the fact that \u000f2SUBSEQ (L) (which is true because L6=;), then\nA\\SUBSEQ (L) =1[\nb=0n[\nk=0Fbk: (2.8)\nFor anyb2f0;1gand for any k\u00150, consider the relation \\is a subse-\nquence of\" on the language Fbk. We de\fne Mbkto be the language consisting\nof all strings in Fbkthat are minimal in this relation. Thus,\nMbk=fx2Fbk: there is no x0inFbknfxgsuch thatx0is a subsequence of xg:\nIt is clear that\nFbk=[\nx2Mbkfy2Fbk:xis a subsequence of yg:\nIfx2Mbk,y2Abk, andxis a subsequence of y, thenymust be in\nSUBSEQ (L) and, therefore, ymust be in Fbk. To prove this, assume that\ny2SUBSEQ (L). Then,x2SUBSEQ (L), contradicting the fact that\nx2Mbk\u0012Fbk\u0012SUBSEQ (L). It follows that\nFbk=[\nx2Mbkfy2Abk:xis a subsequence of yg: (2.9)\nLemma 2.10.7 Letb2f0;1gand0\u0014k\u0014n, and letxbe an element of\nMbk. Then, the language\nfy2Abk:xis a subsequence of yg\nis regular.\nProof. We will prove the claim by means of an example. Assume that b= 1,\nk= 3, andx= 11110001000. Then, the language\nfy2Abk:xis a subsequence of yg\nis described by the regular expression\n11111\u00030000\u000311\u00030000\u0003:\nThis should convince you that the claim is true in general."
            }
        ]
    },
    {
        "section": "Page 89",
        "content": [
            {
                "type": "text",
                "text": "Exercises 81\nLemma 2.10.8 For eachb2f0;1gand each 0\u0014k\u0014n, the setMbkis\n\fnite.\nProof. Again, we will prove the claim by means of an example. Assume\nthatb= 1 andk= 3. Any string in Fbkcan be written as 1a0b1c0d, for some\nintegersa;b;c;d\u00151. Consider the function ':Fbk!N4that is de\fned by\n'(1a0b1c0d) = (a;b;c;d ). Then,'is an injective function, and the following\nis true, for any two strings xandx0inFbk:\nxis a subsequence of x0if and only if '(x) is dominated by '(x0).\nIt follows that the elements of Mbkare in one-to-one correspondence with\nthose elements of '(Fbk) that are minimal in the relation \\is dominated by\".\nThe lemma thus follows from Dickson's Theorem.\nNow we can complete the proof of Higman's Theorem:\n\u000fIt follows from (2.9) and Lemmas 2.10.7 and 2.10.8, that Fbkis the\nunion of \fnitely many regular languages. Therefore, by Theorem 2.3.1,\nFbkis a regular language.\n\u000fIt follows from (2.8) that A\\SUBSEQ (L) is the union of \fnitely many\nregular languages. Therefore, again by Theorem 2.3.1, A\\SUBSEQ (L)\nis a regular language.\n\u000fSinceA\\SUBSEQ (L) is regular and, by Lemma 2.10.6, Ais regular,\nit follows from Lemma 2.10.5 that SUBSEQ (L) is the union of two reg-\nular languages. Therefore, by Theorem 2.3.1, SUBSEQ (L) is a regular\nlanguage.\n\u000fSince SUBSEQ (L) is regular, it follows from Theorem 2.6.4 that the\nlanguage SUBSEQ (L) is regular as well.\nExercises\n2.1For each of the following languages, construct a DFA that accepts the\nlanguage. In all cases, the alphabet is f0;1g.\n1.fw: the length of wis divisible by three g"
            }
        ]
    },
    {
        "section": "Page 90",
        "content": [
            {
                "type": "text",
                "text": "82 Chapter 2. Finite Automata and Regular Languages\n2.fw: 110 is not a substring of wg\n3.fw:wcontains at least \fve 1s g\n4.fw:wcontains the substring 1011 g\n5.fw:wcontains at least two 1s and at most two 0s g\n6.fw:wcontains an odd number of 1s or exactly two 0s g\n7.fw:wbegins with 1 and ends with 0 g\n8.fw: every odd position in wis 1g\n9.fw:whas length at least 3 and its third symbol is 0 g\n10.f\u000f;0g\n2.2For each of the following languages, construct an NFA, with the speci\fed\nnumber of states, that accepts the language. In all cases, the alphabet is\nf0;1g.\n1. The languagefw:wends with 10gwith three states.\n2. The languagefw:wcontains the substring 1011 gwith \fve states.\n3. The languagefw:wcontains an odd number of 1s or exactly two 0s g\nwith six states.\n2.3For each of the following languages, construct an NFA that accepts the\nlanguage. In all cases, the alphabet is f0;1g.\n1.fw:wcontains the substring 11001 g\n2.fw:whas length at least 2 and does not end with 10 g\n3.fw:wbegins with 1 or ends with 0 g\n2.4Convert the following NFA to an equivalent DFA."
            }
        ]
    },
    {
        "section": "Page 91",
        "content": [
            {
                "type": "text",
                "text": "Exercises 83\n1 2a\nba, b\n2.5Convert the following NFA to an equivalent DFA.\n1\n3 2a\na\nb\naε,b\n2.6Convert the following NFA to an equivalent DFA.\n0 1 2 3a, ǫ b a\nǫb\n2.7In the proof of Theorem 2.6.3, we introduced a new start state q0, which\nis also an accept state. Explain why the following is not a valid proof of\nTheorem 2.6.3:\nLetN= (Q1;\u0006;\u000e1;q1;F1) be an NFA, such that A=L(N). De\fne the\nNFAM= (Q1;\u0006;\u000e;q 1;F), where"
            }
        ]
    },
    {
        "section": "Page 92",
        "content": [
            {
                "type": "text",
                "text": "84 Chapter 2. Finite Automata and Regular Languages\n1.F=fq1g[F1.\n2.\u000e:Q1\u0002\u0006\u000f!P (Q1) is de\fned as follows: For any r2Q1and for any\na2\u0006\u000f,\n\u000e(r;a) =8\n<\n:\u000e1(r;a) if r2Q1andr62F1,\n\u000e1(r;a) if r2F1anda6=\u000f,\n\u000e1(r;a)[fq1gifr2F1anda=\u000f.\nThenL(M) =A\u0003.\n2.8Prove Theorem 2.6.4.\n2.9LetAbe a language over the alphabet \u0006 = f0;1gand letAbe the\ncomplement ofA. Thus,Ais the language consisting of all binary strings\nthat are not in A.\nAssume that Ais a regular language. Let M= (Q;\u0006;\u000e;q;F ) be a non-\ndeterministic \fnite automaton (NFA) that accepts A.\nConsider the NFA N= (Q;\u0006;\u000e;q;F), whereF=QnFis the complement\nofF. Thus,Nis obtained from Mby turning all accept states into nonaccept\nstates, and turning all nonaccept states into accept states.\n1. Is it true that the language accepted by Nis equal toA?\n2. Assume now that Mis a deterministic \fnite automaton (DFA) that\nacceptsA. De\fneNas above; thus, turn all accept states into nonac-\ncept states, and turn all nonaccept states into accept states. Is it true\nthat the language accepted by Nis equal toA?\n2.10 Recall the alternative de\fnition for the star of a language Athat we\ngave just before Theorem 2.3.1.\nIn Theorems 2.3.1 and 2.6.2, we have shown that the class of regular\nlanguages is closed under the union and concatenation operations. Since\nA\u0003=S1\nk=0Ak, why doesn't this imply that the class of regular languages is\nclosed under the star operation?\n2.11 LetAandBbe two regular languages over the same alphabet \u0006. Prove\nthat the di\u000berence of AandB, i.e., the language\nAnB=fw:w2Aandw62Bg\nis a regular language."
            }
        ]
    },
    {
        "section": "Page 93",
        "content": [
            {
                "type": "text",
                "text": "Exercises 85\n2.12 For each of the following regular expressions, give two strings that are\nmembers and two strings that are not members of the language described by\nthe expression. The alphabet is \u0006 = fa;bg.\n1.a(ba)\u0003b.\n2. (a[b)\u0003a(a[b)\u0003b(a[b)\u0003a(a[b)\u0003.\n3. (a[ba[bb)(a[b)\u0003.\n2.13 Give regular expressions describing the following languages. In all\ncases, the alphabet is f0;1g.\n1.fw:wcontains at least three 1s g.\n2.fw:wcontains at least two 1s and at most one 0 g,\n3.fw:wcontains an even number of 0s and exactly two 1s g.\n4.fw:wcontains exactly two 0s and at least two 1s g.\n5.fw:wcontains an even number of 0s and each 0 is followed by at least one 1 g.\n6.fw: every odd position in wis 1g.\n2.14 Convert each of the following regular expressions to an NFA.\n1. (0[1)\u0003000(0[1)\u0003\n2. (((10)\u0003(00))[10)\u0003\n3. ((0[1)(11)\u0003[0)\u0003\n2.15 Convert the following DFA to a regular expression."
            }
        ]
    },
    {
        "section": "Page 94",
        "content": [
            {
                "type": "text",
                "text": "86 Chapter 2. Finite Automata and Regular Languages\n1 2\n3a\nab\nbab\n2.16 Convert the following DFA to a regular expression.\n1 2\n3a, ba\nab\nb\n2.17 Convert the following DFA to a regular expression.\na, b\n2.18 1. LetAbe a non-empty regular language. Prove that there exists\nan NFA that accepts Aand that has exactly one accept state."
            }
        ]
    },
    {
        "section": "Page 95",
        "content": [
            {
                "type": "text",
                "text": "Exercises 87\n2. For any string w=w1w2:::wn, we denote by wRthe string obtained\nby readingwbackwards, i.e., wR=wnwn\u00001:::w 2w1. For any language\nA, we de\fne ARto be the language obtained by reading all strings in\nAbackwards, i.e.,\nAR=fwR:w2Ag:\nLetAbe a non-empty regular language. Prove that the language AR\nis also regular.\n2.19 Ifn\u00151 is an integer and w=a1a2:::anis a string, then for any i\nwith 0\u0014i<n , the string a1a2:::aiis called a proper pre\fx ofw. (Ifi= 0,\nthena1a2:::ai=\u000f.)\nFor any language L, we de\fne MIN (L) to be the language\nMIN (L) =fw2L: no proper pre\fx of wbelongs toLg:\nProve the following claim: If Lis a regular language, then MIN (L) is regular\nas well.\n2.20 Use the pumping lemma to prove that the following languages are not\nregular.\n1.fanbmcn+m:n\u00150;m\u00150g.\n2.fanbnc2n:n\u00150g.\n3.fanbman:n\u00150;m\u00150g.\n4.fa2n:n\u00150g. (Remark: a2nis the string consisting of 2nmanya's.)\n5.fanbmck:n\u00150;m\u00150;k\u00150;n2+m2=k2g.\n6.fuvu:u2fa;bg\u0003;u6=\u000f;v2fa;bg\u0003g.\n2.21 Prove that the language\nfambn:m\u00150;n\u00150;m6=ng\nis not regular. (Using the pumping lemma for this one is a bit tricky. You\ncan avoid using the pumping lemma by combining results about the closure\nunder regular operations.)"
            }
        ]
    },
    {
        "section": "Page 96",
        "content": [
            {
                "type": "text",
                "text": "88 Chapter 2. Finite Automata and Regular Languages\n2.22 1. Give an example of a regular language Aand a non-regular lan-\nguageBfor whichA\u0012B.\n2. Give an example of a non-regular language Aand a regular language\nBfor whichA\u0012B.\n2.23 LetAbe a language consisting of \fnitely many strings.\n1. Prove that Ais a regular language.\n2. Letnbe the maximum length of any string in A. Prove that every\ndeterministic \fnite automaton (DFA) that accepts Ahas at least n+ 1\nstates. ( Hint: How is the pumping length chosen in the proof of the\npumping lemma?)\n2.24 LetLbe a regular language, let Mbe a DFA whose language is equal\ntoL, and letpbe the number of states of M. Prove that L6=;if and only\nifLcontains a string of length less than p.\n2.25 LetLbe a regular language, let Mbe a DFA whose language is equal\ntoL, and letpbe the number of states of M. Prove that Lis an in\fnite\nlanguage if and only if Lcontains a string wwithp\u0014jwj\u00142p\u00001.\n2.26 Let \u0006 be a non-empty alphabet, and let Lbe a language over \u0006, i.e.,\nL\u0012\u0006\u0003. We de\fne a binary relation RLon \u0006\u0003\u0002\u0006\u0003, in the following way:\nFor any two strings uandu0in \u0006\u0003,\nuRLu0if and only if (8v2\u0006\u0003:uv2L,u0v2L):\nProve that RLis an equivalence relation.\n2.27 Let \u0006 =f0;1g, let\nL=fw2\u0006\u0003:jwjis oddg;\nand consider the relation RLde\fned in Exercise 2.26.\n1. Prove that for any two strings uandu0in \u0006\u0003,\nuRLu0,juj\u0000ju0jis even."
            }
        ]
    },
    {
        "section": "Page 97",
        "content": [
            {
                "type": "text",
                "text": "Exercises 89\n2. Determine all equivalence classes of the relation RL.\n2.28 Let \u0006 be a non-empty alphabet, and let Lbe a language over \u0006, i.e.,\nL\u0012\u0006\u0003. Recall the equivalence relation RLthat was de\fned in Exercise 2.26.\n1. Assume that Lis a regular language, and let M= (Q;\u0006;\u000e;q 0;F) be\na DFA that accepts L. Letuandu0be strings in \u0006\u0003. Letqbe the\nstate reached, when following the path in the state diagram of M, that\nstarts inq0and that is obtained by reading the string u. Similarly, let\nq0be the state reached, when following the path in the state diagram\nofM, that starts in q0and that is obtained by reading the string u0.\nProve the following: If q=q0, thenuRLu0.\n2. Prove the following claim: If Lis a regular language, then the equiva-\nlence relation RLhas a \fnite number of equivalence classes.\n2.29 LetLbe the language de\fned by\nL=fuuR:u2f0;1g\u0003g:\nIn words, a string is in Lif and only if its length is even, and the second half\nis the reverse of the \frst half. Consider the equivalence relation RLthat was\nde\fned in Exercise 2.26.\n1. Letmandnbe two distinct positive integers and consider the two\nstringsu= 0m1 andu0= 0n1. Prove that:(uRLu0).\n2. Prove that Lis not a regular language, without using the pumping\nlemma.\n3. Use the pumping lemma to prove that Lis not a regular language.\n2.30 In this exercise, we will show that the converse of the pumping lemma\ndoes, in general, not hold. Consider the language\nA=fambncn:m\u00151;n\u00150g[fbnck:n\u00150;k\u00150g:\n1. Show that Asatis\fes the conclusion of the pumping lemma for p= 1.\nThus, show that every string sinAwhose length is at least pcan be\nwritten ass=xyz, such that y6=\u000f,jxyj\u0014p, andxyiz2Afor all\ni\u00150."
            }
        ]
    },
    {
        "section": "Page 98",
        "content": [
            {
                "type": "text",
                "text": "90 Chapter 2. Finite Automata and Regular Languages\n2. Consider the equivalence relation RAthat was de\fned in Exercise 2.26.\nLetnandn0be two distinct non-negative integers and consider the two\nstringsu=abnandu0=abn0. Prove that:(uRAu0).\n3. Prove that Ais not a regular language."
            }
        ]
    },
    {
        "section": "Page 99",
        "content": [
            {
                "type": "text",
                "text": "Chapter 3\nContext-Free Languages\nIn this chapter, we introduce the class of context-free languages. As we\nwill see, this class contains all regular languages, as well as some nonregular\nlanguages such as f0n1n:n\u00150g.\nThe class of context-free languages consists of languages that have some\nsort of recursive structure. We will see two equivalent methods to obtain this\nclass. We start with context-free grammars, which are used for de\fning the\nsyntax of programming languages and their compilation. Then we introduce\nthe notion of (nondeterministic) pushdown automata, and show that these\nautomata have the same power as context-free grammars.\n3.1 Context-free grammars\nWe start with an example. Consider the following \fve (substitution) rules:\nS!AB\nA!a\nA!aA\nB!b\nB!bB\nHere,S,A, andBarevariables ,Sis the start variable , andaandbare\nterminals . We use these rules to derive strings consisting of terminals (i.e.,\nelements offa;bg\u0003), in the following manner:\n1. Initialize the current string to be the string consisting of the start\nvariableS."
            }
        ]
    },
    {
        "section": "Page 100",
        "content": [
            {
                "type": "text",
                "text": "92 Chapter 3. Context-Free Languages\n2. Take any variable in the current string and take any rule that has this\nvariable on the left-hand side. Then, in the current string, replace this\nvariable by the right-hand side of the rule.\n3. Repeat 2. until the current string only contains terminals.\nFor example, the string aaaabb can be derived in the following way:\nS)AB\n)aAB\n)aAbB\n)aaAbB\n)aaaAbB\n)aaaabB\n)aaaabb\nThis derivation can also be represented using a parse tree , as in the \fgure\nbelow:\nS\nA\nA\nA\nAa\na\naab\nbB\nB\nThe \fve rules in this example constitute a context-free grammar. The\nlanguage of this grammar is the set of all strings that"
            }
        ]
    },
    {
        "section": "Page 101",
        "content": [
            {
                "type": "text",
                "text": "3.1. Context-free grammars 93\n\u000fcan be derived from the start variable and\n\u000fonly contain terminals.\nFor this example, the language is\nfambn:m\u00151;n\u00151g;\nbecause every string of the form ambn, for somem\u00151 andn\u00151, can be\nderived from the start variable, whereas no other string over the alphabet\nfa;bgcan be derived from the start variable.\nDe\fnition 3.1.1 A context-free grammar is a 4-tuple G= (V;\u0006;R;S ),\nwhere\n1.Vis a \fnite set, whose elements are called variables ,\n2. \u0006 is a \fnite set, whose elements are called terminals ,\n3.V\\\u0006 =;,\n4.Sis an element of V; it is called the start variable ,\n5.Ris a \fnite set, whose elements are called rules. Each rule has the\nformA!w, whereA2Vandw2(V[\u0006)\u0003.\nIn our example, we have V=fS;A;Bg, \u0006 =fa;bg, and\nR=fS!AB;A!a;A!aA;B!b;B!bBg:\nDe\fnition 3.1.2 LetG= (V;\u0006;R;S ) be a context-free grammar. Let Abe\nan element in Vand letu,v, andwbe strings in ( V[\u0006)\u0003such thatA!w\nis a rule in R. We say that the string uwv can be derived in one step from\nthe stringuAv, and write this as\nuAv)uwv:\nIn other words, by applying the rule A!wto the string uAv, we obtain\nthe stringuwv. In our example, we see that aaAbb)aaaAbb .\nDe\fnition 3.1.3 LetG= (V;\u0006;R;S ) be a context-free grammar. Let u\nandvbe strings in ( V[\u0006)\u0003. We say that vcan be derived from u, and write\nthis asu\u0003)v, if one of the following two conditions holds:"
            }
        ]
    },
    {
        "section": "Page 102",
        "content": [
            {
                "type": "text",
                "text": "94 Chapter 3. Context-Free Languages\n1.u=vor\n2. there exist an integer k\u00152 and a sequence u1;u2;:::;ukof strings in\n(V[\u0006)\u0003, such that\n(a)u=u1,\n(b)v=uk, and\n(c)u1)u2):::)uk.\nIn other words, by starting with the string uand applying rules zero or\nmore times, we obtain the string v. In our example, we see that aaAbB\u0003)\naaaabbbB .\nDe\fnition 3.1.4 LetG= (V;\u0006;R;S ) be a context-free grammar. The\nlanguage ofGis de\fned to be the set of all strings in \u0006\u0003that can be derived\nfrom the start variable S:\nL(G) =fw2\u0006\u0003:S\u0003)wg:\nDe\fnition 3.1.5 A language Lis called context-free , if there exists a context-\nfree grammar Gsuch thatL(G) =L.\n3.2 Examples of context-free grammars\n3.2.1 Properly nested parentheses\nConsider the context-free grammar G= (V;\u0006;R;S ), whereV=fSg, \u0006 =\nfa;bg, and\nR=fS!\u000f;S!aSb;S!SSg:\nWe write the three rules in Ras\nS!\u000fjaSbjSS;\nwhere you can think of \\ j\" as being a short-hand for \\or\"."
            }
        ]
    },
    {
        "section": "Page 103",
        "content": [
            {
                "type": "text",
                "text": "3.2. Examples of context-free grammars 95\nBy applying the rules in R, starting with the start variable S, we obtain,\nfor example,\nS)SS\n)aSbS\n)aSbSS\n)aSSbSS\n)aaSbSbSS\n)aabSbSS\n)aabbSS\n)aabbaSbS\n)aabbabS\n)aabbabaSb\n)aabbabab\nWhat is the language L(G) of this context-free grammar G? If we think\nofaas being a left-parenthesis \\(\", and of bas being a right-parenthesis \\)\",\nthenL(G) is the language consisting of all strings of properly nested paren-\ntheses. Here is the explanation: Any string of properly nested parentheses is\neither\n\u000fempty (which we derive from Sby the rule S!\u000f),\n\u000fconsists of a left-parenthesis, followed by an arbitrary string of properly\nnested parentheses, followed by a right-parenthesis (these are derived\nfromSby \frst applying the rule S!aSb), or\n\u000fconsists of an arbitrary string of properly nested parentheses, followed\nby an arbitrary string of properly nested parentheses (these are derived\nfromSby \frst applying the rule S!SS).\n3.2.2 A context-free grammar for a nonregular lan-\nguage\nConsider the language L1=f0n1n:n\u00150g. We have seen in Section 2.9.1\nthatL1is not a regular language. We claim that L1is a context-free language."
            }
        ]
    },
    {
        "section": "Page 104",
        "content": [
            {
                "type": "text",
                "text": "96 Chapter 3. Context-Free Languages\nIn order to prove this claim, we have to construct a context-free grammar\nG1such thatL(G1) =L1.\nObserve that any string in L1is either\n\u000fempty or\n\u000fconsists of a 0, followed by an arbitrary string in L1, followed by a 1.\nThis leads to the context-free grammar G1= (V1;\u0006;R1;S1), whereV1=\nfS1g, \u0006 =f0;1g, andR1consists of the rules\nS1!\u000fj0S11:\nHence,R1=fS1!\u000f;S1!0S11g.\nTo derive the string 0n1nfrom the start variable S1, we do the following:\n\u000fStarting with S1, apply the rule S1!0S11 exactlyntimes. This gives\nthe string 0nS11n.\n\u000fApply the rule S1!\u000f. This gives the string 0n1n.\nIt is not di\u000ecult to see that these are the only strings that can be derived\nfrom the start variable S1. Thus,L(G1) =L1.\nIn a symmetric way, we see that the context-free grammar G2= (V2;\u0006;R2;S2),\nwhereV2=fS2g, \u0006 =f0;1g, andR2consists of the rules\nS2!\u000fj1S20;\nhas the property that L(G2) =L2, whereL2=f1n0n:n\u00150g. Thus,L2is\na context-free language.\nDe\fneL=L1[L2, i.e.,\nL=f0n1n:n\u00150g[f 1n0n:n\u00150g:\nThe context-free grammar G= (V;\u0006;R;S ), whereV=fS;S 1;S2g, \u0006 =\nf0;1g, andRconsists of the rules\nS!S1jS2\nS1!\u000fj0S11\nS2!\u000fj1S20;\nhas the property that L(G) =L. Hence,Lis a context-free language."
            }
        ]
    },
    {
        "section": "Page 105",
        "content": [
            {
                "type": "text",
                "text": "3.2. Examples of context-free grammars 97\n3.2.3 A context-free grammar for the complement of\na nonregular language\nLetLbe the (nonregular) language L=f0n1n:n\u00150g. We want to prove\nthat the complement LofLis a context-free language. Hence, we want to\nconstruct a context-free grammar Gwhose language is equal to L. Observe\nthat a binary string wis inLif and only if\n1.w= 0m1n, for some integers mandnwith 0\u0014m<n , or\n2.w= 0m1n, for some integers mandnwith 0\u0014n<m , or\n3.wcontains 10 as a substring.\nThus, we can write Las the union of the languages of all strings of type 1.,\ntype 2., and type 3.\nAny string of type 1. is either\n\u000fthe string 1,\n\u000fconsists of a string of type 1., followed by one 1, or\n\u000fconsists of one 0, followed by an arbitrary string of type 1., followed by\none 1.\nThus, using the rules\nS1!1jS11j0S11;\nwe can derive, from S1, all strings of type 1.\nSimilarly, using the rules\nS2!0j0S2j0S21;\nwe can derive, from S2, all strings of type 2.\nAny string of type 3.\n\u000fconsists of an arbitrary binary string, followed by the string 10, followed\nby an arbitrary binary string.\nUsing the rules\nX!\u000fj0Xj1X;"
            }
        ]
    },
    {
        "section": "Page 106",
        "content": [
            {
                "type": "text",
                "text": "98 Chapter 3. Context-Free Languages\nwe can derive, from X, all binary strings. Thus, by combining these with\nthe rule\nS3!X10X;\nwe can derive, from S3, all strings of type 3.\nWe arrive at the context-free grammar G= (V;\u0006;R;S ), whereV=\nfS;S 1;S2;S3;Xg, \u0006 =f0;1g, andRconsists of the rules\nS!S1jS2jS3\nS1!1jS11j0S11\nS2!0j0S2j0S21\nS3!X10X\nX!\u000fj0Xj1X\nTo summarize, we have\nS1\u0003)0m1n;for all integers mandnwith 0\u0014m<n ,\nS2\u0003)0m1n;for all integers mandnwith 0\u0014n<m ,\nX\u0003)u;for each string uinf0;1g\u0003,\nand\nS3\u0003)w;for every binary string wthat contains 10 as a substring.\nFrom these observations, it follows that that L(G) =L.\n3.2.4 A context-free grammar that veri\fes addition\nConsider the language\nL=fanbmcn+m:n\u00150;m\u00150g:\nUsing the pumping lemma for regular languages (Theorem 2.9.1), it can\nbe shown that Lis not a regular language. We will construct a context-\nfree grammar Gwhose language is equal to L, thereby proving that Lis a\ncontext-free language.\nFirst observe that \u000f2L. Therefore, we will take S!\u000fto be one of the\nrules in the grammar.\nLet us see how we can derive all strings in Lfrom the start variable S:"
            }
        ]
    },
    {
        "section": "Page 107",
        "content": [
            {
                "type": "text",
                "text": "3.3. Regular languages are context-free 99\n1. Every time we add an a, we also add a c. In this way, we obtain all\nstrings of the form ancn, wheren\u00150.\n2. Given a string of the form ancn, we start adding bs. Every time we add\nab, we also add a c. Observe that every bhas to be added between\ntheas and thecs. Therefore, we use a variable Bas a \\pointer\" to\nthe position in the current string where a bcan be added: Instead of\nderivingancnfromS, we derive the string anBcn. Then, from B, we\nderive all strings of the form bmcm, wherem\u00150.\nWe obtain the context-free grammar G= (V;\u0006;R;S ), whereV=fS;A;Bg,\n\u0006 =fa;b;cg, andRconsists of the rules\nS!\u000fjA\nA!\u000fjaAcjB\nB!\u000fjbBc\nThe facts that\n\u000fA\u0003)anBcn, for everyn\u00150,\n\u000fB\u0003)bmcm, for everym\u00150,\nimply that the following strings can be derived from the start variable S:\n\u000fS\u0003)anBcn\u0003)anbmcmcn=anbmcn+m, for alln\u00150 andm\u00150.\nIn fact, no other strings in fa;b;cg\u0003can be derived from S. Therefore, we\nhaveL(G) =L. Since\nS)A)B)\u000f;\nwe can simplify this grammar G, by eliminating the rules S!\u000fandA!\u000f.\nThis gives the context-free grammar G0= (V;\u0006;R0;S), whereV=fS;A;Bg,\n\u0006 =fa;b;cg, andR0consists of the rules\nS!A\nA!aAcjB\nB!\u000fjbBc\nFinally, observe that we do not need S; instead, we can use Aas start\nvariable. This gives our \fnal context-free grammar G00= (V;\u0006;R00;A), where\nV=fA;Bg, \u0006 =fa;b;cg, andR00consists of the rules\nA!aAcjB\nB!\u000fjbBc"
            }
        ]
    },
    {
        "section": "Page 108",
        "content": [
            {
                "type": "text",
                "text": "100 Chapter 3. Context-Free Languages\n3.3 Regular languages are context-free\nWe mentioned already that the class of context-free languages includes the\nclass of regular languages. In this section, we will prove this claim.\nTheorem 3.3.1 Let\u0006be an alphabet and let L\u0012\u0006\u0003be a regular language.\nThenLis a context-free language.\nProof. SinceLis a regular language, there exists a deterministic \fnite\nautomaton M= (Q;\u0006;\u000e;q;F ) that accepts L.\nTo prove that Lis context-free, we have to de\fne a context-free grammar\nG= (V;\u0006;R;S ), such that L=L(M) =L(G). Thus,Gmust have the\nfollowing property: For every string w2\u0006\u0003,\nw2L(M) if and only if w2L(G),\nwhich can be reformulated as\nMacceptswif and only if S\u0003)w.\nWe will de\fne the context-free grammar Gin such a way that the following\ncorrespondence holds for any string w=w1w2:::wn:\n\u000fAssume that Mis in state Ajust after it has read the substring\nw1w2:::wi.\n\u000fThen in the context-free grammar G, we haveS\u0003)w1w2:::wiA.\nIn the next step, Mreads the symbol wi+1and switches from state Ato,\nsay, stateB; thus,\u000e(A;wi+1) =B. In order to guarantee that the above\ncorrespondence still holds, we have to add the rule A!wi+1BtoG.\nConsider the moment when Mhas read the entire string w. LetAbe the\nstateMis in at that moment. By the above correspondence, we have\nS\u0003)w1w2:::wnA=wA:\nRecall that Gmust have the property that\nMacceptswif and only if S\u0003)w,\nwhich is equivalent to\nA2Fif and only if S\u0003)w."
            }
        ]
    },
    {
        "section": "Page 109",
        "content": [
            {
                "type": "text",
                "text": "3.3. Regular languages are context-free 101\nWe guarantee this property by adding to Gthe ruleA!\u000ffor every accept\nstateAofM.\nWe are now ready to give the formal de\fnition of the context-free gram-\nmarG= (V;\u0006;R;S ):\n\u000fV=Q, i.e., the variables of Gare the states of M.\n\u000fS=q, i.e., the start variable of Gis the start state of M.\n\u000fRconsists of the rules\nA!aB; whereA2Q,a2\u0006,B2Q, and\u000e(A;a) =B;\nand\nA!\u000f;whereA2F.\nIn words,\n\u000fevery transition \u000e(A;a) =BofM(i.e., when Mis in the state Aand\nreads the symbol a, it switches to the state B) corresponds to a rule\nA!aBin the grammar G,\n\u000fevery accept state AofMcorresponds to a rule A!\u000fin the grammar\nG.\nWe claim that L(G) =L. In order to prove this, we have to show that\nL(G)\u0012LandL\u0012L(G).\nWe prove that L\u0012L(G). Letw=w1w2:::wnbe an arbitrary string\ninL. When the \fnite automaton Mreads the string w, it visits the states\nr0;r1;:::;rn, where\n\u000fr0=q, and\n\u000fri+1=\u000e(ri;wi+1) fori= 0;1;:::;n\u00001.\nSincew2L=L(M), we know that rn2F.\nIt follows from the way we de\fned the grammar Gthat\n\u000ffor eachi= 0;1;:::;n\u00001,ri!wi+1ri+1is a rule in R, and\n\u000frn!\u000fis a rule in R."
            }
        ]
    },
    {
        "section": "Page 110",
        "content": [
            {
                "type": "text",
                "text": "102 Chapter 3. Context-Free Languages\nTherefore, we have\nS=q=r0)w1r1)w1w2r2):::)w1w2:::wnrn)w1w2:::wn=w:\nThis proves that w2L(G).\nThe proof of the claim that L(G)\u0012Lis left as an exercise.\nIn Sections 2.9.1 and 3.2.2, we have seen that the language f0n1n:n\u0015\n0gis not regular, but context-free. Therefore, the class of all context-free\nlanguages properly contains the class of regular languages.\n3.3.1 An example\nLetLbe the language de\fned as\nL=fw2f0;1g\u0003: 101 is a substring of wg:\nIn Section 2.2.2, we have seen that Lis a regular language. In that section,\nwe constructed the following deterministic \fnite automaton Mthat accepts\nL(we have renamed the states):\n0\n11\n00\n10,1S A\nB C\nWe apply the construction given in the proof of Theorem 3.3.1 to convert\nMto a context-free grammar Gwhose language is equal to L. According\nto this construction, we have G= (V;\u0006;R;S ), whereV=fS;A;B;Cg,\n\u0006 =f0;1g, the start variable Sis the start state of M, andRconsists of the\nrules\nS!0Sj1A\nA!0Bj1A\nB!0Sj1C\nC!0Cj1Cj\u000f"
            }
        ]
    },
    {
        "section": "Page 111",
        "content": [
            {
                "type": "text",
                "text": "3.4. Chomsky normal form 103\nConsider the string 010011011, which is an element of L. When the \fnite\nautomaton Mreads this string, it visits the states\nS;S;A;B;S;A;A;B;C;C:\nIn the grammar G, this corresponds to the derivation\nS)0S\n)01A\n)010B\n)0100S\n)01001A\n)010011A\n)0100110B\n)01001101C\n)010011011C\n)010011011:\nHence,\nS\u0003)010011011;\nimplying that the string 010011011 is in the language L(G) of the context-free\ngrammarG.\nThe string 10011 is not in the language L. When the \fnite automaton\nMreads this string, it visits the states\nS;A;B;S;A;A;\ni.e., after the string has been read, Mis in the non-accept state A. In the\ngrammarG, reading the string 10011 corresponds to the derivation\nS)1A\n)10B\n)100S\n)1001A\n)10011A:\nSinceAis not an accept state in M, the grammar Gdoes not contain the\nruleA!\u000f. This implies that the string 10011 cannot be derived from the\nstart variable S. Thus, 10011 is not in the language L(G) ofG."
            }
        ]
    },
    {
        "section": "Page 112",
        "content": [
            {
                "type": "text",
                "text": "104 Chapter 3. Context-Free Languages\n3.4 Chomsky normal form\nThe rules in a context-free grammar G= (V;\u0006;R;S ) are of the form\nA!w;\nwhereAis a variable and wis a string over the alphabet V[\u0006. In this\nsection, we show that every context-free grammar Gcan be converted to a\ncontext-free grammar G0, such that L(G) =L(G0), and the rules of G0are of\na restricted form, as speci\fed in the following de\fnition:\nDe\fnition 3.4.1 A context-free grammar G= (V;\u0006;R;S ) is said to be in\nChomsky normal form , if every rule in Rhas one of the following three forms:\n1.A!BC, whereA,B, andCare elements of V,B6=S, andC6=S.\n2.A!a, whereAis an element of Vandais an element of \u0006.\n3.S!\u000f, whereSis the start variable.\nYou should convince yourself that, for such a grammar, Rcontains the\nruleS!\u000fif and only if \u000f2L(G).\nTheorem 3.4.2 Let\u0006be an alphabet and let L\u0012\u0006\u0003be a context-free lan-\nguage. There exists a context-free grammar in Chomsky normal form, whose\nlanguage is L.\nProof. SinceLis a context-free language, there exists a context-free gram-\nmarG= (V;\u0006;R;S ), such that L(G) =L. We will transform Ginto a\ngrammar that is in Chomsky normal form and whose language is equal to\nL(G). The transformation consists of \fve steps.\nStep 1: Eliminate the start variable from the right-hand side of the rules.\nWe de\fneG1= (V1;\u0006;R1;S1), whereS1is the start variable (which is a\nnew variable), V1=V[fS1g, andR1=R[fS1!Sg. This grammar has\nthe property that\n\u000fthe start variable S1does not occur on the right-hand side of any rule\ninR1, and\n\u000fL(G1) =L(G)."
            }
        ]
    },
    {
        "section": "Page 113",
        "content": [
            {
                "type": "text",
                "text": "3.4. Chomsky normal form 105\nStep 2: An\u000f-ruleis a rule that is of the form A!\u000f, whereAis a variable\nthat is not equal to the start variable. In the second step, we eliminate all\n\u000f-rules from G1.\nWe consider all \u000f-rules, one after another. Let A!\u000fbe one such rule,\nwhereA2V1andA6=S1. We modify G1as follows:\n1. Remove the rule A!\u000ffrom the current set R1.\n2. For each rule in the current set R1that is of the form\n(a)B!A, add the rule B!\u000ftoR1, unless this rule has already\nbeen deleted from R1; observe that in this way, we replace the two-\nstep derivation B)A)\u000fby the one-step derivation B)\u000f;\n(b)B!uAv (whereuandvare strings that are not both empty),\nadd the rule B!uvtoR1; observe that in this way, we replace\nthe two-step derivation B)uAv)uvby the one-step derivation\nB)uv;\n(c)B!uAvAw (whereu,v, andware strings), add the rules B!\nuvw,B!uAvw , andB!uvAw toR1; ifu=v=w=\u000fand\nthe ruleB!\u000fhas already been deleted from R1, then we do not\nadd the rule B!\u000f;\n(d) treat rules in which Aoccurs more than twice on the right-hand\nside in a similar fashion.\nWe repeat this process until all \u000f-rules have been eliminated. Let R2\nbe the set of rules, after all \u000f-rules have been eliminated. We de\fne G2=\n(V2;\u0006;R2;S2), whereV2=V1andS2=S1. This grammar has the property\nthat\n\u000fthe start variable S2does not occur on the right-hand side of any rule\ninR2,\n\u000fR2does not contain any \u000f-rule (it may contain the rule S2!\u000f), and\n\u000fL(G2) =L(G1) =L(G).\nStep 3: Aunit-rule is a rule that is of the form A!B, whereAandBare\nvariables. In the third step, we eliminate all unit-rules from G2."
            }
        ]
    },
    {
        "section": "Page 114",
        "content": [
            {
                "type": "text",
                "text": "106 Chapter 3. Context-Free Languages\nWe consider all unit-rules, one after another. Let A!Bbe one such\nrule, where AandBare elements of V2. We know that B6=S2. We modify\nG2as follows:\n1. Remove the rule A!Bfrom the current set R2.\n2. For each rule in the current set R2that is of the form B!u, where\nu2(V2[\u0006)\u0003, add the rule A!uto the current set R2, unless this is\na unit-rule that has already been eliminated.\nObserve that in this way, we replace the two-step derivation A)B)\nuby the one-step derivation A)u.\nWe repeat this process until all unit-rules have been eliminated. Let\nR3be the set of rules, after all unit-rules have been eliminated. We de\fne\nG3= (V3;\u0006;R3;S3), whereV3=V2andS3=S2. This grammar has the\nproperty that\n\u000fthe start variable S3does not occur on the right-hand side of any rule\ninR3,\n\u000fR3does not contain any \u000f-rule (it may contain the rule S3!\u000f),\n\u000fR3does not contain any unit-rule, and\n\u000fL(G3) =L(G2) =L(G1) =L(G).\nStep 4: Eliminate all rules having more than two symbols on the right-hand\nside.\nFor each rule in the current set R3that is of the form A!u1u2:::uk,\nwherek\u00153 and each uiis an element of V3[\u0006, we modify G3as follows:\n1. Remove the rule A!u1u2:::ukfrom the current set R3.\n2. Add the following rules to the current set R3:\nA!u1A1\nA1!u2A2\nA2!u3A3\n...\nAk\u00003!uk\u00002Ak\u00002\nAk\u00002!uk\u00001uk"
            }
        ]
    },
    {
        "section": "Page 115",
        "content": [
            {
                "type": "text",
                "text": "3.4. Chomsky normal form 107\nwhereA1;A2;:::;Ak\u00002are new variables that are added to the current\nsetV3.\nObserve that in this way, we replace the one-step derivation A)\nu1u2:::ukby the (k\u00001)-step derivation\nA)u1A1)u1u2A2):::)u1u2:::uk\u00002Ak\u00002)u1u2:::uk:\nLetR4be the set of rules, and let V4be the set of variables, after all rules\nwith more than two symbols on the right-hand side have been eliminated. We\nde\fneG4= (V4;\u0006;R4;S4), whereS4=S3. This grammar has the property\nthat\n\u000fthe start variable S4does not occur on the right-hand side of any rule\ninR4,\n\u000fR4does not contain any \u000f-rule (it may contain the rule S4!\u000f),\n\u000fR4does not contain any unit-rule,\n\u000fR4does not contain any rule with more than two symbols on the right-\nhand side, and\n\u000fL(G4) =L(G3) =L(G2) =L(G1) =L(G).\nStep 5: Eliminate all rules of the form A!u1u2, whereu1andu2are not\nboth variables.\nFor each rule in the current set R4that is of the form A!u1u2, where\nu1andu2are elements of V4[\u0006, butu1andu2are not both contained in\nV4, we modify G3as follows:\n1. Ifu12\u0006 andu22V4, then replace the rule A!u1u2in the current\nsetR4by the two rules A!U1u2andU1!u1, whereU1is a new\nvariable that is added to the current set V4.\nObserve that in this way, we replace the one-step derivation A)u1u2\nby the two-step derivation A)U1u2)u1u2.\n2. Ifu12V4andu22\u0006, then replace the rule A!u1u2in the current\nsetR4by the two rules A!u1U2andU2!u2, whereU2is a new\nvariable that is added to the current set V4.\nObserve that in this way, we replace the one-step derivation A)u1u2\nby the two-step derivation A)u1U2)u1u2."
            }
        ]
    },
    {
        "section": "Page 116",
        "content": [
            {
                "type": "text",
                "text": "108 Chapter 3. Context-Free Languages\n3. Ifu12\u0006,u22\u0006, andu16=u2, then replace the rule A!u1u2in the\ncurrent set R4by the three rules A!U1U2,U1!u1, andU2!u2,\nwhereU1andU2are new variables that are added to the current set\nV4.\nObserve that in this way, we replace the one-step derivation A)u1u2\nby the three-step derivation A)U1U2)u1U2)u1u2.\n4. Ifu12\u0006,u22\u0006, andu1=u2, then replace the rule A!u1u2=u1u1\nin the current set R4by the two rules A!U1U1andU1!u1, where\nU1is a new variable that is added to the current set V4.\nObserve that in this way, we replace the one-step derivation A)\nu1u2=u1u1by the three-step derivation A)U1U1)u1U1)u1u1.\nLetR5be the set of rules, and let V5be the set of variables, after Step 5\nhas been completed. We de\fne G5= (V5;\u0006;R5;S5), whereS5=S4. This\ngrammar has the property that\n\u000fthe start variable S5does not occur on the right-hand side of any rule\ninR5,\n\u000fR5does not contain any \u000f-rule (it may contain the rule S5!\u000f),\n\u000fR5does not contain any unit-rule,\n\u000fR5does not contain any rule with more than two symbols on the right-\nhand side,\n\u000fR5does not contain any rule of the form A!u1u2, whereu1andu2\nare not both variables of V5, and\n\u000fL(G5) =L(G4) =L(G3) =L(G2) =L(G1) =L(G).\nSince the grammar G5is in Chomsky normal form, the proof is complete."
            }
        ]
    },
    {
        "section": "Page 117",
        "content": [
            {
                "type": "text",
                "text": "3.4. Chomsky normal form 109\n3.4.1 An example\nConsider the context-free grammar G= (V;\u0006;R;A ), whereV=fA;Bg,\n\u0006 =f0;1g,Ais the start variable, and Rconsists of the rules\nA!BABjBj\u000f\nB!00j\u000f\nWe apply the construction given in the proof of Theorem 3.4.2 to convert\nthis grammar to a context-free grammar in Chomsky normal form whose\nlanguage is the same as that of G. Throughout the construction, upper case\nletters will denote variables.\nStep 1: Eliminate the start variable from the right-hand side of the rules.\nWe introduce a new start variable S, and add the rule S!A. This gives\nthe following grammar:\nS!A\nA!BABjBj\u000f\nB!00j\u000f\nStep 2: Eliminate all \u000f-rules.\nWe take the \u000f-ruleA!\u000f, and remove it. Then we consider all rules that\ncontainAon the right-hand side. There are two such rules:\n\u000fS!A; we add the rule S!\u000f;\n\u000fA!BAB ; we add the rule A!BB.\nThis gives the following grammar:\nS!Aj\u000f\nA!BABjBjBB\nB!00j\u000f\nWe take the \u000f-ruleB!\u000f, and remove it. Then we consider all rules that\ncontainBon the right-hand side. There are three such rules:\n\u000fA!BAB ; we add the rules A!AB,A!BA, andA!A;\n\u000fA!B; we do not add the rule A!\u000f, because it has already been\nremoved;"
            }
        ]
    },
    {
        "section": "Page 118",
        "content": [
            {
                "type": "text",
                "text": "110 Chapter 3. Context-Free Languages\n\u000fA!BB; we add the rule A!B, but not the rule A!\u000f(because it\nhas already been removed).\nAt this moment, we have the following grammar:\nS!Aj\u000f\nA!BABjBjBBjABjBAjA\nB!00\nSince all\u000f-rules have been eliminated, this completes Step 2. (Observe that\nthe ruleS!\u000fis allowed, because Sis the start variable.)\nStep 3: Eliminate all unit-rules.\nWe take the unit-rule A!A. We can remove this rule, without adding\nany new rule. At this moment, we have the following grammar:\nS!Aj\u000f\nA!BABjBjBBjABjBA\nB!00\nWe take the unit-rule S!A, remove it, and add the rules\nS!BABjBjBBjABjBA:\nThis gives the following grammar:\nS!\u000fjBABjBjBBjABjBA\nA!BABjBjBBjABjBA\nB!00\nWe take the unit-rule S!B, remove it, and add the rule S!00. This\ngives the following grammar:\nS!\u000fjBABjBBjABjBAj00\nA!BABjBjBBjABjBA\nB!00\nWe take the unit-rule A!B, remove it, and add the rule A!00. This\ngives the following grammar:\nS!\u000fjBABjBBjABjBAj00\nA!BABjBBjABjBAj00\nB!00"
            }
        ]
    },
    {
        "section": "Page 119",
        "content": [
            {
                "type": "text",
                "text": "3.5. Pushdown automata 111\nSince all unit-rules have been eliminated, this concludes Step 3.\nStep 4: Eliminate all rules having more than two symbols on the right-hand\nside. There are two such rules:\n\u000fWe take the rule S!BAB , remove it, and add the rules S!BA 1\nandA1!AB.\n\u000fWe take the rule A!BAB , remove it, and add the rules A!BA 2\nandA2!AB.\nThis gives the following grammar:\nS!\u000fjBBjABjBAj00jBA 1\nA!BBjABjBAj00jBA 2\nB!00\nA1!AB\nA2!AB\nStep 4 is now completed.\nStep 5: Eliminate all rules, whose right-hand side contains exactly two\nsymbols, which are not both variables. There are three such rules:\n\u000fWe replace the rule S!00 by the rules S!A3A3andA3!0.\n\u000fWe replace the rule A!00 by the rules A!A4A4andA4!0.\n\u000fWe replace the rule B!00 by the rules B!A5A5andA5!0.\nThis gives the following grammar, which is in Chomsky normal form:\nS!\u000fjBBjABjBAjBA 1jA3A3\nA!BBjABjBAjBA 2jA4A4\nB!A5A5\nA1!AB\nA2!AB\nA3!0\nA4!0\nA5!0"
            }
        ]
    },
    {
        "section": "Page 120",
        "content": [
            {
                "type": "text",
                "text": "112 Chapter 3. Context-Free Languages\n3.5 Pushdown automata\nIn this section, we introduce nondeterministic pushdown automata. As we\nwill see, the class of languages that can be accepted by these automata is\nexactly the class of context-free languages.\nWe start with an informal description of a deterministic pushdown au-\ntomaton. Such an automaton consists of the following, see also Figure 3.1.\n1. There is a tape which is divided into cells. Each cell stores a symbol\nbelonging to a \fnite set \u0006, called the tape alphabet . There is a special\nsymbol 2that is not contained in \u0006; this symbol is called the blank\nsymbol . If a cell contains 2, then this means that the cell is actually\nempty.\n2. There is a tape head which can move along the tape, one cell to the\nright per move. This tape head can also read the cell it currently scans.\n3. There is a stack containing symbols from a \fnite set \u0000, called the stack\nalphabet . This set contains a special symbol $.\n4. There is a stack head which can read the top symbol of the stack. This\nhead can also popthe top symbol, and it can push symbols of \u0000 onto\nthe stack.\n5. There is a state control , which can be in any one of a \fnite number\nofstates . The set of states is denoted by Q. The setQcontains one\nspecial state q, called the start state .\nThe input for a pushdown automaton is a string in \u0006\u0003. This input string\nis stored on the tape of the pushdown automaton and, initially, the tape head\nis on the leftmost symbol of the input string. Initially, the stack only contains\nthe special symbol $, and the pushdown automaton is in the start state q.\nIn one computation step, the pushdown automaton does the following:\n1. Assume that the pushdown automaton is currently in state r. Letabe\nthe symbol of \u0006 that is read by the tape head, and let Abe the symbol\nof \u0000 that is on top of the stack.\n2. Depending on the current state r, the tape symbol a, and the stack\nsymbolA,"
            }
        ]
    },
    {
        "section": "Page 121",
        "content": [
            {
                "type": "text",
                "text": "3.5. Pushdown automata 113\nstate controlaababbabab2 tape\n6\n$AABA\nstack-\nFigure 3.1: A pushdown automaton.\n(a) the pushdown automaton switches to a state r0ofQ(which may\nbe equal to r),\n(b) the tape head either moves one cell to the right or stays at the\ncurrent cell, and\n(c) the top symbol Ais replaced by a string wthat belongs to \u0000\u0003. To\nbe more precise,\ni. ifw=\u000f, thenAis popped from the stack, whereas\nii. ifw=B1B2:::Bk, withk\u00151 andB1;B2;:::;Bk2\u0000, then\nAis replaced by w, andBkbecomes the new top symbol of\nthe stack.\nLater, we will specify when the pushdown automaton accepts the input\nstring.\nWe now give a formal de\fnition of a deterministic pushdown automaton.\nDe\fnition 3.5.1 Adeterministic pushdown automaton is a 5-tuple M=\n(\u0006;\u0000;Q;\u000e;q ), where"
            }
        ]
    },
    {
        "section": "Page 122",
        "content": [
            {
                "type": "text",
                "text": "114 Chapter 3. Context-Free Languages\n1. \u0006 is a \fnite set, called the tape alphabet ; the blank symbol 2is not\ncontained in \u0006,\n2. \u0000 is a \fnite set, called the stack alphabet ; this alphabet contains the\nspecial symbol $,\n3.Qis a \fnite set, whose elements are called states ,\n4.qis an element of Q; it is called the start state ,\n5.\u000eis called the transition function , which is a function\n\u000e:Q\u0002(\u0006[f2g)\u0002\u0000!Q\u0002fN;Rg\u0002\u0000\u0003:\nThe transition function \u000ecan be thought of as being the \\program\" of the\npushdown automaton. This function tells us what the automaton can do in\none \\computation step\": Let r2Q,a2\u0006[f2g, andA2\u0000. Furthermore,\nletr02Q,\u001b2fR;Ng, andw2\u0000\u0003be such that\n\u000e(r;a;A ) = (r0;\u001b;w ): (3.1)\nThis transition means that if\n\u000fthe pushdown automaton is in state r,\n\u000fthe tape head reads the symbol a, and\n\u000fthe top symbol on the stack is A,\nthen\n\u000fthe pushdown automaton switches to state r0,\n\u000fthe tape head moves according to \u001b: if\u001b=R, then it moves one cell\nto the right; if \u001b=N, then it does not move, and\n\u000fthe top symbol Aon the stack is replaced by the string w.\nWe will write the computation step (3.1) in the form of the instruction\nraA!r0\u001bw:\nWe now specify the computation of the pushdown automaton M= (\u0006;\u0000;Q;\u000e;q )."
            }
        ]
    },
    {
        "section": "Page 123",
        "content": [
            {
                "type": "text",
                "text": "3.6. Examples of pushdown automata 115\nStart con\fguration: Initially, the pushdown automaton is in the start state\nq, the tape head is on the leftmost symbol of the input string a1a2:::an, and\nthe stack only contains the special symbol $.\nComputation and termination: Starting in the start con\fguration, the\npushdown automaton performs a sequence of computation steps as described\nabove. It terminates at the moment when the stack becomes empty. (Hence,\nif the stack never gets empty, the pushdown automaton does notterminate.)\nAcceptance: The pushdown automaton accepts the input string a1a2:::an2\n\u0006\u0003, if\n1. the automaton terminates on this input, and\n2. at the time of termination (i.e., at the moment when the stack gets\nempty), the tape head is on the cell immediately to the right of the cell\ncontaining the symbol an(this cell must contain the blank symbol 2).\nIn all other cases, the pushdown automaton rejects the input string. Thus,\nthe pushdown automaton rejects this string if\n1. the automaton does not terminate on this input (i.e., the computation\n\\loops forever\") or\n2. at the time of termination, the tape head is not on the cell immediately\nto the right of the cell containing the symbol an.\nWe denote by L(M) the language accepted by the pushdown automaton\nM. Thus,\nL(M) =fw2\u0006\u0003:Macceptswg:\nThe pushdown automaton described above is deterministic. For a non-\ndeterministic pushdown automata, the current computation step may not\nbe uniquely de\fned, but the automaton can make a choice out of a \fnite\nnumber of possibilities. In this case, the transition function \u000eis a function\n\u000e:Q\u0002(\u0006[f2g)\u0002\u0000!Pf(Q\u0002fN;Rg\u0002\u0000\u0003);\nwherePf(K) is the set of all \fnite subsets of the set K.\nWe say that a nondeterministic pushdown automaton Maccepts an input\nstring, if there exists an accepting computation, in the sense as described for\ndeterministic pushdown automata. We say that Mrejects an input string, if\nevery computation on this string is rejecting. As before, we denote by L(M)\nthe set of all strings in \u0006\u0003that are accepted by M."
            }
        ]
    },
    {
        "section": "Page 124",
        "content": [
            {
                "type": "text",
                "text": "116 Chapter 3. Context-Free Languages\n3.6 Examples of pushdown automata\n3.6.1 Properly nested parentheses\nWe will show how to construct a deterministic pushdown automaton, that\naccepts the set of all strings of properly nested parentheses. Observe that a\nstringwinf(;)g\u0003is properly nested if and only if\n\u000fin every pre\fx of w, the number of \\(\" is greater than or equal to the\nnumber of \\)\", and\n\u000fin the complete string w, the number of \\(\" is equal to the number of\n\\)\".\nWe will use the tape symbol afor \\(\", and the tape symbol bfor \\)\".\nThe idea is as follows. Recall that initially, the stack only contains the\nspecial symbol $. The pushdown automaton reads the input string from left\nto right. For every ait reads, it pushes the symbol Sonto the stack, and\nfor everybit reads, it pops the top symbol from the stack. In this way, the\nnumber of symbols Son the stack will always be equal to the number of as\nthat have been read minus the number of bs that have been read; additionally,\nthe bottom of the stack will contain the special symbol $. The input string\nis properly nested if and only if (i) this di\u000berence is always non-negative and\n(ii) this di\u000berence is zero once the entire input string has been read. Hence,\nthe input string is accepted if and only if during this process, (i) the stack\nalways contains at least the special symbol $ and (ii) at the end, the stack\nonly contains the special symbol $ (which will then be popped in the \fnal\nstep).\nBased on this discussion, we obtain the deterministic pushdown automa-\ntonM= (\u0006;\u0000;Q;\u000e;q ), where \u0006 =fa;bg, \u0000 =f$;Sg,Q=fqg, and the\ntransition function \u000eis speci\fed by the following instructions:"
            }
        ]
    },
    {
        "section": "Page 125",
        "content": [
            {
                "type": "text",
                "text": "3.6. Examples of pushdown automata 117\nqa$!qR$Sbecause of the a,Sis pushed onto the stack\nqaS!qRSS because of the a,Sis pushed onto the stack\nqbS!qR\u000f because of the b, the top element is popped\nfrom the stack\nqb$!qN\u000f the number of bs read is larger than the number\nofas read; the stack is made empty (hence,\nthe computation terminates before the entire\nstring has been read), and the input string is rejected\nq2$!qN\u000f the entire input string has been read; the stack is\nmade empty, and the input string is accepted\nq2S!qNS the entire input string has been read, it contains\nmoreas thanbs; no changes are made (thus, the\nautomaton does not terminate), and the input string\nis rejected\n3.6.2 Strings of the form 0n1n\nWe construct a deterministic pushdown automata that accepts the language\nf0n1n:n\u00150g.\nThe automaton uses two states q0andq1, whereq0is the start state.\nInitially, the automaton is in state q0.\n\u000fFor each 0 that it reads, the automaton pushes one symbol Sonto the\nstack and stays in state q0.\n\u000fWhen the \frst 1 is read, the automaton switches to state q1. From that\nmoment,\n{for each 1 that is read, the automaton pops the top symbol from\nthe stack and stays in state q1;\n{if a 0 is read, the automaton does not make any change and,\ntherefore, does not terminate.\nBased on this discussion, we obtain the deterministic pushdown automa-\ntonM= (\u0006;\u0000;Q;\u000e;q 0), where \u0006 =f0;1g, \u0000 =f$;Sg,Q=fq0;q1g,q0is\nthe start state, and the transition function \u000eis speci\fed by the following\ninstructions:"
            }
        ]
    },
    {
        "section": "Page 126",
        "content": [
            {
                "type": "text",
                "text": "118 Chapter 3. Context-Free Languages\nq00$!q0R$SpushSonto the stack\nq00S!q0RSS pushSonto the stack\nq01$!q0N$ \frst symbol in the input is 1; loop forever\nq01S!q1R\u000f \frst 1 is encountered\nq02$!q0N\u000f input string is empty; accept\nq02S!q0NS input only consists of 0s; loop forever\nq10$!q1N$ 0 to the right of 1; loop forever\nq10S!q1NS 0 to the right of 1; loop forever\nq11$!q1N$ too many 1s; loop forever\nq11S!q1R\u000f pop top symbol from the stack\nq12$!q1N\u000f accept\nq12S!q1NS too many 0s; loop forever\n3.6.3 Strings with bin the middle\nWe will construct a nondeterministic pushdown automaton that accepts the\nsetLof all strings infa;bg\u0003having an odd length and whose middle symbol\nisb, i.e.,\nL=fvbw:v2fa;bg\u0003;w2fa;bg\u0003;jvj=jwjg:\nThe idea is as follows. The automaton uses two states qandq0, whereq\nis the start state. These states have the following meaning:\n\u000fIf the automaton is in state q, then it has not reached the middle symbol\nbof the input string.\n\u000fIf the automaton is in state q0, then it has read the middle symbol b.\nObserve that since the automaton can only make one single pass over the\ninput string, it has to \\guess\" (i.e., use nondeterminism) when it reaches the\nmiddle of the string.\n\u000fIf the automaton is in state q, then, when reading the current tape\nsymbol,\n{it either pushes one symbol Sonto the stack and stays in state q\n{or, in case the current tape symbol is b, it \\guesses\" that it has\nreached the middle of the input string, by switching to state q0.\n\u000fIf the automaton is in state q0, then, when reading the current tape\nsymbol, it pops the top symbol Sfrom the stack and stays in state q0."
            }
        ]
    },
    {
        "section": "Page 127",
        "content": [
            {
                "type": "text",
                "text": "3.7. Equivalence of PDA's and CFG's 119\nIn this way, the number of symbols Son the stack will always be equal to the\ndi\u000berence of (i) the number of symbols in the part to the left of the middle\nsymbolbthat have been read and (ii) the number of symbols in the part\nto the right of the middle symbol bthat have been read; additionally, the\nbottom of the stack will contain the special symbol $.\nThe input string is accepted if and only if, at the moment when the blank\nsymbol 2is read, the automaton is in state q0and the top symbol on the\nstack is $. In this case, the stack is made empty and, thus, the computation\nterminates.\nWe obtain the nondeterministic pushdown automaton M= (\u0006;\u0000;Q;\u000e;q ),\nwhere \u0006 =fa;bg, \u0000 =f$;Sg,Q=fq;q0g,qis the start state, and the\ntransition function \u000eis speci\fed by the following instructions:\nqa$!qR$S pushSonto the stack\nqaS!qRSS pushSonto the stack\nqb$!q0R$ reached the middle\nqb$!qR$S did not reach the middle; push Sonto the stack\nqbS!q0RS reached the middle\nqbS!qRSS did not reach the middle; push Sonto the stack\nq2$!qN$ input string is empty; loop forever\nq2S!qNS loop forever\nq0a$!q0N\u000f stack is empty; terminate, but reject, because\nthe entire input string has not been read\nq0aS!q0R\u000f pop top symbol from stack\nq0b$!q0N\u000f stack is empty; terminate, but reject, because\nthe entire input string has not been read\nq0bS!q0R\u000f pop top symbol from stack\nq02$!q0N\u000f accept\nq02S!q0NS loop forever\nRemark 3.6.1 It can be shown that there is no deterministic pushdown\nautomaton that accepts the language L. The reason is that a deterministic\npushdown automaton cannot determine when it reaches the middle of the\ninput string. Thus, unlike as for \fnite automata, nondeterministic pushdown\nautomata are more powerful than their deterministic counterparts."
            }
        ]
    },
    {
        "section": "Page 128",
        "content": [
            {
                "type": "text",
                "text": "120 Chapter 3. Context-Free Languages\n3.7 Equivalence of pushdown automata and\ncontext-free grammars\nThe main result of this section is that nondeterministic pushdown automata\nand context-free grammars are equivalent in power:\nTheorem 3.7.1 Let\u0006be an alphabet and let A\u0012\u0006\u0003be a language. Then\nAis context-free if and only if there exists a nondeterministic pushdown\nautomaton that accepts A.\nWe will only prove one direction of this theorem. That is, we will show\nhow to convert an arbitrary context-free grammar to a nondeterministic push-\ndown automaton.\nLetG= (V;\u0006;R;$) be a context-free grammar, where Vis the set of\nvariables, \u0006 is the set of terminals, Ris the set of rules, and $ is the start\nvariable. By Theorem 3.4.2, we may assume that Gis in Chomsky normal\nform. Hence, every rule in Rhas one of the following three forms:\n1.A!BC, whereA,B, andCare variables, B6= $, andC6= $.\n2.A!a, whereAis a variable and ais a terminal.\n3. $!\u000f.\nWe will construct a nondeterministic pushdown automaton Mthat ac-\ncepts the language L(G) of this grammar G. Observe that Mmust have the\nfollowing property: For every string w=a1a2:::an2\u0006\u0003,\nw2L(G) if and only if Macceptsw.\nThis can be reformulated as follows:\n$\u0003)a1a2:::an\nif and only if there exists a computation of Mthat starts in the initial\ncon\fguration\na1\u0001\u0001\u0001ai\u0001\u0001\u0001an2\n6\n$-"
            }
        ]
    },
    {
        "section": "Page 129",
        "content": [
            {
                "type": "text",
                "text": "3.7. Equivalence of PDA's and CFG's 121\nand ends in the con\fguration\na1\u0001\u0001\u0001ai\u0001\u0001\u0001an2\n6\n;-\nwhere;indicates that the stack is empty.\nAssume that $\u0003)a1a2:::an. Then there exists a derivation (using the\nrules ofR) of the string a1a2:::anfrom the start variable $. We may assume\nthat in each step in this derivation, a rule is applied to the leftmost variable\nin the current string. Hence, because the grammar Gis in Chomsky normal\nform, at any moment during the derivation, the current string has the form\na1a2:::ai\u00001AkAk\u00001:::A 1; (3.2)\nfor some integers iandkwith 1\u0014i\u0014n+ 1 andk\u00150, and variables\nA1;A2;:::;Ak. (In particular, at the start of the derivation, we have i= 1\nandk= 1, and the current string is Ak= $. At the end of the derivation,\nwe havei=n+ 1 andk= 0, and the current string is a1a2:::an.)\nWe will de\fne the pushdown automaton Min such a way that the current\nstring (3.2) corresponds to the con\fguration\na1\u0001\u0001\u0001ai\u0001\u0001\u0001an2\n6\nA1...Ak-\nBased on this discussion, we obtain the nondeterministic pushdown au-\ntomatonM= (\u0006;V;fqg;\u000e;q), where\n\u000fthe tape alphabet is the set \u0006 of terminals of G,\n\u000fthe stack alphabet is the set Vof variables of G,\n\u000fthe set of states consists of one state q, which is the start state, and\n\u000fthe transition function \u000eis obtained from the rules in R, in the following\nway:"
            }
        ]
    },
    {
        "section": "Page 130",
        "content": [
            {
                "type": "text",
                "text": "122 Chapter 3. Context-Free Languages\n{For each rule inRthat is of the form A!BC, withA;B;C2V,\nthe pushdown automaton Mhas the instructions\nqaA!qNCB; for alla2\u0006:\n{For each rule inRthat is of the form A!a, withA2Vand\na2\u0006, the pushdown automaton Mhas the instruction\nqaA!qR\u000f:\n{IfRcontains the rule $ !\u000f, then the pushdown automaton M\nhas the instruction\nq2$!qN\u000f:\nThis concludes the de\fnition of M. It remains to prove that L(M) =\nL(G), i.e., the language of the nondeterministic pushdown automaton Mis\nequal to the language of the context-free grammar G. Hence, we have to\nshow that for every string w2\u0006\u0003,\nw2L(G) if and only if w2L(M);\nwhich can be rewritten as\n$\u0003)wif and only if Macceptsw.\nClaim 3.7.2 Leta1a2:::anbe a string in \u0006\u0003, letA1;A2;:::;Akbe variables\ninV, and letiandkbe integers with 1\u0014i\u0014n+ 1 andk\u00150. Then the\nfollowing holds:\n$\u0003)a1a2:::ai\u00001AkAk\u00001:::A 1\nif and only if there exists a computation of Mfrom the initial con\fguration\na1\u0001\u0001\u0001ai\u0001\u0001\u0001an2\n6\n$-\nto the con\fguration"
            }
        ]
    },
    {
        "section": "Page 131",
        "content": [
            {
                "type": "text",
                "text": "3.7. Equivalence of PDA's and CFG's 123\na1\u0001\u0001\u0001ai\u0001\u0001\u0001an2\n6\nA1...Ak-\nProof. The claim can be proved by induction. Let\nw=a1a2:::ai\u00001AkAk\u00001:::A 1:\nAssume that k\u00151 and assume that the claim is true for the string w. Then\nwe have to show that the claim is still true after applying a rule in Rto the\nleftmost variable Akinw. Since the grammar is in Chomsky normal form,\nthe rule to be applied is either of the form Ak!BCor of the form Ak!ai.\nIn both cases, the property mentioned in the claim is maintained.\nWe now use Claim 3.7.2 to prove that L(M) =L(G). Letw=a1a2:::an\nbe an arbitrary string in \u0006\u0003. By applying Claim 3.7.2, with i=n+ 1 and\nk= 0, we see that w2L(G), i.e.,\n$\u0003)a1a2:::an;\nif and only if there exists a computation of Mfrom the initial con\fguration\na1\u0001\u0001\u0001ai\u0001\u0001\u0001an2\n6\n$-\nto the con\fguration\na1\u0001\u0001\u0001ai\u0001\u0001\u0001an2\n6\n;-\nBut this means that w2L(G) if and only if the automaton Maccepts the\nstringw.\nThis concludes the proof of the fact that every context-free grammar can\nbe converted to a nondeterministic pushdown automaton. As mentioned\nalready, we will not give the conversion in the other direction. We \fnish this\nsection with the following observation:"
            }
        ]
    },
    {
        "section": "Page 132",
        "content": [
            {
                "type": "text",
                "text": "124 Chapter 3. Context-Free Languages\nTheorem 3.7.3 Let\u0006be an alphabet and let A\u0012\u0006\u0003be a context-free lan-\nguage. Then there exists a nondeterministic pushdown automaton that ac-\nceptsAand has only one state.\nProof. SinceAis context-free, there exists a context-free grammar G0such\nthatL(G0) =A. By Theorem 3.4.2, there exists a context-free grammar G\nthat is in Chomsky normal form and for which L(G) =L(G0). The construc-\ntion given above converts Gto a nondeterministic pushdown automaton M\nthat has only one state and for which L(M) =L(G).\n3.8 The pumping lemma for context-free lan-\nguages\nIn Section 2.9, we proved the pumping lemma for regular languages and\nused it to prove that certain languages are not regular. In this section, we\ngeneralize the pumping lemma to context-free languages. The idea is to\nconsider the parse tree (see Section 3.1) that describes the derivation of a\nsu\u000eciently long string in the context-free language L. Since the number of\nvariables in the corresponding context-free grammar Gis \fnite, there is at\nleast one variable, say Aj, that occurs more than once on the longest root-\nto-leaf path in the parse tree. The subtree which is sandwiched between two\noccurrences of Ajon this path can be copied any number of times. This will\nresult in a legal parse tree and, hence, in a \\pumped\" string that is in the\nlanguageL.\nTheorem 3.8.1 (Pumping Lemma for Context-Free Languages) Let\nLbe a context-free language. Then there exists an integer p\u00151, called the\npumping length , such that the following holds: Every string sinL, with\njsj\u0015p, can be written as s=uvxyz , such that\n1.jvyj\u00151(i.e.,vandyare not both empty),\n2.jvxyj\u0014p, and\n3.uvixyiz2L, for alli\u00150."
            }
        ]
    },
    {
        "section": "Page 133",
        "content": [
            {
                "type": "text",
                "text": "3.8. The pumping lemma for context-free languages 125\n3.8.1 Proof of the pumping lemma\nThe proof of the pumping lemma will use the following result about parse\ntrees:\nLemma 3.8.2 LetGbe a context-free grammar in Chomsky normal form,\nletsbe a non-empty string in L(G), and letTbe a parse tree for s. Let`be\nthe height of T, i.e.,`is the number of edges on a longest root-to-leaf path\ninT. Then\njsj\u00142`\u00001:\nProof. The claim can be proved by induction on `. By looking at some\nsmall values of `and using the fact that Gis in Chomsky normal form, you\nshould be able to verify the claim.\nNow we can start with the proof of the pumping lemma. Let Lbe a\ncontext-free language and let \u0006 be the alphabet of L. By Theorem 3.4.2, there\nexists a context-free grammar in Chomsky normal form, G= (V;\u0006;R;S ),\nsuch thatL=L(G).\nDe\fnerto be the number of variables of Gand de\fne p= 2r. We will\nprove that the value of pcan be used as the pumping length. Consider an\narbitrary string sinLsuch thatjsj\u0015p, and letTbe a parse tree for s. Let\n`be the height of T. Then, by Lemma 3.8.2, we have\njsj\u00142`\u00001:\nOn the other hand, we have\njsj\u0015p= 2r:\nBy combining these inequalities, we see that 2r\u00142`\u00001, which can be rewrit-\nten as\n`\u0015r+ 1:\nConsider the nodes on a longest root-to-leaf path in T. Since this path\nconsists of`edges, it consists of `+ 1 nodes. The \frst `of these nodes store\nvariables, which we denote by A0;A1;:::;A`\u00001(whereA0=S), and the last\nnode (which is a leaf) stores a terminal, which we denote by a.\nSince`\u00001\u0000r\u00150, the sequence\nA`\u00001\u0000r;A`\u0000r;:::;A`\u00001"
            }
        ]
    },
    {
        "section": "Page 134",
        "content": [
            {
                "type": "text",
                "text": "126 Chapter 3. Context-Free Languages\nof variables is well-de\fned. Observe that this sequence consists of r+ 1\nvariables. Since the number of variables in the grammar Gis equal to r,\nthe pigeonhole principle implies that there is a variable that occurs at least\ntwice in this sequence. In other words, there are indices jandk, such that\n`\u00001\u0000r\u0014j < k\u0014`\u00001 andAj=Ak. Refer to the \fgure below for an\nillustration.\nS\nAj\nAk\nu v x y z\nsA0=S\nA1\nAℓ−1−r\nAℓ−r\nAℓ−2\nAℓ−1\nar+1\nvariables\nRecall that Tis a parse tree for the string s. Therefore, the terminals\nstored at the leaves of T, in the order from left to right, form s. As indicated\nin the \fgure above, the nodes storing the variables AjandAkpartitions\ninto \fve substrings u,v,x,y, andz, such that s=uvxyz .\nIt remains to prove that the three properties stated in the pumping lemma"
            }
        ]
    },
    {
        "section": "Page 135",
        "content": [
            {
                "type": "text",
                "text": "3.8. The pumping lemma for context-free languages 127\nhold. We start with the third property, i.e., we prove that\nuvixyiz2L;for alli\u00150.\nIn the grammar G, we have\nS\u0003)uAjz: (3.3)\nSinceAj\u0003)vAkyandAk=Aj, we have\nAj\u0003)vAjy: (3.4)\nFinally, since Ak\u0003)xandAk=Aj, we have\nAj\u0003)x: (3.5)\nFrom (3.3) and (3.5), it follows that\nS\u0003)uAjz\u0003)uxz;\nwhich implies that the string uxzis in the language L. Similarly, it follows\nfrom (3.3), (3.4), and (3.5) that\nS\u0003)uAjz\u0003)uvAjyz\u0003)uvvAjyyz\u0003)uvvxyyz:\nHence, the string uv2xy2zis in the language L. In general, for each i\u00150,\nthe stringuvixyizis in the language L, because\nS\u0003)uAjz\u0003)uviAjyiz\u0003)uvixyiz:\nThis proves that the third property in the pumping lemma holds.\nNext we show that the second property holds. That is, we prove that\njvxyj\u0014p. Consider the subtree rooted at the node storing the variable\nAj. The path from the node storing Ajto the leaf storing the terminal\nais a longest path in this subtree. (Convince yourself that this is true.)\nMoreover, this path consists of `\u0000jedges. Since Aj\u0003)vxy, this subtree\nis a parse tree for the string vxy(whereAjis used as the start variable).\nTherefore, by Lemma 3.8.2, we can conclude that jvxyj\u00142`\u0000j\u00001. We know\nthat`\u00001\u0000r\u0014j, which is equivalent to `\u0000j\u00001\u0014r. It follows that\njvxyj\u00142`\u0000j\u00001\u00142r=p:"
            }
        ]
    },
    {
        "section": "Page 136",
        "content": [
            {
                "type": "text",
                "text": "128 Chapter 3. Context-Free Languages\nFinally, we show that the \frst property in the pumping lemma holds.\nThat is, we prove that jvyj\u00151. Recall that\nAj\u0003)vAky:\nLet the \frst rule used in this derivation be Aj!BC. (Since the variables\nAjandAk, even though they are equal, are stored at di\u000berent nodes of the\nparse tree, and since the grammar Gis in Chomsky normal form, this \frst\nrule exists.) Then\nAj)BC\u0003)vAky:\nObserve that the string BChas length two. Moreover, by applying rules of\na grammar in Chomsky normal form, strings cannot become shorter. (Here,\nwe use the fact that the start variable does not occur on the right-hand side\nof any rule.) Therefore, we have jvAkyj\u00152. But this implies that jvyj\u00151.\nThis completes the proof of the pumping lemma.\n3.8.2 Applications of the pumping lemma\nFirst example\nConsider the language\nA=fanbncn:n\u00150g:\nWe will prove by contradiction that Ais not a context-free language.\nAssume that Ais a context-free language. Let p\u00151 be the pumping\nlength, as given by the pumping lemma. Consider the string s=apbpcp.\nObserve that s2Aandjsj= 3p\u0015p. Hence, by the pumping lemma, scan\nbe written as s=uvxyz , wherejvyj\u00151,jvxyj\u0014p, anduvixyiz2Afor all\ni\u00150.\nObserve that the pumping lemma does not tell us the location of the\nsubstringvxyin the string s, it only gives us an upper bound on the length\nof this substring. Therefore, we have to consider three cases, depending on\nthe location of vxyins.\nCase 1: The substring vxydoes not contain any c.\nConsider the string uv2xy2z=uvvxyyz . Sincejvyj \u0015 1, this string\ncontains more than pmanyas or more than pmanybs. Since it contains\nexactlypmanycs, it follows that this string is not in the language A. This\nis a contradiction because, by the pumping lemma, the string uv2xy2zis in\nA."
            }
        ]
    },
    {
        "section": "Page 137",
        "content": [
            {
                "type": "text",
                "text": "3.8. The pumping lemma for context-free languages 129\nCase 2: The substring vxydoes not contain any a.\nConsider the string uv2xy2z=uvvxyyz . Sincejvyj \u0015 1, this string\ncontains more than pmanybs or more than pmanycs. Since it contains\nexactlypmanyas, it follows that this string is not in the language A. This\nis a contradiction because, by the pumping lemma, the string uv2xy2zis in\nA.\nCase 3: The substring vxycontains at least one aand at least one c.\nSinces=apbpcp, this implies that jvxyj>p, which again contradicts the\npumping lemma.\nThus, in all of the three cases, we have obtained a contradiction. There-\nfore, we have shown that the language Ais not context-free.\nSecond example\nConsider the languages\nA=fwwR:w2fa;bg\u0003g;\nwherewRis the string obtained by writing wbackwards, and\nB=fww:w2fa;bg\u0003g:\nEven though these languages look similar, we will show that Ais context-free\nandBis not context-free.\nConsider the following context-free grammar, in which Sis the start vari-\nable:\nS!\u000fjaSajbSb:\nIt is easy to see that the language of this grammar is exactly the language A.\nTherefore,Ais context-free. Alternatively, we can show that Ais context-\nfree, by constructing a (nondeterministic) pushdown automaton that accepts\nA. This automaton has two states qandq0, whereqis the start state. If the\nautomaton is in state q, then it did not yet \fnish reading the leftmost half of\nthe input string; it pushes all symbols read onto the stack. If the automaton\nis in stateq0, then it is reading the rightmost half of the input string; for each\nsymbol read, it checks whether it is equal to the symbol on top of the stack\nand, if so, pops the top symbol from the stack. The pushdown automaton\nuses nondeterminism to \\guess\" when to switch from state qto stateq0(i.e.,\nwhen it has completed reading the leftmost half of the input string)."
            }
        ]
    },
    {
        "section": "Page 138",
        "content": [
            {
                "type": "text",
                "text": "130 Chapter 3. Context-Free Languages\nAt this point, you should convince yourself that the two approaches above,\nwhich showed that Ais context-free, do notwork forB. The reason why\nthey do not work is that the language Bis not context-free, as we will prove\nnow.\nAssume that Bis a context-free language. Let p\u00151 be the pumping\nlength, as given by the pumping lemma. At this point, we must choose a\nstringsinB, whose length is at least p, and that does not satisfy the three\nproperties stated in the pumping lemma. Let us try the string s=apbapb.\nThens2Bandjsj= 2p+ 2\u0015p. Hence, by the pumping lemma, scan be\nwritten ass=uvxyz , where (i)jvyj\u00151, (ii)jvxyj\u0014p, and (iii)uvixyiz2B\nfor alli\u00150. It may happen that p\u00153,u=ap\u00001,v=a,x=b,y=a,\nandz=ap\u00001b. If this is the case, then properties (i), (ii), and (iii) hold,\nand, thus, we do not get a contradiction. In other words, we have chosen\nthe \\wrong\" string s. This string is \\wrong\", because there is only one b\nbetween the as. Because of this, vcan be in the leftmost block of as, and\nycan be in the rightmost block of as. Observe that if there were at least p\nmanybs between the as, then this would not happen, because jvxyj\u0014p.\nBased on the discussion above, we choose s=apbpapbp. Observe that\ns2Bandjsj= 4p\u0015p. Hence, by the pumping lemma, scan be written as\ns=uvxyz , wherejvyj\u00151,jvxyj\u0014p, anduvixyiz2Bfor alli\u00150. Based\non the location of vxyin the string s, we distinguish three cases:\nCase 1: The substring vxyoverlaps both the leftmost half and the rightmost\nhalf ofs.\nSincejvxyj\u0014p, the substring vxyis contained in the \\middle\" part of s,\ni.e.,vxyis contained in the block bpap. Consider the string uv0xy0z=uxz.\nSincejvyj\u00151, we know that at least one of vandyis non-empty.\n\u000fIfv6=\u000f, thenvcontains at least one bfrom the leftmost block of bs in\ns, whereasydoes not contain any bfrom the rightmost block of bs ins.\nTherefore, in the string uxz, the leftmost block of bs contains fewer bs\nthan the rightmost block of bs. Hence, the string uxzis not contained\ninB.\n\u000fIfy6=\u000f, thenycontains at least one afrom the rightmost block of\nas ins, whereasvdoes not contain any afrom the leftmost block of\nas ins. Therefore, in the string uxz, the leftmost block of as contains\nmoreas than the rightmost block of as. Hence, the string uxzis not\ncontained in B."
            }
        ]
    },
    {
        "section": "Page 139",
        "content": [
            {
                "type": "text",
                "text": "3.8. The pumping lemma for context-free languages 131\nIn both cases, we conclude that the string uxz is not an element of the\nlanguageB. But, by the pumping lemma, this string is contained in B.\nCase 2: The substring vxyis in the leftmost half of s.\nIn this case, none of the strings uxz,uv2xy2z,uv3xy3z,uv4xy4z, etc.,\nis contained in B. But, by the pumping lemma, each of these strings is\ncontained in B.\nCase 3: The substring vxyis in the rightmost half of s.\nThis case is symmetric to Case 2: None of the strings uxz,uv2xy2z,\nuv3xy3z,uv4xy4z, etc., is contained in B. But, by the pumping lemma, each\nof these strings is contained in B.\nTo summarize, in each of the three cases, we have obtained a contradic-\ntion. Therefore, the language Bis not context-free.\nThird example\nWe have seen in Section 3.2.4 that the language\nfambncm+n:m\u00150;n\u00150g\nis context-free. Using the pumping lemma for regular languages, it is easy to\nprove that this language is not regular. In other words, context-free gram-\nmars can verify addition, whereas \fnite automata are not powerful enough\nfor this. We now consider the problem of verifying multiplication: Let Abe\nthe language de\fned as\nA=fambncmn:m\u00150;n\u00150g:\nWe will prove by contradiction that Ais not a context-free language.\nAssume that Ais context-free. Let p\u00151 be the pumping length, as\ngiven by the pumping lemma. Consider the string s=apbpcp2. Then,s2A\nandjsj= 2p+p2\u0015p. Hence, by the pumping lemma, scan be written as\ns=uvxyz , wherejvyj\u00151,jvxyj\u0014p, anduvixyiz2Afor alli\u00150.\nThere are three possible cases, depending on the locations of vandyin\nthe strings.\nCase 1: The substring vdoes not contain any aand does not contain any\nb, and the substring ydoes not contain any aand does not contain any b."
            }
        ]
    },
    {
        "section": "Page 140",
        "content": [
            {
                "type": "text",
                "text": "132 Chapter 3. Context-Free Languages\nConsider the string uv2xy2z. Sincejvyj\u00151, this string consists of p\nmanyas,pmanybs, but more than p2manycs. Therefore, this string is not\ncontained in A. But, by the pumping lemma, it is contained in A.\nCase 2: The substring vdoes not contain any cand the substring ydoes\nnot contain any c.\nConsider again the string uv2xy2z. This string consists of p2manycs.\nSincejvyj\u00151, in this string,\n\u000fthe number of as is at least p+ 1 and the number of bs is at least p, or\n\u000fthe number of as is at least pand the number of bs is at least p+ 1.\nTherefore, the number of as multiplied by the number of bs is at least p(p+1),\nwhich is larger than p2. Therefore, uv2xy2zis not contained in A. But, by\nthe pumping lemma, this string is contained in A.\nCase 3: The substring vcontains at least one band the substring ycontains\nat least one c.\nSincejvxyj\u0014p, the substring vydoes not contain any a. Thus, we can\nwritevy=bjck, wherej\u00151 andk\u00151. Consider the string uxz. We can\nwrite this string as uxz=apbp\u0000jcp2\u0000k. Since, by the pumping lemma, this\nstring is contained in A, we havep(p\u0000j) =p2\u0000k, which implies that jp=k.\nThus,\njvxyj\u0015jvyj=j+k=j+jp\u00151 +p:\nBut, by the pumping lemma, we have jvxyj\u0014p.\nObserve that, since jvxyj\u0014p, the above three cases cover all possibilities\nfor the locations of vandyin the string s. In each of the three cases, we\nhave obtained a contradiction. Therefore, the language Ais not context-free.\nExercises\n3.1Construct context-free grammars that generate the following languages.\nIn all cases, \u0006 =f0;1g.\n\u000ff02n1n:n\u00150g\n\u000ffw:wcontains at least three 1s g\n\u000ffw: the length of wis odd and its middle symbol is 0 g"
            }
        ]
    },
    {
        "section": "Page 141",
        "content": [
            {
                "type": "text",
                "text": "Exercises 133\n\u000ffw:wis a palindromeg.\nApalindrome is a string whaving the property that w=wR, i.e.,\nreadingwfrom left to right gives the same result as reading wfrom\nright to left.\n\u000ffw:wstarts and ends with the same symbol g\n\u000ffw:wstarts and ends with di\u000berent symbols g\n3.2LetG= (V;\u0006;R;S ) be the context-free grammar, where V=fA;B;Sg,\n\u0006 =f0;1g,Sis the start variable, and Rconsists of the rules\nS!0Sj1Aj\u000f\nA!0Bj1S\nB!0Aj1B\nDe\fne the following language L:\nL=fw2f0;1g\u0003:wis the binary representation of a non-negative\ninteger that is divisible by three g[f\u000fg:\nProve that L=L(G). (Hint: The variables S,A, andBare used to\nremember the remainder after division by three.)\n3.3LetG= (V;\u0006;R;S ) be the context-free grammar, where V=fA;B;Sg,\n\u0006 =fa;bg,Sis the start variable, and Rconsists of the rules\nS!aBjbA\nA!ajaSjBAA\nB!bjbSjABB\n\u000fProve that ababba2L(G).\n\u000fProve that L(G) is the set of all non-empty strings wover the alphabet\nfa;bgsuch that the number of as inwis equal to the number of bs in\nw.\n3.4LetAandBbe context-free languages over the same alphabet \u0006.\n\u000fProve that the union A[BofAandBis also context-free.\n\u000fProve that the concatenation ABofAandBis also context-free."
            }
        ]
    },
    {
        "section": "Page 142",
        "content": [
            {
                "type": "text",
                "text": "134 Chapter 3. Context-Free Languages\n\u000fProve that the star A\u0003ofAis also context-free.\n3.5De\fne the following two languages AandB:\nA=fambncn:m\u00150;n\u00150g\nand\nB=fambmcn:m\u00150;n\u00150g:\n\u000fProve that both AandBare context-free, by constructing two context-\nfree grammars, one that generates Aand one that generates B.\n\u000fWe have seen in Section 3.8.2 that the language\nfanbncn:n\u00150g\nis not context-free. Explain why this implies that the intersection of\ntwo context-free languages is not necessarily context-free.\n\u000fUse De Morgan's Law to conclude that the complement of a context-\nfree language is not necessarily context-free.\n3.6LetAbe a context-free language and let Bbe a regular language.\n\u000fProve that the intersection A\\BofAandBis context-free.\n\u000fProve that the set-di\u000berence\nAnB=fw:w2A;w62Bg\nofAandBis context-free.\n\u000fIs the set-di\u000berence of two context-free languages necessarily context-\nfree?\n3.7LetLbe the language consisting of all non-empty strings wover the\nalphabetfa;bgsuch that\n\u000fthe number of as inwis equal to the number of bs inw,\n\u000fwdoes not contain the substring abba, and\n\u000fwdoes not contain the substring bbaa."
            }
        ]
    },
    {
        "section": "Page 143",
        "content": [
            {
                "type": "text",
                "text": "Exercises 135\nIn this exercise, you will prove that Lis context-free.\nLetAbe the language consisting of all non-empty strings wover the\nalphabetfa;bgsuch that the number of as inwis equal to the number of bs\ninw. In Exercise 3.3, you have shown that Ais context-free.\nLetBbe the language consisting of all strings wover the alphabet fa;bg\nsuch that\n\u000fwdoes not contain the substring abba, and\n\u000fwdoes not contain the substring bbaa.\n1. Give a regular expression that describes the complement of B.\n2. Argue that Bis a regular language.\n3. Use Exercise 3.6 to argue that Lis a context-free language.\n3.8Construct (deterministic or nondeterministic) pushdown automata that\naccept the following languages.\n1.f02n1n:n\u00150g.\n2.f0n1m0n:n\u00151;m\u00151g.\n3.fw2f0;1g\u0003:wcontains more 1s than 0s g.\n4.fwwR:w2f0;1g\u0003g.\n(Ifw=w1:::wn, thenwR=wn:::w 1.)\n5.fw2f0;1g\u0003:wis a palindromeg.\n3.9LetLbe the language\nL=fambn: 0\u0014m\u0014n\u00142mg:\n1. Prove that Lis context-free, by constructing a context-free grammar\nwhose language is equal to L.\n2. Prove that Lis context-free, by constructing a nondeterministic push-\ndown automaton that accepts L.\n3.10 Prove that the following languages are not context-free."
            }
        ]
    },
    {
        "section": "Page 144",
        "content": [
            {
                "type": "text",
                "text": "136 Chapter 3. Context-Free Languages\n\u000ffanba2nba3n:n\u00150g.\n\u000ffanbnanbn:n\u00150g.\n\u000ffambnck:m\u00150;n\u00150;k= max(m;n)g.\n\u000ffw#x:wis a substring of x, andw;x2fa;bg\u0003g.\nFor example, the string aba#abbababbb is in the language, whereas the\nstringaba#baabbaabb is not in the language. The alphabet is fa;b;#g.\n\u000f\nfw2fa;b;cg\u0003:wcontains more b's thana's and\nwcontains more c's thana'sg:\n\u000ff1n:nis a prime number g.\n\u000ff(abn)n:n\u00150g. (The parentheses are not part of the alphabet; thus,\nthe alphabet isfa;b;g.)\n3.11 LetLbe a language consisting of \fnitely many strings. Show that L\nis regular and, therefore, context-free. Let kbe the maximum length of any\nstring inL.\n\u000fProve that every context-free grammar in Chomsky normal form that\ngeneratesLhas more than log kvariables. (The logarithm is in base\n2.)\n\u000fProve that there is a context-free grammar that generates Land that\nhas only one variable.\n3.12 LetLbe a context-free language. Prove that there exists an integer\np\u00151, such that the following is true: For every string sinLwithjsj\u0015p,\nthere exists a string s0inLsuch thatjsj<js0j\u0014jsj+p."
            }
        ]
    },
    {
        "section": "Page 145",
        "content": [
            {
                "type": "text",
                "text": "Chapter 4\nTuring Machines and the\nChurch-Turing Thesis\nIn the previous chapters, we have seen several computational devices that\ncan be used to accept or generate regular and context-free languages. Even\nthough these two classes of languages are fairly large, we have seen in Sec-\ntion 3.8.2 that these devices are not powerful enough to accept simple lan-\nguages such as A=fambncmn:m\u00150;n\u00150g. In this chapter, we introduce\nthe Turing machine, which is a simple model of a real computer. Turing ma-\nchines can be used to accept all context-free languages, but also languages\nsuch asA. We will argue that every problem that can be solved on a real\ncomputer can also be solved by a Turing machine (this statement is known\nas the Church-Turing Thesis). In Chapter 5, we will consider the limitations\nof Turing machines and, hence, of real computers.\n4.1 De\fnition of a Turing machine\nWe start with an informal description of a Turing machine. Such a machine\nconsists of the following, see also Figure 4.1.\n1. There are ktapes , for some \fxed k\u00151. Each tape is divided into\ncells, and is in\fnite both to the left and to the right. Each cell stores\na symbol belonging to a \fnite set \u0000, which is called the tape alphabet .\nThe tape alphabet contains the blank symbol 2. If a cell contains 2,\nthen this means that the cell is actually empty."
            }
        ]
    },
    {
        "section": "Page 146",
        "content": [
            {
                "type": "text",
                "text": "138 Chapter 4. Turing Machines and the Church-Turing Thesis\nstate control\n. . .222aababbabab222. . .?\n. . .222baab2ab222. . .?\nFigure 4.1: A Turing machine with k= 2tapes.\n2. Each tape has a tape head which can move along the tape, one cell\nper move. It can also read the cell it currently scans and replace the\nsymbol in this cell by another symbol.\n3. There is a state control , which can be in any one of a \fnite number of\nstates . The \fnite set of states is denoted by Q. The setQcontains\nthree special states: a start state , an accept state , and a reject state .\nThe Turing machine performs a sequence of computation steps . In one\nsuch step, it does the following:\n1. Immediately before the computation step, the Turing machine is in a\nstaterofQ, and each of the ktape heads is on a certain cell.\n2. Depending on the current state rand theksymbols that are read by\nthe tape heads,\n(a) the Turing machine switches to a state r0ofQ(which may be\nequal tor),\n(b) each tape head writes a symbol of \u0000 in the cell it is currently\nscanning (this symbol may be equal to the symbol currently stored\nin the cell), and"
            }
        ]
    },
    {
        "section": "Page 147",
        "content": [
            {
                "type": "text",
                "text": "4.1. De\fnition of a Turing machine 139\n(c) each tape head either moves one cell to the left, moves one cell to\nthe right, or stays at the current cell.\nWe now give a formal de\fnition of a deterministic Turing machine.\nDe\fnition 4.1.1 Adeterministic Turing machine is a 7-tuple\nM= (\u0006;\u0000;Q;\u000e;q;q accept;qreject);\nwhere\n1. \u0006 is a \fnite set, called the input alphabet ; the blank symbol 2is not\ncontained in \u0006,\n2. \u0000 is a \fnite set, called the tape alphabet ; this alphabet contains the\nblank symbol 2, and \u0006\u0012\u0000,\n3.Qis a \fnite set, whose elements are called states ,\n4.qis an element of Q; it is called the start state ,\n5.qaccept is an element of Q; it is called the accept state ,\n6.qreject is an element of Q; it is called the reject state ,\n7.\u000eis called the transition function , which is a function\n\u000e:Q\u0002\u0000k!Q\u0002\u0000k\u0002fL;R;Ngk:\nThe transition function \u000eis basically the \\program\" of the Turing ma-\nchine. This function tells us what the machine can do in \\one computation\nstep\": Let r2Q, and leta1;a2;:::;ak2\u0000. Furthermore, let r02Q,\na0\n1;a0\n2;:::;a0\nk2\u0000, and\u001b1;\u001b2;:::;\u001bk2fL;R;Ngbe such that\n\u000e(r;a1;a2;:::;ak) = (r0;a0\n1;a0\n2;:::;a0\nk;\u001b1;\u001b2;:::;\u001bk): (4.1)\nThis transition means that if\n\u000fthe Turing machine is in state r, and\n\u000fthe head of the i-th tape reads the symbol ai, 1\u0014i\u0014k,\nthen"
            }
        ]
    },
    {
        "section": "Page 148",
        "content": [
            {
                "type": "text",
                "text": "140 Chapter 4. Turing Machines and the Church-Turing Thesis\n\u000fthe Turing machine switches to state r0,\n\u000fthe head of the i-th tape replaces the scanned symbol aiby the symbol\na0\ni, 1\u0014i\u0014k, and\n\u000fthe head of the i-th tape moves according to \u001bi, 1\u0014i\u0014k: if\u001bi=L,\nthen the tape head moves one cell to the left; if \u001bi=R, then it moves\none cell to the right; if \u001bi=N, then the tape head does not move.\nWe will write the computation step (4.1) in the form of the instruction\nra1a2:::ak!r0a0\n1a0\n2:::a0\nk\u001b1\u001b2:::\u001bk:\nWe now specify the computation of the Turing machine\nM= (\u0006;\u0000;Q;\u000e;q;q accept;qreject):\nStart con\fguration: The input is a string over the input alphabet \u0006.\nInitially, this input string is stored on the \frst tape, and the head of this\ntape is on the leftmost symbol of the input string. Initially, all other k\u00001\ntapes are empty, i.e., only contain blank symbols, and the Turing machine is\nin the start state q.\nComputation and termination: Starting in the start con\fguration, the\nTuring machine performs a sequence of computation steps as described above.\nThe computation terminates at the moment when the Turing machine en-\nters the accept state qaccept or the reject state qreject. (Hence, if the Turing\nmachine never enters the states qaccept andqreject, the computation does not\nterminate.)\nAcceptance: The Turing machine Maccepts the input string w2\u0006\u0003, if the\ncomputation on this input terminates in the state qaccept. If the computation\non this input terminates in the state qreject, thenMrejects the input string\nw.\nWe denote by L(M) the language accepted by the Turing machine M.\nThus,L(M) is the set of all strings in \u0006\u0003that are accepted by M.\nObserve that a string w2\u0006\u0003does not belong to L(M) if and only if on\ninputw,\n\u000fthe computation of Mterminates in the state qreject or\n\u000fthe computation of Mdoes not terminate."
            }
        ]
    },
    {
        "section": "Page 149",
        "content": [
            {
                "type": "text",
                "text": "4.2. Examples of Turing machines 141\n4.2 Examples of Turing machines\n4.2.1 Accepting palindromes using one tape\nWe will show how to construct a Turing machine with one tape, that decides\nwhether or not any input string w2fa;bg\u0003is apalindrome . Recall that the\nstringwis called a palindrome, if reading wfrom left to right gives the same\nresult as reading wfrom right to left. Examples of palindromes are abba,\nbaabbbbaab , and the empty string \u000f.\nStart of the computation: The tape contains the input string w, the tape\nhead is on the leftmost symbol of w, and the Turing machine is in the start\nstateq0.\nIdea: The tape head reads the leftmost symbol of w, deletes this symbol\nand \\remembers\" it by means of a state. Then the tape head moves to\nthe rightmost symbol and tests whether it is equal to the (already deleted)\nleftmost symbol.\n\u000fIf they are equal, then the rightmost symbol is deleted, the tape head\nmoves to the new leftmost symbol, and the whole process is repeated.\n\u000fIf they are not equal, the Turing machine enters the reject state, and\nthe computation terminates.\nThe Turing machine enters the accept state as soon as the string currently\nstored on the tape is empty.\nWe will use the input alphabet \u0006 = fa;bgand the tape alphabet \u0000 =\nfa;b;2g. The setQof states consists of the following eight states:\nq0: start state; tape head is on the leftmost symbol\nqa: leftmost symbol was a; tape head is moving to the right\nqb: leftmost symbol was b; tape head is moving to the right\nq0\na: reached rightmost symbol; test whether it is equal to a, and delete it\nq0\nb: reached rightmost symbol; test whether it is equal to b, and delete it\nqL: test was positive; tape head is moving to the left\nqaccept : accept state\nqreject : reject state"
            }
        ]
    },
    {
        "section": "Page 150",
        "content": [
            {
                "type": "text",
                "text": "142 Chapter 4. Turing Machines and the Church-Turing Thesis\nThe transition function \u000eis speci\fed by the following instructions:\nq0a!qa2R q aa!qaaR q ba!qbaR\nq0b!qb2R q ab!qabR q bb!qbbR\nq02!qaccept qa2!q0\na2L q b2!q0\nb2L\nq0\naa!qL2L q0\nba!qreject qLa!qLaL\nq0\nab!qreject q0\nbb!qL2L q Lb!qLbL\nq0\na2!qaccept q0\nb2!qaccept qL2!q02R\nYou should go through the computation of this Turing machine for some\nsample inputs, for example abba,b,abband the empty string (which is a\npalindrome).\n4.2.2 Accepting palindromes using two tapes\nWe again consider the palindrome problem, but now we use a Turing machine\nwith two tapes.\nStart of the computation: The \frst tape contains the input string wand\nthe head of the \frst tape is on the leftmost symbol of w. The second tape is\nempty and its tape head is at an arbitrary position. The Turing machine is\nin the start state q0.\nIdea: First, the input string wis copied to the second tape. Then the head\nof the \frst tape moves back to the leftmost symbol of w, while the head of\nthe second tape stays at the rightmost symbol of w. Finally, the actual test\nstarts: The head of the \frst tape moves to the right and, at the same time,\nthe head of the second tape moves to the left. While moving, the Turing\nmachine tests whether the two tape heads read the same symbol in each\nstep.\nThe input alphabet is \u0006 = fa;bgand the tape alphabet is \u0000 = fa;b;2g.\nThe setQof states consists of the following \fve states:\nq0: start state; copy wto the second tape\nq1:whas been copied; head of \frst tape moves to the left\nq2: head of \frst tape moves to the right; head of second tape moves\nto the left; until now, all tests were positive\nqaccept : accept state\nqreject : reject state"
            }
        ]
    },
    {
        "section": "Page 151",
        "content": [
            {
                "type": "text",
                "text": "4.2. Examples of Turing machines 143\nThe transition function \u000eis speci\fed by the following instructions:\nq0a2!q0aaRR q 1aa!q1aaLN\nq0b2!q0bbRR q 1ab!q1abLN\nq022!q122LL q 1ba!q1baLN\nq1bb!q1bbLN\nq12a!q22aRN\nq12b!q22bRN\nq122!qaccept\nq2aa!q2aaRL\nq2ab!qreject\nq2ba!qreject\nq2bb!q2bbRL\nq222!qaccept\nAgain, you should run this Turing machine for some sample inputs.\n4.2.3 Accepting anbncnusing one tape\nWe will construct1a Turing machine with one tape that accepts the language\nfanbncn:n\u00150g:\nRecall that we have proved in Section 3.8.2 that this language is not context-\nfree.\nStart of the computation: The tape contains the input string wand the\ntape head is on the leftmost symbol of w. The Turing machine is in the start\nstate.\nIdea: In the previous examples, the tape alphabet \u0000 was equal to the union\nof the input alphabet \u0006 and f2g. In this example, we will add one symbol\ndto the tape alphabet. As we will see, this simpli\fes the construction of\nthe Turing machine. Thus, the input alphabet is \u0006 = fa;b;cgand the tape\nalphabet is \u0000 =fa;b;c;d; 2g. Recall that the input string wbelongs to \u0006\u0003.\nThe general approach is to split the computation into two stages.\n1Thanks to Michael Fleming for pointing out an error in a previous version of this\nconstruction."
            }
        ]
    },
    {
        "section": "Page 152",
        "content": [
            {
                "type": "text",
                "text": "144 Chapter 4. Turing Machines and the Church-Turing Thesis\nStage 1: In this stage, we check if the string wis in the language described\nby the regular expression a\u0003b\u0003c\u0003. If this is the case, then we walk back to\nthe leftmost symbol. For this stage, we use the following states, besides the\nstatesqaccept andqreject:\nqa: start state; we are reading the block of a's\nqb: we are reading the block of b's\nqc: we are reading the block of c's\nqL: walk to the leftmost symbol\nStage 2: In this stage, we repeat the following: Walk along the string from\nleft to right, replace the leftmost abyd, replace the leftmost bbyd, replace\nthe leftmost cbyd, and walk back to the leftmost symbol.\nFor this stage, we use the following states:\nq0\na: start state of Stage 2; search for the leftmost a\nq0\nb: leftmost ahas been replaced by d;\nsearch for the leftmost b\nq0\nc: leftmost ahas been replaced by d;\nleftmostbhas been replaced by d;\nsearch for the leftmost c\nq0\nL: leftmost ahas been replaced by d;\nleftmostbhas been replaced by d;\nleftmostchas been replaced by d;\nwalk to the leftmost symbol\nThe transition function \u000eis speci\fed by the following instructions:\nqaa!qaaR q ba!qreject\nqab!qbbR q bb!qbbR\nqac!qccR q bc!qccR\nqad!cannot happen qbd!cannot happen\nqa2!qL2L q b2!qL2L\nqca!qreject qLa!qLaL\nqcb!qreject qLb!qLbL\nqcc!qccR q Lc!qLcL\nqcd!cannot happen qLd!cannot happen\nqc2!qL2L q L2!q0\na2R"
            }
        ]
    },
    {
        "section": "Page 153",
        "content": [
            {
                "type": "text",
                "text": "4.2. Examples of Turing machines 145\nq0\naa!q0\nbdR q0\nba!q0\nbaR\nq0\nab!qreject q0\nbb!q0\ncdR\nq0\nac!qreject q0\nbc!qreject\nq0\nad!q0\nadR q0\nbd!q0\nbdR\nq0\na2!qaccept q0\nb2!qreject\nq0\nca!qreject q0\nLa!q0\nLaL\nq0\ncb!q0\ncbR q0\nLb!q0\nLbL\nq0\ncc!q0\nLdL q0\nLc!q0\nLcL\nq0\ncd!q0\ncdR q0\nLd!q0\nLdL\nq0\nc2!qreject q0\nL2!q0\na2R\nWe remark that Stage 1 is really necessary for this Turing machine: If we\nomit this stage, and use only Stage 2, then the string aabcbc will be accepted.\n4.2.4 Accepting anbncnusing tape alphabet fa;b;c;2g\nWe consider again the language fanbncn:n\u00150g. In the previous section,\nwe presented a Turing machine that uses an extra symbol d. The reader may\nwonder if we can construct a Turing machine for this language that does not\nuse any extra symbols. We will show below that this is indeed possible.\nStart of the computation: The tape contains the input string wand the\ntape head is on the leftmost symbol of w. The Turing machine is in the start\nstateq0.\nIdea: Repeat the following Stages 1 and 2, until the string is empty.\nStage 1. Walk along the string from left to right, delete the leftmost a,\ndelete the leftmost b, and delete the rightmost c.\nStage 2. Shift the substring of bs andcs one position to the left; then walk\nback to the leftmost symbol.\nThe input alphabet is \u0006 = fa;b;cgand the tape alphabet is \u0000 = fa;b;c;2g."
            }
        ]
    },
    {
        "section": "Page 154",
        "content": [
            {
                "type": "text",
                "text": "146 Chapter 4. Turing Machines and the Church-Turing Thesis\nFor Stage 1, we use the following states:\nq0: start state; tape head is on the leftmost symbol\nqa: leftmost ahas been deleted; have not read b\nqb: leftmost bhas been deleted; have not read c\nqc: leftmost chas been read; tape head moves to the right\nq0\nc: tape head is on the rightmost c\nq1: rightmost chas been deleted; tape head is on the rightmost\nsymbol or 2\nqaccept : accept state\nqreject : reject state\nThe transitions for Stage 1 are speci\fed by the following instructions:\nq0a!qa2R q aa!qaaR\nq0b!qreject qab!qb2R\nq0c!qreject qac!qreject\nq02!qaccept qa2!qreject\nqba!qreject qca!qreject\nqbb!qbbR q cb!qreject\nqbc!qccR q cc!qccR\nqb2!qreject qc2!q0\nc2L\nq0\ncc!q12L\nFor Stage 2, we use the following states:\nq1: as above; tape head is on the rightmost symbol or on 2\nqc: copycone cell to the left\nqb: copybone cell to the left\nq2: done with shifting; head moves to the left\nAdditionally, we use a state q0\n1which has the following meaning: If the input\nstring is of the form aibc, for somei\u00151, then after Stage 1, the tape contains\nthe stringai\u0000122, the tape head is on the 2immediately to the right of the\nas, and the Turing machine is in state q1. In this case, we move one cell to\nthe left; if we then read 2, theni= 1, and we accept; otherwise, we read a,\nand we reject."
            }
        ]
    },
    {
        "section": "Page 155",
        "content": [
            {
                "type": "text",
                "text": "4.2. Examples of Turing machines 147\nThe transitions for Stage 2 are speci\fed by the following instructions:\nq1a!cannot happen q0\n1a!qreject\nq1b!qreject q0\n1b!cannot happen\nq1c!qc2L q0\n1c!cannot happen\nq12!q0\n12L q0\n12!qaccept\nqca!cannot happen qba!cannot happen\nqcb!qbcL qbb!qbbL\nqcc!qccL qbc!cannot happen\nqc2!qreject qb2!q2bL\nq2a!q2aL\nq2b!cannot happen\nq2c!cannot happen\nq22!q02R\n4.2.5 Accepting ambncmnusing one tape\nWe will sketch how to construct a Turing machine with one tape that accepts\nthe language\nfambncmn:m\u00150;n\u00150g:\nRecall that we have proved in Section 3.8.2 that this language is not context-\nfree.\nThe input alphabet is \u0006 = fa;b;cgand the tape alphabet is \u0000 = fa;b;c; $;2g,\nwhere the purpose of the symbol $ will become clear below.\nStart of the computation: The tape contains the input string wand the\ntape head is on the leftmost symbol of w. The Turing machine is in the start\nstate.\nIdea: Observe that a string ambnckis in the language if and only if for every\na, the string contains nmanycs. Based on this, the computation consists of\nthe following stages:\nStage 1. Walk along the input string wfrom left to right and check whether\nwis an element of the language described by the regular expression a\u0003b\u0003c\u0003.\nIf this is not the case, then reject the input string. Otherwise, go to Stage 2.\nStage 2. Walk back to the leftmost symbol of w. Go to Stage 3.\nStage 3. In this stage, the Turing machine does the following:"
            }
        ]
    },
    {
        "section": "Page 156",
        "content": [
            {
                "type": "text",
                "text": "148 Chapter 4. Turing Machines and the Church-Turing Thesis\n\u000fReplace the leftmost aby the blank symbol 2.\n\u000fWalk to the leftmost b.\n\u000fZigzag between the bs andcs; each time, replace the leftmost bby the\nsymbol $, and replace the rightmost cby the blank symbol 2. If, for\nsomeb, there is no cleft, the Turing machine rejects the input string.\n\u000fContinue zigzagging until there are no bs left. Then go to Stage 4.\nObserve that in this third stage, the string ambnckis transformed to the\nstringam\u00001$nck\u0000n.\nStage 4. In this stage, the Turing machine does the following:\n\u000fReplace each $ by b.\n\u000fWalk to the leftmost a.\nHence, in this fourth stage, the string am\u00001$nck\u0000nis transformed to the string\nam\u00001bnck\u0000n.\nObserve that the input string ambnckis in the language if and only if the\nstringam\u00001bnck\u0000nis in the language. Therefore, the Turing machine repeats\nStages 3 and 4, until there are no as left. At that moment, it checks whether\nthere are any cs left; if so, it rejects the input string; otherwise, it accepts\nthe input string.\nWe hope that you believe that this description of the algorithm can be\nturned into a formal description of a Turing machine.\n4.3 Multi-tape Turing machines\nIn Section 4.2, we have seen two Turing machines that accept palindromes;\nthe \frst Turing machine has one tape, whereas the second one has two tapes.\nYou will have noticed that the two-tape Turing machine was easier to obtain\nthan the one-tape Turing machine. This leads to the question whether multi-\ntape Turing machines are more powerful than their one-tape counterparts.\nThe answer is \\no\":\nTheorem 4.3.1 Letk\u00151be an integer. Any k-tape Turing machine can\nbe converted to an equivalent one-tape Turing machine."
            }
        ]
    },
    {
        "section": "Page 157",
        "content": [
            {
                "type": "text",
                "text": "4.3. Multi-tape Turing machines 149\nProof.2We will sketch the proof for the case when k= 2. Let M=\n(\u0006;\u0000;Q;\u000e;q;q accept;qreject) be a two-tape Turing machine. Our goal is to\nconvertMto an equivalent one-tape Turing machine N. That is,Nshould\nhave the property that for all strings w2\u0006\u0003,\n\u000fMacceptswif and only if Nacceptsw,\n\u000fMrejectswif and only if Nrejectsw,\n\u000fMdoes not terminate on input wif and only if Ndoes not terminate\non inputw.\nThe tape alphabet of the one-tape Turing machine Nis\n\u0000[f_x:x2\u0000g[f #g:\nIn words, we take the tape alphabet \u0000 of M, and add, for each x2\u0000, the\nsymbol _x. Moreover, we add a special symbol #.\nThe Turing machine Nwill be de\fned in such a way that any con\fgura-\ntion of the two-tape Turing machine M, for example\n. . .210012. . .\n6\n. . .2aaba2. . .\n6\ncorresponds to the following con\fguration of the one-tape Turing machine\nN:\n. . .2#10_01#a_aba#2. . .\n6\n2Thanks to Sergio Cabello for pointing out an error in a previous version of this proof."
            }
        ]
    },
    {
        "section": "Page 158",
        "content": [
            {
                "type": "text",
                "text": "150 Chapter 4. Turing Machines and the Church-Turing Thesis\nThus, the contents of the two tapes of Mare encoded on the single tape of\nN. The dotted symbols are used to indicate the positions of the two tape\nheads ofM, whereas the three occurrences of the special symbol # are used\nto mark the boundaries of the strings on the two tapes of M.\nThe Turing machine Nsimulates one computation step of M, in the\nfollowing way:\n\u000fThroughout the simulation of this step, N\\remembers\" the current\nstate ofM.\n\u000fAt the start of the simulation, the tape head of Nis on the leftmost\nsymbol #.\n\u000fNwalks along the string to the right until it \fnds the \frst dotted\nsymbol. (This symbol indicates the location of the head on the \frst tape\nofM.)Nremembers this \frst dotted symbol and continues walking\nto the right until it \fnds the second dotted symbol. (This symbol\nindicates the location of the head on the second tape of M.) Again,N\nremembers this second dotted symbol.\n\u000fAt this moment, Nis still at the second dotted symbol. Nupdates\nthis part of the tape, by making the change that Mwould make on its\nsecond tape. (This change is given by the transition function of M; it\ndepends on the current state of Mand the two symbols that Mreads\non its two tapes.)\n\u000fNwalks to the left until it \fnds the \frst dotted symbol. Then, it\nupdates this part of the tape, by making the change that Mwould\nmake on its \frst tape.\n\u000fIn the previous two steps, in which the tape is updated, it may be\nnecessary to shift a part of the tape.\n\u000fFinally,Nremembers the new state of Mand walks back to the left-\nmost symbol #.\nIt should be clear that the Turing machine Ncan be constructed by\nintroducing appropriate states."
            }
        ]
    },
    {
        "section": "Page 159",
        "content": [
            {
                "type": "text",
                "text": "4.4. The Church-Turing Thesis 151\n4.4 The Church-Turing Thesis\nWe all have some intuitive notion of what an algorithm is. This notion will\nprobably be something like \\an algorithm is a procedure consisting of com-\nputation steps that can be speci\fed in a \fnite amount of text\". For example,\nany \\computational process\" that can be speci\fed by a Java program, should\nbe considered an algorithm. Similarly, a Turing machine speci\fes a \\com-\nputational process\" and, therefore, should be considered an algorithm. This\nleads to the question of whether it is possible to give a mathematical de\fni-\ntion of an \\algorithm\". We just saw that every Java program represents an\nalgorithm and that every Turing machine also represents an algorithm. Are\nthese two notions of an algorithm equivalent? The answer is \\yes\". In fact,\nthe following theorem states that many di\u000berent notions of \\computational\nprocess\" are equivalent. (We hope that you have gained su\u000ecient intuition,\nso that none of the claims in this theorem comes as a surprise to you.)\nTheorem 4.4.1 The following computation models are equivalent, i.e., any\none of them can be converted to any other one:\n1. One-tape Turing machines.\n2.k-tape Turing machines, for any k\u00151.\n3. Non-deterministic Turing machines.\n4. Java programs.\n5. C ++programs.\n6. Lisp programs.\nIn other words, if we de\fne the notion of an algorithm using any of the\nmodels in this theorem, then it does not matter which model we take: All\nthese models give the same notion of an algorithm.\nThe problem of de\fning the notion of an algorithm goes back to David\nHilbert. On August 8, 1900, at the Second International Congress of Math-\nematicians in Paris, Hilbert presented a list of problems that he considered\ncrucial for the further development of mathematics. Hilbert's 10th problem\nis the following:"
            }
        ]
    },
    {
        "section": "Page 160",
        "content": [
            {
                "type": "text",
                "text": "152 Chapter 4. Turing Machines and the Church-Turing Thesis\nDoes there exist a \fnite process that decides whether or not any\ngiven polynomial with integer coe\u000ecients has integral roots?\nOf course, in our language, Hilbert asked whether or not there exists an\nalgorithm that decides, when given an arbitrary polynomial equation (with\ninteger coe\u000ecients) such as\n12x3y7z5+ 7x2y4z\u0000x4+y2z7\u0000z3+ 10 = 0;\nwhether or not this equation has a solution in integers. In 1970, Matiyasevich\nproved that such an algorithm does notexist. Of course, in order to prove\nthis claim, we \frst have to agree on what an algorithm is. In the beginning\nof the twentieth century, mathematicians gave several de\fnitions, such as\nTuring machines (1936) and the \u0015-calculus (1936), and they proved that all\nthese are equivalent. Later, after programming languages were invented, it\nwas shown that these older notions of an algorithm are equivalent to notions\nof an algorithm that are based on C programs, Java programs, Lisp programs,\nPascal programs, etc.\nIn other words, all attempts to give a rigorous de\fnition of the notion of\nan algorithm led to the same concept. Because of this, computer scientists\nnowadays agree on what is called the Church-Turing Thesis:\nChurch-Turing Thesis: Every computational process that is intuitively\nconsidered to be an algorithm can be converted to a Turing machine.\nIn other words, this basically states that we de\fne an algorithm to be a\nTuring machine. At this point, you should ask yourself, whether the Church-\nTuring Thesis can be proved . Alternatively, what has to be done in order to\ndisprove this thesis?\nExercises\n4.1Construct a Turing machine with one tape, that accepts the language\nf02n1n:n\u00150g:\nAssume that, at the start of the computation, the tape head is on the leftmost\nsymbol of the input string."
            }
        ]
    },
    {
        "section": "Page 161",
        "content": [
            {
                "type": "text",
                "text": "Exercises 153\n4.2Construct a Turing machine with one tape, that accepts the language\nfw:wcontains twice as many 0s as 1s g:\nAssume that, at the start of the computation, the tape head is on the leftmost\nsymbol of the input string.\n4.3LetAbe the language\nA=fw2fa;b;cg\u0003:wcontains more bs thanas and\nwcontains more cs thanasg:\nGive an informal description (in plain English) of a Turing machine with one\ntape, that accepts the language A.\n4.4Construct a Turing machine with one tape that receives as input a non-\nnegative integer xand returns as output the integer x+ 1. Integers are\nrepresented as binary strings.\nStart of the computation: The tape contains the binary representation\nof the input x. The tape head is on the leftmost symbol and the Turing\nmachine is in the start state q0. For example, if x= 431, the tape looks as\nfollows:\n. . .222110101111222. . .\n6\nEnd of the computation: The tape contains the binary representation of\nthe integer x+ 1. The tape head is on the leftmost symbol and the Turing\nmachine is in the \fnal state q1. For our example, the tape looks as follows:\n. . .222110110000222. . .\n6\nThe Turing machine in this exercise does not have an accept state or a\nreject state; instead, it has a \fnal state q1. As soon as state q1is entered,\nthe Turing machine terminates. At termination, the contents of the tape is\nthe output of the Turing machine."
            }
        ]
    },
    {
        "section": "Page 162",
        "content": [
            {
                "type": "text",
                "text": "154 Chapter 4. Turing Machines and the Church-Turing Thesis\n4.5Construct a Turing machine with two tapes that receives as input two\nnon-negative integers xandy, and returns as output the integer x+y.\nIntegers are represented as binary strings.\nStart of the computation: The \frst tape contains the binary represen-\ntation ofxand its head is on the rightmost symbol of x. The second tape\ncontains the binary representation of yand its head is on the rightmost bit\nofy. At the start, the Turing machine is in the start state q0.\nEnd of the computation: The \frst tape contains the binary representation\nofxand its head is on the rightmost symbol of x. The second tape contains\nthe binary representation of the integer x+y(thus, the integer yis \\gone\").\nThe head of the second tape is on the rightmost bit of x+y. The Turing\nmachine is in the \fnal state q1.\n4.6Give an informal description (in plain English) of a Turing machine with\none tape that receives as input two non-negative integers xandy, and returns\nas output the integer x+y. Integers are represented as binary strings. If you\nare an adventurous student, you may give a formal de\fnition of your Turing\nmachine.\n4.7Construct a Turing machine with one tape that receives as input an\nintegerx\u00151 and returns as output the integer x\u00001. Integers are represented\nin binary.\nStart of the computation: The tape contains the binary representation of\nthe inputx. The tape head is on the rightmost symbol of xand the Turing\nmachine is in the start state q0.\nEnd of the computation: The tape contains the binary representation of\nthe integer x\u00001. The tape head is on the rightmost bit of x\u00001 and the\nTuring machine is in the \fnal state q1.\n4.8Give an informal description (in plain English) of a Turing machine with\nthree tapes that receives as input two non-negative integers xandy, and\nreturns as output the integer xy. Integers are represented as binary strings.\nStart of the computation: The \frst tape contains the binary represen-\ntation ofxand its head is on the rightmost symbol of x. The second tape\ncontains the binary representation of yand its head is on the rightmost sym-\nbol ofy. The third tape is empty and its head is at an arbitrary location.\nThe Turing machine is in the start state q0."
            }
        ]
    },
    {
        "section": "Page 163",
        "content": [
            {
                "type": "text",
                "text": "Exercises 155\nEnd of the computation: The \frst and second tapes are empty. The third\ntape contains the binary representation of the product xyand its head is on\nthe rightmost bit of xy. The Turing machine is in the \fnal state q1.\nHint: Use the Turing machines of Exercises 4.5 and 4.7.\n4.9Construct a Turing machine with one tape that receives as input a string\nof the form 1nfor some integer n\u00150; thus, the input is a string of nmany\n1s. The output of the Turing machine is the string 1n21n. Thus, this Turing\nmachine makes a copy of its input.\nThe input alphabet is \u0006 = f1gand the tape alphabet is \u0000 = f1;2g.\nStart of the computation: The tape contains a string of the form 1n, for\nsome integer n\u00150, the tape head is on the leftmost symbol, and the Turing\nmachine is in the start state. For example, if n= 4, the tape looks as follows:\n. . .2221111222. . .\n6\nEnd of the computation: The tape contains the string 1n21n, the tape\nhead is on the 2in the middle of this string, and the Turing machine is in\nthe \fnal state. For our example, the tape looks as follows:\n. . .222111121111222. . .\n6\nThe Turing machine in this exercise does not have an accept state or a\nreject state; instead, it has a \fnal state. As soon as this state is entered, the\nTuring machine terminates. At termination, the contents of the tape is the\noutput of the Turing machine."
            }
        ]
    },
    {
        "section": "Page 164",
        "content": [
            {
                "type": "text",
                "text": "156 Chapter 4. Turing Machines and the Church-Turing Thesis"
            }
        ]
    },
    {
        "section": "Page 165",
        "content": [
            {
                "type": "text",
                "text": "Chapter 5\nDecidable and Undecidable\nLanguages\nWe have seen in Chapter 4 that Turing machines form a model for \\everything\nthat is intuitively computable\". In this chapter, we consider the limitations\nof Turing machines. That is, we ask ourselves the question whether or not\n\\everything\" is computable. As we will see, the answer is \\no\". In fact, we\nwill even see that \\most\" problems are not solvable by Turing machines and,\ntherefore, not solvable by computers.\n5.1 Decidability\nIn Chapter 4, we have de\fned when a Turing machine accepts an input string\nand when it rejects an input string. Based on this, we de\fne the following\nclass of languages.\nDe\fnition 5.1.1 Let \u0006 be an alphabet and let A\u0012\u0006\u0003be a language. We\nsay thatAisdecidable , if there exists a Turing machine M, such that for\nevery string w2\u0006\u0003, the following holds:\n1. Ifw2A, then the computation of the Turing machine M, on the input\nstringw, terminates in the accept state.\n2. Ifw62A, then the computation of the Turing machine M, on the input\nstringw, terminates in the reject state."
            }
        ]
    },
    {
        "section": "Page 166",
        "content": [
            {
                "type": "text",
                "text": "158 Chapter 5. Decidable and Undecidable Languages\nIn other words, the language Ais decidable, if there exists an algorithm\nthat (i) terminates on every input string w, and (ii) correctly tells us whether\nw2Aorw62A.\nA language Athat is not decidable is called undecidable . For such a\nlanguage, there does notexist an algorithm that satis\fes (i) and (ii) above.\nIn Section 4.2, we have seen several examples of languages that are de-\ncidable.\nIn the following subsections, we will give some examples of decidable and\nundecidable languages. These examples involve languages Awhose elements\nare pairs of the form ( C;w), whereCis some computation model (for ex-\nample, a deterministic \fnite automaton) and wis a string over the alphabet\n\u0006. The pair ( C;w) is in the language Aif and only if the string wis in the\nlanguage of the computation model C. For di\u000berent computation models C,\nwe will ask the question whether Ais decidable, i.e., whether an algorithm\nexists that decides, for any input ( C;w), whether or not this input belongs\nto the language A. Since the input to any algorithm is a string over some\nalphabet, we must encode the pair ( C;w) as a string. In all cases that we\nconsider, such a pair can be described using a \fnite amount of text. There-\nfore, we assume, without loss of generality, that binary strings are used for\nthese encodings. Throughout the rest of this chapter, we will denote the\nbinary encoding of a pair ( C;w) by\nhC;wi:\n5.1.1 The language ADFA\nWe de\fne the following language:\nADFA=fhM;wi:Mis a deterministic \fnite automaton that\naccepts the string wg.\nKeep in mind that hM;widenotes the binary string that forms an en-\ncoding of the \fnite automaton Mand the string wthat is given as input to\nM.\nWe claim that the language ADFAis decidable. In order to prove this,\nwe have to construct an algorithm with the following property, for any given\ninput string u:\n\u000fIfuis the encoding of a deterministic \fnite automaton Mand a string\nw(i.e.,uis in the correct format hM;wi), and ifMacceptsw, then"
            }
        ]
    },
    {
        "section": "Page 167",
        "content": [
            {
                "type": "text",
                "text": "5.1. Decidability 159\nthe algorithm terminates in its accept state.\n\u000fIn all other cases, the algorithm terminates in its reject state.\nAn algorithm that exactly does this, is easy to obtain: On input u, the algo-\nrithm \frst checks whether or not uencodes a deterministic \fnite automaton\nMand a string w. If this is not the case, then it terminates and rejects\nthe input string u. Otherwise, the algorithm \\constructs\" Mandw, and\nthen simulates the computation of Mon the input string w. IfMaccepts\nw, then the algorithm terminates and accepts the input string u. IfMdoes\nnot accept w, then the algorithm terminates and rejects the input string u.\nThus, we have proved the following result:\nTheorem 5.1.2 The language ADFAis decidable.\n5.1.2 The language ANFA\nWe de\fne the following language:\nANFA=fhM;wi:Mis a nondeterministic \fnite automaton that\naccepts the string wg.\nTo prove that this language is decidable, consider the algorithm that\ndoes the following: On input u, the algorithm \frst checks whether or not\nuencodes a nondeterministic \fnite automaton Mand a string w. If this is\nnot the case, then it terminates and rejects the input string u. Otherwise,\nthe algorithm constructs Mandw. Since a computation of M(on inputw)\nis not unique, the algorithm \frst converts Mto an equivalent deterministic\n\fnite automaton N. Then, it proceeds as in Section 5.1.1.\nObserve that the construction for converting a nondeterministic \fnite au-\ntomaton to a deterministic \fnite automaton (see Section 2.5) is algorithmic,\nin the sense that it can be described by an algorithm. Because of this, the\nalgorithm described above is a valid algorithm; it accepts all strings uthat\nare inANFA, and it rejects all strings uthat are not in ANFA. Thus, we have\nproved the following result:\nTheorem 5.1.3 The language ANFAis decidable."
            }
        ]
    },
    {
        "section": "Page 168",
        "content": [
            {
                "type": "text",
                "text": "160 Chapter 5. Decidable and Undecidable Languages\n5.1.3 The language ACFG\nWe de\fne the following language:\nACFG=fhG;wi:Gis a context-free grammar such that w2L(G)g:\nWe claim that this language is decidable. In order to prove this claim, con-\nsider a string uthat encodes a context-free grammar G= (V;\u0006;S;R ) and a\nstringw2\u0006\u0003. Deciding whether or not w2L(G) is equivalent to deciding\nwhether or not S\u0003)w. A \frst idea to decide this is by trying all possible\nderivations that start with the start variable Sand that use rules of R. The\nproblem is that, in case w62L(G), it is not clear how many such derivations\nhave to be checked before we can be sure that wis not in the language of\nG: Ifw2L(G), then it may be that wcan be derived from S, only by \frst\nderiving a very long string, say v, and then use rules to shorten it so as to\nobtain the string w. Since there is no obvious upper bound on the length of\nthe stringv, we have to be careful.\nThe trick is to do the following. First, convert the grammar Gto an\nequivalent grammar G0in Chomsky normal form. (The construction given\nin Section 3.4 can be described by an algorithm.) Let nbe the length of the\nstringw. Then, ifw2L(G) =L(G0), any derivation of winG0, from the\nstart variable of G0, consists of exactly 2 n\u00001 steps (where a \\step\" is de\fned\nas applying one rule of G0). Hence, we can decide whether or not w2L(G),\nby trying all possible derivations, in G0, consisting of 2 n\u00001 steps. If one of\nthese (\fnite number of) derivations leads to the string w, thenw2L(G).\nOtherwise, w62L(G). Thus, we have proved the following result:\nTheorem 5.1.4 The language ACFG is decidable.\nIn fact, the arguments above imply the following result:\nTheorem 5.1.5 Every context-free language is decidable.\nProof. Let \u0006 be an alphabet and let A\u0012\u0006\u0003be an arbitrary context-free\nlanguage. There exists a context-free grammar in Chomsky normal form,\nwhose language is equal to A. Given an arbitrary string w2\u0006\u0003, we have\nseen above how we can decide whether or not wcan be derived from the\nstart variable of this grammar."
            }
        ]
    },
    {
        "section": "Page 169",
        "content": [
            {
                "type": "text",
                "text": "5.1. Decidability 161\n5.1.4 The language ATM\nAfter having seen the languages ADFA,ANFA, andACFG, it is natural to\nconsider the language\nATM=fhM;wi:Mis a Turing machine that accepts the string wg:\nWe will prove that this language is undecidable. Before we give the proof,\nlet us mention what this means:\nThere is no algorithm that, when given an arbitrary algorithm M\nand an arbitrary input string wforM, decides in a \fnite amount\nof time, whether or not Macceptsw.\nThe proof of the claim that ATMis undecidable is by contradiction. Thus,\nwe assume that ATMis decidable. Then there exists a Turing machine H\nthat has the following property. For every input string hM;wiforH:\n\u000fIfhM;wi2ATM(i.e.,Macceptsw), thenHterminates in its accept\nstate.\n\u000fIfhM;wi62ATM(i.e.,MrejectsworMdoes not terminate on input\nw), thenHterminates in its reject state.\n\u000fIn particular, Hterminates on any input hM;wi.\nWe construct a new Turing machine D, that does the following: On input\nhMi, the Turing machine DusesHas a subroutine to determine what M\ndoes when it is given its own description as input. Once Dhas determined\nthis information, it does the opposite of whatHdoes.\nTuring machine D:On inputhMi, whereMis a Turing machine,\nthe new Turing machine Ddoes the following:\nStep 1: Run the Turing machine Hon the inputhM;hMii.\nStep 2:\n\u000fIfHterminates in its accept state, then Dterminates in its\nreject state.\n\u000fIfHterminates in its reject state, then Dterminates in its\naccept state."
            }
        ]
    },
    {
        "section": "Page 170",
        "content": [
            {
                "type": "text",
                "text": "162 Chapter 5. Decidable and Undecidable Languages\nFirst observe that this new Turing machine Dterminates on any input\nstringhMi, becauseHterminates on every input. Next observe that, for any\ninput stringhMiforD:\n\u000fIfhM;hMii2ATM(i.e.,MacceptshMi), thenDterminates in its\nreject state.\n\u000fIfhM;hMii62ATM(i.e.,MrejectshMiorMdoes not terminate on\ninputhMi), thenDterminates in its accept state.\nThis means that for any string hMi:\n\u000fIfMacceptshMi, thenDrejectshMi.\n\u000fIfMrejectshMiorMdoes not terminate on input hMi, thenD\nacceptshMi.\nWe now consider what happens if we give the Turing machine Dthe string\nhDias input, i.e., we take M=D:\n\u000fIfDacceptshDi, thenDrejectshDi.\n\u000fIfDrejectshDiorDdoes not terminate on input hDi, thenDaccepts\nhDi.\nSinceDterminates on every input string, this means that\n\u000fIfDacceptshDi, thenDrejectshDi.\n\u000fIfDrejectshDi, thenDacceptshDi.\nThis is clearly a contradiction. Therefore, the Turing machine Hthat decides\nthe language ATMcannot exist and, thus, ATMis undecidable. We have\nproved the following result:\nTheorem 5.1.6 The language ATMis undecidable."
            }
        ]
    },
    {
        "section": "Page 171",
        "content": [
            {
                "type": "text",
                "text": "5.1. Decidability 163\n5.1.5 The Halting Problem\nWe de\fne the following language:\nHalt =fhP;wi:Pis a Java program that terminates on\nthe input string wg.\nTheorem 5.1.7 The language Halt is undecidable.\nProof. The proof is by contradiction. Thus, we assume that the language\nHalt is decidable. Then there exists a Java program Hthat takes as input a\nstring of the form hP;wi, wherePis an arbitrary Java program and wis an\narbitrary input for P. The program Hhas the following property:\n\u000fIfhP;wi 2 Halt (i.e., program Pterminates on input w), thenH\noutputs true.\n\u000fIfhP;wi62Halt (i.e., program Pdoes not terminate on input w), then\nHoutputs false.\n\u000fIn particular, Hterminates on any input hP;wi.\nWe will write the output of HasH(P;w). Moreover, we will denote by P(w)\nthe computation obtained by running the program Pon the input w. Hence,\nH(P;w) =\u001atrue ifP(w) terminates,\nfalse ifP(w) does not terminate.\nConsider the following algorithm Q, which takes as input the encoding\nhPiof an arbitrary Java program P:\nAlgorithm Q(hPi):\nwhileH(P;hPi) =true\ndohave a beer\nendwhile\nSinceHis a Java program, this new algorithm Qcan also be written as\na Java program. Observe that\nQ(hPi) terminates if and only if H(P;hPi) =false."
            }
        ]
    },
    {
        "section": "Page 172",
        "content": [
            {
                "type": "text",
                "text": "164 Chapter 5. Decidable and Undecidable Languages\nThis means that for every Java program P,\nQ(hPi) terminates if and only if P(hPi) does not terminate. (5.1)\nWhat happens if we run the Java program Qon the input string hQi?\nIn other words, what happens if we run Q(hQi)? Then, in (5.1), we have to\nreplace all occurrences of PbyQ. Hence,\nQ(hQi) terminates if and only if Q(hQi) does not terminate.\nThis is obviously a contradiction, and we can conclude that the Java program\nHdoes not exist. Therefore, the language Halt is undecidable.\nRemark 5.1.8 In this proof, we run the Java program Qon the inputhQi.\nThis means that the input to Qis a description of itself. In other words, we\ngiveQitself as input. This is an example of what is called self-reference . An-\nother example of self-reference can be found in Remark 5.1.8 of the textbook\nIntroduction to Theory of Computation by A. Maheshwari and M. Smid.\n5.2 Countable sets\nThe proofs that we gave in Sections 5.1.4 and 5.1.5 seem to be bizarre. In\nthis section, we will convince you that these proofs in fact use a technique\nthat you have seen in the course COMP 1805: Cantor's Diagonalization .\nLetAandBbe two sets and let f:A!Bbe a function. Recall that f\nis called a bijection , if\n\u000ffisone-to-one (orinjective ), i.e., for any two distinct elements aand\na0inA, we havef(a)6=f(a0), and\n\u000ffisonto (orsurjective ), i.e., for each element b2B, there exists an\nelementa2A, such that f(a) =b.\nThe set of natural numbers is denoted by N. That is, N=f1;2;3;:::g.\nDe\fnition 5.2.1 LetAandBbe two sets. We say that AandBhave the\nsame size , if there exists a bijection f:A!B.\nDe\fnition 5.2.2 LetAbe a set. We say that Aiscountable , ifAis \fnite,\norAandNhave the same size."
            }
        ]
    },
    {
        "section": "Page 173",
        "content": [
            {
                "type": "text",
                "text": "5.2. Countable sets 165\nIn other words, if Ais an in\fnite and countable set, then there exists a\nbijectionf:N!A, and we can write Aas\nA=ff(1);f(2);f(3);f(4);:::g:\nSincefis a bijection, every element of Aoccurs exactly once in the set on\nthe right-hand side. This means that we can number the elements of Ausing\nthe positive integers: Every element of Areceives a unique number.\nTheorem 5.2.3 The following sets are countable:\n1. The set Zof integers:\nZ=f:::;\u00003;\u00002;\u00001;0;1;2;3;:::g:\n2. The Cartesian product N\u0002N:\nN\u0002N=f(m;n) :m2N;n2Ng:\n3. The set Qof rational numbers:\nQ=fm=n :m2Z;n2Z;n6= 0g:\nProof. To prove that the set Zis countable, we have to give each element of\nZa unique number in N. We obtain this numbering, by listing the elements\nofZin the following order:\n0;1;\u00001;2;\u00002;3;\u00003;4;\u00004;:::\nIn this (in\fnite) list, every element of Zoccurs exactly once. The number of\nan element of Zis given by its position in this list.\nFormally, de\fne the function f:N!Zby\nf(n) =\u001an=2 if nis even;\n\u0000(n\u00001)=2 ifnis odd:\nThis function fis a bijection and, therefore, the sets NandZhave the same\nsize. Hence, the set Zis countable.\nFor the proofs of the other two claims, we refer to the course COMP 1805.\nWe now use Cantor's Diagonalization principle to prove that the set of\nreal numbers is not countable:"
            }
        ]
    },
    {
        "section": "Page 174",
        "content": [
            {
                "type": "text",
                "text": "166 Chapter 5. Decidable and Undecidable Languages\nTheorem 5.2.4 The set Rof real numbers is not countable.\nProof. De\fne\nA=fx2R: 0\u0014x<1g:\nWe will prove that the set Ais not countable. This will imply that the set\nRis not countable, because A\u0012R.\nThe proof that Ais not countable is by contradiction. So we assume that\nAis countable. Then there exists a bijection f:N!A. Thus, for each\nn2N,f(n) is a real number between zero and one. We can write\nA=ff(1);f(2);f(3);:::g; (5.2)\nwhere every element of Aoccurs exactly once in the set on the right-hand\nside.\nConsider the real number f(1). We can write this number in decimal\nnotation as\nf(1) = 0:d11d12d13:::;\nwhere each d1iis a digit in the set f0;1;2;:::; 9g. In general, for every n2N,\nwe can write the real number f(n) as\nf(n) = 0:dn1dn2dn3:::;\nwhere, again, each dniis a digit inf0;1;2;:::; 9g.\nWe de\fne the real number\nx= 0:d1d2d3:::;\nwhere, for each integer n\u00151,\ndn=\u001a4 ifdnn6= 4,\n5 ifdnn= 4.\nObserve that xis a real number between zero and one, i.e., x2A. Therefore,\nby (5.2), there is an element n2N, such that f(n) =x. We compare the\nn-th digits of f(n) andx:\n\u000fThen-th digit of f(n) is equal to dnn.\n\u000fThen-th digit of xis equal todn."
            }
        ]
    },
    {
        "section": "Page 175",
        "content": [
            {
                "type": "text",
                "text": "5.2. Countable sets 167\nSincef(n) andxare equal, their n-th digits must be equal, i.e., dnn=dn.\nBut, by the de\fnition of dn, we havednn6=dn. This is a contradiction and,\ntherefore, the set Ais not countable.\nNotice how we de\fned the real number x: For eachn\u00151, then-th digit\nofxisnotequal to the n-th digit off(n). Therefore, for each n\u00151,x6=f(n)\nand, thus,x62A.\nThe \fnal result of this section is the fact that for every set A, its power\nset\nP(A) =fB:B\u0012Ag\nis \\strictly larger\" than A. De\fne the function f:A!P (A) by\nf(a) =fag;\nfor anyainA. Sincefis one-to-one, we can say that P(A) is \\at least as\nlarge as\"A.\nTheorem 5.2.5 LetAbe an arbitrary set. Then AandP(A)do not have\nthe same size.\nProof. The proof is by contradiction. Thus, we assume that there exists a\nbijectiong:A!P (A). De\fne the set Bas\nB=fa2A:a62g(a)g:\nSinceB2P(A) andgis a bijection, there exists an element ainAsuch that\ng(a) =B.\nFirst assume that a2B. Sinceg(a) =B, we havea2g(a). But then,\nfrom the de\fnition of the set B, we havea62B, which is a contradiction.\nNext assume that a62B. Sinceg(a) =B, we havea62g(a). But\nthen, from the de\fnition of the set B, we havea2B, which is again a\ncontradiction.\nWe conclude that the bijection gdoes not exist. Therefore, AandP(A)\ndo not have the same size."
            }
        ]
    },
    {
        "section": "Page 176",
        "content": [
            {
                "type": "text",
                "text": "168 Chapter 5. Decidable and Undecidable Languages\n5.2.1 The Halting Problem revisited\nNow that we know about countability, we give a di\u000berent way to look at the\nproof in Section 5.1.5 of the fact that the language\nHalt =fhP;wi:Pis a Java program that terminates on\nthe input string wg\nis undecidable. You should convince yourself that the proof given below\nfollows the same reasoning as the one used in the proof of Theorem 5.2.4.\nWe \frst argue that the set of all Java programs is countable. Indeed,\nevery Java program Pcan be described by a \fnite amount of text. In fact,\nwe have been using hPito denote such a description by a binary string. For\nany integer n\u00150, there are at most 2n(i.e., \fnitely many) Java programs\nPwhose description hPihas lengthn. Therefore, to obtain a list of all Java\nprograms, we do the following:\n\u000fList all Java programs Pwhose description hPihas length 0. (Well,\nthe empty string does not describe any Java program, so in this step,\nnothing happens.)\n\u000fList all Java programs Pwhose description hPihas length 1.\n\u000fList all Java programs Pwhose description hPihas length 2.\n\u000fList all Java programs Pwhose description hPihas length 3.\n\u000fEtcetera, etcetera.\nIn this in\fnite list, every Java program occurs exactly once. Therefore, the\nset of all Java programs is countable.\nConsider an in\fnite list\nP1;P2;P3;:::\nin which every Java program occurs exactly once.\nAssume that the language Halt is decidable. Then there exists a Java\nprogramHthat decides this language. We may assume that, on input hP;wi,\nHreturns true ifPterminates on input w, and false ifPdoes not terminate\non inputw.\nWe construct a new Java program Dthat does the following:"
            }
        ]
    },
    {
        "section": "Page 177",
        "content": [
            {
                "type": "text",
                "text": "5.3. Rice's Theorem 169\nAlgorithm D:On inputhPni, wherenis a positive integer, the\nnew Java program Ddoes the following:\nStep 1: Run the Java program Hon the inputhPn;hPnii.\nStep 2:\n\u000fIfHreturns true, thenDgoes into an in\fnite loop.\n\u000fIfHreturns false, thenDreturns true and terminates its com-\nputation.\nObserve that Dcan be written as a Java program. Therefore, there exists\nan integern\u00151 such that D=Pn. The next two observations follow from\nthe pseudocode:\n\u000fIfDterminates on input hPni, thenHreturns false on inputhPn;hPnii,\ni.e.,Pndoes not terminate on input hPni.\n\u000fIfDdoes not terminate on input hPni, thenHreturns true on input\nhPn;hPnii, i.e.,Pnterminates on input hPni.\nThus,\n\u000fDterminates on input hPniif and only if Pndoes not terminate on\ninputhPni.\nSinceD=Pn, this becomes\n\u000fDterminates on input hDiif and only if Ddoes not terminate on input\nhDi.\nThus, we have obtained a contradiction.\nRemark 5.2.6 Wede\fned the Java program Din such a way that, for each\nn\u00151, the computation of Don inputhPnidi\u000bers from the computation of\nPnon inputhPni. Hence, for each n\u00151,D6=Pn. However, since Dis a\nJava program, there must be an integer n\u00151 such that D=Pn."
            }
        ]
    },
    {
        "section": "Page 178",
        "content": [
            {
                "type": "text",
                "text": "170 Chapter 5. Decidable and Undecidable Languages\n5.3 Rice's Theorem\nWe have seen two examples of undecidable languages: ATMandHalt. In this\nsection, we prove that many languages involving Turing machines (or Java\nprograms) are undecidable.\nDe\fneTto be the set of binary encodings of all Turing machines, i.e.,\nT=fhMi:Mis a Turing machine with input alphabet f0,1gg:\nTheorem 5.3.1 (Rice) LetPbe a subset ofTsuch that\n1.P6=;, i.e., there exists a Turing machine Msuch thathMi2P ,\n2.Pis a proper subset of T, i.e., there exists a Turing machine Nsuch\nthathNi62P , and\n3. for any two Turing machines M1andM2withL(M1) =L(M2),\n(a) either bothhM1iandhM2iare inPor\n(b) none ofhM1iandhM2iis inP.\nThen the language Pis undecidable.\nYou can think ofPas the set of all Turing machines that satisfy a certain\nproperty. The \frst two conditions state that at least one Turing machine\nsatis\fes this property and not all Turing machines satisfy this property. The\nthird condition states that, for any Turing machine M, whether or not M\nsatis\fes this property only depends on the language L(M) ofM.\nHere are some examples of languages that satisfy the conditions in Rice's\nTheorem:\nP1=fhMi:Mis a Turing machine and \u000f2L(M)g;\nP2=fhMi:Mis a Turing machine and L(M) =f1011;001100gg;\nP3=fhMi:Mis a Turing machine and L(M) is a regular language g:\nYou are encouraged to verify that Rice's Theorem indeed implies that each\nofP1,P2, andP3is undecidable."
            }
        ]
    },
    {
        "section": "Page 179",
        "content": [
            {
                "type": "text",
                "text": "5.3. Rice's Theorem 171\n5.3.1 Proof of Rice's Theorem\nThe strategy of the proof is as follows: Assuming that the language Pis\ndecidable, we show that the language\nHalt =fhM;wi:Mis a Turing machine that terminates on\nthe input string wg\nis decidable. This will contradict Theorem 5.1.7.\nThe assumption that Pis decidable implies the existence of a Turing\nmachineHthat decidesP. Observe that Htakes as input a binary string\nhMiencoding a Turing machine M. In order to show that Halt is decidable,\nwe need a Turing machine that takes as input a binary string hM;wiencoding\na Turing machine Mand a binary string w. In the rest of this section, we\nwill explain how this Turing machine can be obtained.\nLetM1be a Turing machine that, for any input string, switches in its\n\frst computation step from its start state to its reject state. In other words,\nM1is a Turing machine with L(M1) =;. We assume that\nhM1i62P:\n(At the end of the proof, we will consider the case when hM1i2P .) We also\nchoose a Turing machine M2such that\nhM2i2P:\nConsider a \fxed Turing machine Mand a \fxed binary string w. We\nconstruct a new Turing machine TMwthat takes as input an arbitrary binary\nstringx:\nTuring machine TMw(x):\nrun Turing machine Mon inputw;\nifMterminates\nthen runM2on inputx;\nifM2terminates in the accept state\nthen terminate in the accept state\nelse ifM2terminates in the reject state\nthen terminate in the reject state\nendif\nendif\nendif"
            }
        ]
    },
    {
        "section": "Page 180",
        "content": [
            {
                "type": "text",
                "text": "172 Chapter 5. Decidable and Undecidable Languages\nWe determine the language L(TMw) of this new Turing machine. In other\nwords, we determine which strings xare accepted by TMw.\n\u000fAssume that Mterminates on input w, i.e.,hM;wi2Halt. Then it\nfollows from the pseudocode that for any string x,\nxis accepted by TMwif and only if xis accepted by M2.\nThus,L(TMw) =L(M2).\n\u000fAssume that Mdoes not terminate on input w, i.e.,hM;wi62Halt.\nThen it follows from the pseudocode that for any string x,TMwdoes\nnot terminate on input x. Thus,L(TMw) =;. In particular, L(TMw) =\nL(M1).\nRecall thathM1i62P , whereashM2i2P . Then the following follows from\nthe third condition in Rice's Theorem:\n\u000fIfhM;wi2Halt, thenhTMwi2P .\n\u000fIfhM;wi62Halt, thenhTMwi62P .\nThus, we have obtained a connection between the languages Pand Halt.\nThis suggests that we proceed as follows.\nAssume that the language Pis decidable. Let Hbe a Turing machine\nthat decidesP. Then, for any Turing machine M,\n\u000fifhMi2P , thenHaccepts the string hMi,\n\u000fifhMi62P , thenHrejects the string hMi, and\n\u000fHterminates on any input string.\nWe construct a new Turing machine H0that takes as input an arbitrary\nstringhM;wi, whereMis a Turing machine and wis a binary string:\nTuring machine H0(hM;wi):\nconstruct the Turing machine TMwdescribed above;\nrunHon inputhTMwi;\nifHterminates in the accept state\nthen terminate in the accept state\nelse terminate in the reject state\nendif"
            }
        ]
    },
    {
        "section": "Page 181",
        "content": [
            {
                "type": "text",
                "text": "5.4. Enumerability 173\nIt follows from the pseudocode that H0terminates on any input. We\nobserve the following:\n\u000fAssume thathM;wi2Halt. Then we have seen before that hTMwi2P .\nSinceHdecides the language P, it follows that Haccepts the string\nhTMwi. Therefore, from the pseudocode, H0accepts the string hM;wi.\n\u000fAssume thathM;wi62Halt. Then we have seen before that hTMwi62\nP. SinceHdecides the language P, it follows that Hrejects (and\nterminates on) the string hTMwi. Therefore, from the pseudocode, H0\nrejects (and terminates on) the string hM;wi.\nWe have shown that the Turing machine H0decides the language Halt.\nThis is a contradiction and, therefore, we conclude that the language Pis\nundecidable.\nUntil now, we assumed that hM1i62P . IfhM1i2P , then we repeat the\nproof withPreplaced by its complement P. This revised proof then shows\nthatPis undecidable. Since for every language L,\nLis decidable if and only if Lis decidable ;\nwe again conclude that Pis undecidable.\n5.4 Enumerability\nWe now come to the last class of languages in this chapter:\nDe\fnition 5.4.1 Let \u0006 be an alphabet and let A\u0012\u0006\u0003be a language. We\nsay thatAisenumerable , if there exists a Turing machine M, such that for\nevery string w2\u0006\u0003, the following holds:\n1. Ifw2A, then the computation of the Turing machine M, on the input\nstringw, terminates in the accept state.\n2. Ifw62A, then the computation of the Turing machine M, on the input\nstringw, does not terminate in the accept state. That is, either the\ncomputation terminates in the reject state or the computation does not\nterminate."
            }
        ]
    },
    {
        "section": "Page 182",
        "content": [
            {
                "type": "text",
                "text": "174 Chapter 5. Decidable and Undecidable Languages\nIn other words, the language Ais enumerable, if there exists an algorithm\nhaving the following property. If w2A, then the algorithm terminates on\nthe input string wand tells us that w2A. On the other hand, if w62A,\nthen either (i) the algorithm terminates on the input string wand tells us\nthatw62Aor (ii) the algorithm does not terminate on the input string w,\nin which case it does not tell us that w62A.\nIn Section 5.5, we will show where the term \\enumerable\" comes from.\nThe following theorem follows immediately from De\fnitions 5.1.1 and 5.4.1.\nTheorem 5.4.2 Every decidable language is enumerable.\nIn the following subsections, we will give some examples of enumerable\nlanguages.\n5.4.1 Hilbert's problem\nWe have seen Hilbert's problem in Section 4.4: Is there an algorithm that\ndecides, for any given polynomial pwith integer coe\u000ecients, whether or not\nphas integral roots? If we formulate this problem in terms of languages,\nthen Hilbert asked whether or not the language\nHilbert =fhpi:pis a polynomial with integer coe\u000ecients\nthat has an integral root g\nis decidable. As usual, hpidenotes the binary string that forms an encoding\nof the polynomial p.\nAs we mentioned in Section 4.4, it was proven by Matiyasevich in 1970\nthat the language Hilbert is not decidable. We claim, that this language\nis enumerable. In order to prove this claim, we have to construct an al-\ngorithm Hilbert with the following property: For any input polynomial p\nwith integer coe\u000ecients,\n\u000fifphas an integral root, then algorithm Hilbert will \fnd one in a\n\fnite amount of time,\n\u000fifpdoes not have an integral root, then either algorithm Hilbert ter-\nminates and tells us that pdoes not have an integral root, or algorithm\nHilbert does not terminate."
            }
        ]
    },
    {
        "section": "Page 183",
        "content": [
            {
                "type": "text",
                "text": "5.4. Enumerability 175\nRecall that Zdenotes the set of integers. Algorithm Hilbert does the\nfollowing, on any input polynomial pwith integer coe\u000ecients. Let nde-\nnote the number of variables in p. Algorithm Hilbert tries all elements\n(x1;x2;:::;xn)2Zn, in a systematic way, and for each such element, it\ncomputesp(x1;x2;:::;xn). If this value is zero, then algorithm Hilbert\nterminates and accepts the input.\nWe observe the following:\n\u000fIfhpi2 Hilbert , then algorithm Hilbert terminates and accepts p,\nprovided we are able to visit all elements ( x1;x2;:::;xn)2Znin a\n\\systematic way\".\n\u000fIfhpi62Hilbert , thenp(x1;x2;:::;xn)6= 0 for all ( x1;x2;:::;xn)2Zn\nand, therefore, algorithm Hilbert does not terminate.\nThese are exactly the requirements for the language Hilbert to be enumerable.\nIt remains to explain how we visit all elements ( x1;x2;:::;xn)2Znin a\nsystematic way. For any integer d\u00150, letHddenote the hypercube in Zn\nwith sides of length 2 dthat is centered at the origin. That is, Hdconsists\nof the set of all points ( x1;x2;:::;xn) inZn, such that\u0000d\u0014xi\u0014dfor all\n1\u0014i\u0014nand there exists at least one index jfor whichxj=dorxj=\u0000d.\nWe observe that Hdcontains a \fnite number of elements. In fact, if d\u00151,\nthen this number is equal to (2 d+ 1)n\u0000(2d\u00001)n. The algorithm will visit\nall elements ( x1;x2;:::;xn)2Zn, in the following order: First, it visits the\norigin, which is the only element of H0. Then, it visits all elements of H1,\nfollowed by all elements of H2, etc., etc.\nTo summarize, we obtain the following algorithm, proving that the lan-\nguage Hilbert is enumerable:"
            }
        ]
    },
    {
        "section": "Page 184",
        "content": [
            {
                "type": "text",
                "text": "176 Chapter 5. Decidable and Undecidable Languages\nAlgorithm Hilbert (hpi):\nn:= the number of variables in p;\nd:= 0;\nwhiled\u00150\ndo for each (x1;x2;:::;xn)2Hd\ndoR:=p(x1;x2;:::;xn);\nifR= 0\nthen terminate and accept\nendif\nendfor ;\nd:=d+ 1\nendwhile\nTheorem 5.4.3 The language Hilbert is enumerable.\n5.4.2 The language ATM\nWe have shown in Section 5.1.4 that the language\nATM=fhM;wi:Mis a Turing machine that accepts the string wg:\nis undecidable. In this section, we will prove that this language is enumerable.\nThus, we have to construct an algorithm Phaving the following property,\nfor any given input string u:\n\u000fIf\n{uencodes a Turing machine Mand an input string wforM(i.e.,\nuis in the correct format hM;wi) and\n{hM;wi2ATM(i.e.,Macceptsw),\nthen algorithm Pterminates in its accept state.\n\u000fIn all other cases, either algorithm Pterminates in its reject state, or\nalgorithmPdoes not terminate.\nOn input string u=hM;wi, which is in the correct format, algorithm Pdoes\nthe following:"
            }
        ]
    },
    {
        "section": "Page 185",
        "content": [
            {
                "type": "text",
                "text": "5.5. Where does the term \\enumerable\" come from? 177\n1. It simulates the computation of Mon inputw.\n2. IfMterminates in its accept state, then Pterminates in its accept\nstate.\n3. IfMterminates in its reject state, then Pterminates in its reject state.\n4. IfMdoes not terminate, then Pdoes not terminate.\nHence, ifu=hM;wi2ATM, thenMacceptswand, therefore, Paccepts\nu. On the other hand, if u=hM;wi62ATM, thenMdoes not accept w. This\nmeans that, on input w,Meither terminates in its reject state or does not\nterminate. But this implies that, on input u,Peither terminates in its reject\nstate or does not terminate. This proves that algorithm Phas the properties\nthat are needed in order to show that the language ATMis enumerable. We\nhave proved the following result:\nTheorem 5.4.4 The language ATMis enumerable.\n5.5 Where does the term \\enumerable\" come\nfrom?\nIn De\fnition 5.4.1, we have de\fned what it means for a language to be\nenumerable. In this section, we will see where this term comes from.\nDe\fnition 5.5.1 Let \u0006 be an alphabet and let A\u0012\u0006\u0003be a language. An\nenumerator forAis a Turing machine Ehaving the following properties:\n1. Besides the standard features as in Section 4.1, Ehas a print tape and\naprint state . During its computation, Ewrites symbols of \u0006 on the\nprint tape. Each time, Eenters the print state, the current string on\nthe print tape is sent to the printer and the print tape is made empty.\n2. At the start of the computation, all tapes are empty and Eis in the\nstart state.\n3. Every string winAis sent to the printer at least once.\n4. Every string wthat is not in Ais never sent to the printer."
            }
        ]
    },
    {
        "section": "Page 186",
        "content": [
            {
                "type": "text",
                "text": "178 Chapter 5. Decidable and Undecidable Languages\nThus, an enumerator EforAreally enumerates all strings in the language\nA. There is no particular order in which the strings of Aare sent to the\nprinter. Moreover, a string in Amay be sent to the printer multiple times.\nIf the language Ais in\fnite, then the Turing machine Eobviously does not\nterminate; however, every string in A(and only strings in A) will be sent to\nthe printer at some time during the computation.\nTo give an example, let A=f0n:n\u00150g. The following Turing machine\nis an enumerator for A.\nTuring machine StringsOfZeros :\nn:= 0;\nwhile 1 + 1 = 2\ndo fori:= 1ton\ndowrite 0 on the print tape\nendfor ;\nenter the print state;\nn:=n+ 1\nendwhile\nIn the rest of this section, we will prove the following result.\nTheorem 5.5.2 A language is enumerable if and only if it has an enumer-\nator.\nFor the \frst part of the proof, assume that the language Ahas an enu-\nmeratorE. We construct the following Turing machine M, which takes an\narbitrary string was input:\nTuring machine M(w):\nrunE; every time Eenters the print state:\nletvbe the string on the print tape;\nifw=v\nthen terminate in the accept state\nendif\nThe Turing machine Mhas the following properties:\n\u000fIfw2A, thenwwill be sent to the printer at some time during the"
            }
        ]
    },
    {
        "section": "Page 187",
        "content": [
            {
                "type": "text",
                "text": "5.5. Where does the term \\enumerable\" come from? 179\ncomputation of E. It follows from the pseudocode that, on input w,\nMterminates in the accept state.\n\u000fIfw62A, thenEwill never sent wto the printer. It follows from the\npseudocode that, on input w,Mdoes not terminate.\nThus,Msatis\fes the conditions in De\fnition 5.4.1. We conclude that the\nlanguageAis enumerable.\nTo prove the converse, we now assume that Ais enumerable. Let Mbe\na Turing machine that satis\fes the conditions in De\fnition 5.4.1.\nWe \fx an in\fnite list\ns1;s2;s3;:::\nof all strings in \u0006\u0003. For example, if \u0006 = f0;1g, then we can take this list to\nbe\n\u000f;0;1;00;01;10;11;000;001;010;100;011;101;110;111;:::\nWe construct the following Turing machine E, which takes the empty\nstring as input:\nTuring machine E:\nn:= 1;\nwhile 1 + 1 = 2\ndo fori:= 1ton\ndorunMfornsteps on the input string si;\nifMacceptssiwithinnsteps\nthen writesion the print tape;\nenter the print state\nendif\nendfor ;\nn:=n+ 1\nendwhile\nWe claim that Eis an enumerator for the language A. To prove this, it\nis obvious that any string that is sent to the printer by Ebelongs toA.\nIt remains to prove that every string in Awill be sent to the printer by E.\nLetwbe a string in A. Then, on input w, the Turing machine Mterminates\nin the accept state. Let mbe the number of steps made by Mon inputw.\nLetibe the index such that w=si. De\fnen= max(m;i). Consider the"
            }
        ]
    },
    {
        "section": "Page 188",
        "content": [
            {
                "type": "text",
                "text": "180 Chapter 5. Decidable and Undecidable Languages\nn-th iteration of the while-loop and the i-th iteration of the for-loop. In this\niteration,Macceptssi=winm\u0014nsteps and, therefore, wis sent to the\nprinter.\n5.6 Most languages are not enumerable\nIn this section, we will prove that most languages are not enumerable. The\nproof is based on the following two facts:\n\u000fThe set consisting of all enumerable languages is countable; we will\nprove this in Section 5.6.1.\n\u000fThe set consisting of all languages is not countable; we will prove this\nin Section 5.6.2.\n5.6.1 The set of enumerable languages is countable\nWe de\fne the set Eas\nE=fA:A\u0012f0;1g\u0003is an enumerable language g:\nIn words,Eis the set whose elements are the enumerable languages. Every\nelement ofEis an enumerable language. Hence, every element of the set E\nis itself a set consisting of strings.\nLemma 5.6.1 The setEis countable.\nProof. LetA\u0012f0;1g\u0003be an enumerable language. There exists a Turing\nmachineTAthat satis\fes the conditions in De\fnition 5.4.1. This Turing\nmachineTAcan be uniquely speci\fed by a string in English. This string can\nbe converted to a binary string sA. Hence, the binary string sAis a unique\nencoding of the Turing machine TA.\nConsider the set\nS=fsA:A\u0012f0;1g\u0003is an enumerable language g:\nObserve that the function f:E!S , de\fned by f(A) =sAfor eachA2E,\nis a bijection. Therefore, the sets EandShave the same size. Hence, in\norder to prove that the set Eis countable, it is su\u000ecient to prove that the\nsetSis countable."
            }
        ]
    },
    {
        "section": "Page 189",
        "content": [
            {
                "type": "text",
                "text": "5.6. Most languages are not enumerable 181\nWhy is the setScountable? For each integer n\u00150, there are exactly 2n\nbinary strings of length n. Since there are binary strings that are not encod-\nings of Turing machines, the set Scontains at most 2nstrings of length n.\nIn particular, the number of strings in Shaving length nis \fnite. Therefore,\nwe obtain an in\fnite list of the elements of Sin the following way:\n\u000fList all strings in Shaving length 0. (Well, the empty string is not in\nS, so in this step, nothing happens.)\n\u000fList all strings in Shaving length 1.\n\u000fList all strings in Shaving length 2.\n\u000fList all strings in Shaving length 3.\n\u000fEtcetera, etcetera.\nIn this in\fnite list, every element of Soccurs exactly once. Therefore, Sis\ncountable.\n5.6.2 The set of all languages is not countable\nWe de\fne the set Las\nL=fA:A\u0012f0;1g\u0003is a languageg:\nIn words,Lis the set consisting of all languages. Every element of the set L\nis a set consisting of strings.\nLemma 5.6.2 The setLis not countable.\nProof. We de\fne the set Bas\nB=fw:wis an in\fnite binary sequence g:\nWe claim that this set is not countable. The proof of this claim is almost\nidentical to the proof of Theorem 5.2.4. We assume that the set Bis count-\nable. Then there exists a bijection f:N!B. Thus, for each n2N,f(n) is\nan in\fnite binary sequence. We can write\nB=ff(1);f(2);f(3);:::g; (5.3)"
            }
        ]
    },
    {
        "section": "Page 190",
        "content": [
            {
                "type": "text",
                "text": "182 Chapter 5. Decidable and Undecidable Languages\nwhere every element of Boccurs exactly once in the set on the right-hand\nside.\nWe de\fne the in\fnite binary sequence w=w1w2w3:::, where, for each\nintegern\u00151,\nwn=\u001a1 if then-th bit off(n) is 0,\n0 if then-th bit off(n) is 1.\nSincew2B, it follows from (5.3) that there is an element n2N, such that\nf(n) =w. Hence, the n-th bits off(n) andware equal. But, by de\fnition,\nthesen-th bits are not equal. This is a contradiction and, therefore, the set\nBis not countable.\nIn the rest of the proof, we will show that the sets LandBhave the same\nsize. SinceBis not countable, this will imply that Lis not countable.\nIn order to prove that LandBhave the same size, we have to show that\nthere exists a bijection\ng:L!B:\nWe \frst observe that the set f0;1g\u0003is countable, because for each integer\nn\u00150, there are only \fnitely many (to be precise, exactly 2n) binary strings\nof lengthn. In fact, we can write\nf0;1g\u0003=f\u000f;0;1;00;01;10;11;000;001;010;100;011;101;110;111;:::g:\nFor each integer n\u00151, we denote by snthen-th string in this list. Hence,\nf0;1g\u0003=fs1;s2;s3;:::g: (5.4)\nNow we are ready to de\fne the bijection g:L ! B : LetA2 L, i.e.,\nA\u0012f0;1g\u0003is a language. We de\fne the in\fnite binary sequence g(A) as\nfollows: For each integer n\u00151, then-th bit ofg(A) is equal to\n\u001a1 ifsn2A,\n0 ifsn62A.\nIn words, the in\fnite binary sequence g(A) contains a 1 exactly in those\npositionsnfor which the string snin (5.4) is in the language A.\nTo give an example, assume that Ais the language consisting of all binary\nstrings that start with 0. The following table gives the corresponding in\fnite\nbinary sequence g(A) (this sequence is obtained by reading the rightmost\ncolumn from top to bottom):"
            }
        ]
    },
    {
        "section": "Page 191",
        "content": [
            {
                "type": "text",
                "text": "5.6. Most languages are not enumerable 183\nf0;1g\u0003Ag(A)\n\u000f not inA 0\n0 inA 1\n1 not inA 0\n00 inA 1\n01 inA 1\n10 not inA 0\n11 not inA 0\n000 inA 1\n001 inA 1\n010 inA 1\n100 not inA 0\n011 inA 1\n101 not inA 0\n110 not inA 0\n111 not inA 0\n.........\nThe function gde\fned above has the following properties:\n\u000fIfAandA0are two di\u000berent languages in L, theng(A)6=g(A0).\n\u000fFor every in\fnite binary sequence winB, there exists a language Ain\nL, such that g(A) =w.\nThis means that the function gis a bijection from LtoB.\n5.6.3 There are languages that are not enumerable\nWe have proved that the set\nE=fA:A\u0012f0;1g\u0003is an enumerable language g\nis countable, whereas the set\nL=fA:A\u0012f0;1g\u0003is a languageg\nis not countable. This means that there are \\more\" languages in Lthan\nthere are inE, proving the following result:"
            }
        ]
    },
    {
        "section": "Page 192",
        "content": [
            {
                "type": "text",
                "text": "184 Chapter 5. Decidable and Undecidable Languages\nTheorem 5.6.3 There exist languages that are not enumerable.\nThe proof given above shows the existence of languages that are not\nenumerable. However, the proof does not give us a speci\fc example of a\nlanguage that is not enumerable. In the next sections, we will see examples\nof such languages. Before we move on to these examples, we mention the\ndi\u000berence between being countable and being enumerable:\n\u000fAny language Ais countable, i.e., we can number the elements of A\nand, thus, write\nA=fs1;s2;s3;s4;:::g:\n\u000fIf the language Ais enumerable, then, by Theorem 5.5.2, there is an\nalgorithm that produces this numbering.\n\u000fIf the language Ais not enumerable, then, again by Theorem 5.5.2,\nthere does not exist an algorithm that produces this numbering.\n5.7 The relationship between decidable and\nenumerable languages\nWe know from Theorem 5.4.2 that every decidable language is enumerable.\nOn the other hand, we know from Theorems 5.1.6 and 5.4.4 that the converse\nis not true. The following result should not come as a surprise:\nTheorem 5.7.1 Let\u0006be an alphabet and let A\u0012\u0006\u0003be a language. Then,\nAis decidable if and only if both Aand its complement Aare enumerable.\nProof. We \frst assume that Ais decidable. Then, by Theorem 5.4.2, A\nis enumerable. Since Ais decidable, it is not di\u000ecult to see that Ais also\ndecidable. Then, again by Theorem 5.4.2, Ais enumerable.\nTo prove the converse, we assume that both AandAare enumerable.\nSinceAis enumerable, there exists a Turing machine M1, such that for any\nstringw2\u0006\u0003, the following holds:\n\u000fIfw2A, then the computation of M1, on the input string w, terminates\nin the accept state of M1."
            }
        ]
    },
    {
        "section": "Page 193",
        "content": [
            {
                "type": "text",
                "text": "5.7. Decidable versus enumerable languages 185\n\u000fIfw62A, then the computation of M1, on the input string w, terminates\nin the reject state of M1or does not terminate.\nSimilarly, since Ais enumerable, there exists a Turing machine M2, such that\nfor any string w2\u0006\u0003, the following holds:\n\u000fIfw2A, then the computation of M2, on the input string w, terminates\nin the accept state of M2.\n\u000fIfw62A, then the computation of M2, on the input string w, terminates\nin the reject state of M2or does not terminate.\nWe construct a two-tape Turing machine M:\nTwo-tape Turing machine M:For any input string w2\u0006\u0003,M\ndoes the following:\n\u000fMsimulates the computation of M1, on inputw, on the \frst\ntape, and, simultaneously, it simulates the computation of M2,\non inputw, on the second tape.\n\u000fIf the simulation of M1terminates in the accept state of M1,\nthenMterminates in its accept state.\n\u000fIf the simulation of M2terminates in the accept state of M2,\nthenMterminates in its reject state.\nObserve the following:\n\u000fIfw2A, thenM1terminates in its accept state and, therefore, M\nterminates in its accept state.\n\u000fIfw62A, thenM2terminates in its accept state and, therefore, M\nterminates in its reject state.\nWe conclude that the Turing machine Maccepts all strings in A, and rejects\nall strings that are not in A. This proves that the language Ais decidable.\nWe now use Theorem 5.7.1 to give examples of languages that are not\nenumerable:"
            }
        ]
    },
    {
        "section": "Page 194",
        "content": [
            {
                "type": "text",
                "text": "186 Chapter 5. Decidable and Undecidable Languages\nTheorem 5.7.2 The language ATMis not enumerable.\nProof. We know from Theorems 5.4.4 and 5.1.6 that the language ATMis\nenumerable but not decidable. Combining these facts with Theorem 5.7.1\nimplies that the language ATMis not enumerable.\nThe following result can be proved in exactly the same way:\nTheorem 5.7.3 The language Halt is not enumerable.\n5.8 A language Asuch that both AandAare\nnot enumerable\nIn Theorem 5.7.2, we have seen that the complement of the language ATM\nis not enumerable. In Theorem 5.4.4, however, we have shown that the\nlanguageATMitself is enumerable. In this section, we consider the language\nEQTM=fhM1;M2i:M1andM2are Turing machines\nandL(M1) =L(M2)g.\nWe will show the following result:\nTheorem 5.8.1 Both EQTMand its complement EQTMare not enumer-\nable.\n5.8.1 EQTMis not enumerable\nConsider a \fxed Turing machine Mand a \fxed binary string w. We construct\na new Turing machine TMwthat takes as input an arbitrary binary string x:\nTuring machine TMw(x):\nrun Turing machine Mon inputw;\nterminate in the accept state\nWe determine the language L(TMw) of this new Turing machine. In other\nwords, we determine which strings xare accepted by TMw.\n\u000fAssume that Mterminates on input w, i.e.,hM;wi2Halt. Then it\nfollows from the pseudocode that every string xis accepted by TMw.\nThus,L(TMw) =f0;1g\u0003."
            }
        ]
    },
    {
        "section": "Page 195",
        "content": [
            {
                "type": "text",
                "text": "5.8. Both AandAnot enumerable 187\n\u000fAssume that Mdoes not terminate on input w, i.e.,hM;wi2Halt.\nThen it follows from the pseudocode that, for any string x,TMwdoes\nnot terminate on input x. Thus,L(TMw) =;.\nAssume that the language EQTMis enumerable. We will show that Halt\nis enumerable as well, which will contradict Theorem 5.7.3.\nSince EQTMis enumerable, there exists a Turing machine Hhaving the\nfollowing property, for any two Turing machines M1andM2:\n\u000fIfL(M1) =L(M2), then, on inputhM1;M2i,Hterminates in the accept\nstate.\n\u000fIfL(M1)6=L(M2), then, on input hM1;M2i,Heither terminates in\nthe reject state or does not terminate.\nWe construct a new Turing machine H0that takes as input an arbitrary\nstringhM;wi, whereMis a Turing machine and wis a binary string:\nTuring machine H0(hM;wi):\nconstruct a Turing machine M1that rejects every input string;\nconstruct the Turing machine TMwdescribed above;\nrunHon inputhM1;TMwi;\nifHterminates in the accept state\nthen terminate in the accept state\nelse ifHterminates in the reject state\nthen terminate in the reject state\nendif\nendif\nWe observe the following:\n\u000fAssume thathM;wi2Halt. Then we have seen before that L(TMw) =\n;. By our choice of M1, we haveL(M1) =;as well. Therefore, H\naccepts (and terminates on) the input hM1;TMwi. It follows from the\npseudocode that H0accepts (and terminates on) the string hM;wi.\n\u000fAssume thathM;wi62Halt, i.e.,hM;wi2Halt. Then we have seen\nbefore that L(TMw)6=;=L(M1). Therefore, on input hM1;TMwi,H\neither terminates in the reject state or does not terminate. It follows"
            }
        ]
    },
    {
        "section": "Page 196",
        "content": [
            {
                "type": "text",
                "text": "188 Chapter 5. Decidable and Undecidable Languages\nfrom the pseudocode that, on input hM;wi,H0either terminates in the\nreject state or does not terminate.\nThus, the Turing machine H0has the properties needed to show that\nthe language Halt is enumerable. This is a contradiction and, therefore, we\nconclude that the language EQTMis not enumerable.\n5.8.2 EQTMis not enumerable\nThis proof is symmetric to the one in Section 5.8.1. For a \fxed Turing\nmachineMand a \fxed binary string w, we will use the same Turing machine\nTMwas in Section 5.8.1.\nAssume that the language EQTMis enumerable. We will show that Halt\nis enumerable as well, which will contradict Theorem 5.7.3.\nSince EQTMis enumerable, there exists a Turing machine Hhaving the\nfollowing property, for any two Turing machines M1andM2:\n\u000fIfL(M1)6=L(M2), then, on inputhM1;M2i,Hterminates in the accept\nstate.\n\u000fIfL(M1) =L(M2), then, on input hM1;M2i,Heither terminates in\nthe reject state or does not terminate.\nWe construct a new Turing machine H0that takes as input an arbitrary\nstringhM;wi, whereMis a Turing machine and wis a binary string:\nTuring machine H0(hM;wi):\nconstruct a Turing machine M1that accepts every input string;\nconstruct the Turing machine TMwof Section 5.8.1;\nrunHon inputhM1;TMwi;\nifHterminates in the accept state\nthen terminate in the accept state\nelse ifHterminates in the reject state\nthen terminate in the reject state\nendif\nendif\nWe observe the following:"
            }
        ]
    },
    {
        "section": "Page 197",
        "content": [
            {
                "type": "text",
                "text": "Exercises 189\n\u000fAssume thathM;wi2Halt. Then we have seen before that L(TMw) =\n;. Thus, by our choice of M1, we haveL(TMw)6=L(M1). Therefore, H\naccepts (and terminates on) the input hM1;TMwi. It follows from the\npseudocode that H0accepts (and terminates on) the string hM;wi.\n\u000fAssume thathM;wi62Halt. ThenL(TMw) =f0;1g\u0003=L(M1) and, on\ninputhM1;TMwi,Heither terminates in the reject state or does not\nterminate. It follows from the pseudocode that, on input hM;wi,H0\neither terminates in the reject state or does not terminate.\nThus, the Turing machine H0has the properties needed to show that\nthe language Halt is enumerable. This is a contradiction and, therefore, we\nconclude that the language EQTMis not enumerable.\nExercises\n5.1Prove that the language\nfw2f0;1g\u0003:wis the binary representation of 2nfor somen\u00150g\nis decidable. In other words, construct a Turing machine that gets as input\nan arbitrary number x2N, represented in binary as a string w, and that\ndecides whether or not xis a power of two.\n5.2LetFbe the set of all functions f:N!N. Prove that Fis not\ncountable.\n5.3A function f:N!Nis called computable , if there exists a Turing\nmachine, that gets as input an arbitrary positive integer n, written in binary,\nand gives as output the value of f(n), again written in binary. This Turing\nmachine has a \fnal state. As soon as the Turing machine enters this \fnal\nstate, the computation terminates, and the output is the binary string that\nis written on its tape.\nProve that there exist functions f:N!Nthat are not computable.\n5.4Letnbe a \fxed positive integer, and let kbe the number of bits in the\nbinary representation of n. (Hence,k= 1 +blognc.) Construct a Turing\nmachine with one tape, tape alphabet f0;1;2g, and exactly k+ 1 states\nq0;q1;:::;qk, that does the following:"
            }
        ]
    },
    {
        "section": "Page 198",
        "content": [
            {
                "type": "text",
                "text": "190 Chapter 5. Decidable and Undecidable Languages\nStart of the computation: The tape is empty, i.e., every cell of the tape\ncontains 2, and the Turing machine is in the start state q0.\nEnd of the computation: The tape contains the binary representation of\nthe integer n, the tape head is on the rightmost bit of the binary represen-\ntation ofn, and the Turing machine is in the \fnal state qk.\nThe Turing machine in this exercise does not have an accept state or a\nreject state; instead, it has a \fnal state qk. As soon as state qkis entered,\nthe Turing machine terminates.\n5.5Give an informal description (in plain English) of a Turing machine\nwith three tapes, that gets as input the binary representation of an arbitrary\nintegerm\u00151, and returns as output the unary representation of m.\nStart of the computation: The \frst tape contains the binary representa-\ntion of the input m. The other two tapes are empty (i.e., contain only 2s).\nThe Turing machine is in the start state.\nEnd of the computation: The third tape contains the unary representation\nofm, i.e., a string consisting of mmany ones. The Turing machine is in the\n\fnal state.\nThe Turing machine in this exercise does not have an accept state or a\nreject state; instead, it has a \fnal state. As soon as this \fnal state is entered,\nthe Turing machine terminates.\nHint: Use the second tape to maintain a string of ones, whose length is\na power of two.\n5.6In this exercise, you are asked to prove that the busy beaver function\nBB:N!Nis not computable.\nFor any integer n\u00151, we de\fne TMnto be the set of all Turing machines\nM, such that\n\u000fMhas one tape,\n\u000fMhas exactly nstates,\n\u000fthe tape alphabet of Misf0;1;2g, and\n\u000fMterminates, when given the empty string \u000fas input."
            }
        ]
    },
    {
        "section": "Page 199",
        "content": [
            {
                "type": "text",
                "text": "Exercises 191\nFor every Turing machine M2TMn, we de\fne f(M) to be the number of\nones on the tape, after the computation of M, on the empty input string,\nhas terminated.\nThe busy beaver function BB:N!Nis de\fned as\nBB(n) := maxff(M) :M2TMng;for everyn\u00151.\nIn words, BB(n) is the maximum number of ones that any Turing machine\nwithnstates can produce, when given the empty string as input, and as-\nsuming the Turing machine terminates on this input.\nProve that the function BBis not computable.\nHint: Assume that BBis computable. Then there exists a Turing ma-\nchineMthat, for any given n\u00151, computes the value of BB(n). Fix a large\nintegern\u00151. De\fne (in plain English) a Turing machine that, when given\nthe empty string as input, terminates and outputs a string consisting of more\nthan BB(n) many ones. Use Exercises 5.4 and 5.5 to argue that there exists\nsuch a Turing machine having O(logn) states. Then, if you assume that n\nis large enough, the number of states is at most n.\n5.7Since the set\nT=fM:Mis a Turing machine g\nis countable, there is an in\fnite list\nM1;M2;M3;M4;:::;\nsuch that every Turing machine occurs exactly once in this list.\nFor any positive integer n, lethnidenote the binary representation of n;\nobserve thathniis a binary string.\nLetAbe the language de\fned as\nA=fhni: the Turing machine Mnterminates on the input string hni,\nand it rejects this string g:\nProve that the language Ais undecidable.\n5.8Consider the three languages\nEmpty =fhMi:Mis a Turing machine for which L(M) =;g;"
            }
        ]
    },
    {
        "section": "Page 200",
        "content": [
            {
                "type": "text",
                "text": "192 Chapter 5. Decidable and Undecidable Languages\nUselessState =fhM;qi:Mis a Turing machine, qis a state of M,\nfor every input string w, the computation of Mon\ninputwnever visits state qg,\nand\nEQTM=fhM1;M2i:M1andM2are Turing machines\nandL(M1) =L(M2)g.\n\u000fUse Rice's Theorem to show that Empty is undecidable.\n\u000fUse the \frst part to show that UselessState is undecidable.\n\u000fUse the \frst part to show that EQTMis undecidable.\n5.9Consider the language\nREG TM=fhMi:Mis a Turing machine whose language L(M) is regularg:\nUse Rice's Theorem to prove that REG TMis undecidable.\n5.10 We have seen in Section 5.1.4 that the language\nATM=fhM;wi:Mis a Turing machine that accepts wg\nis undecidable. Consider the language REG TMof Exercise 5.9. The questions\nbelow will lead you through a proof of the claim that the language REG TM\nis undecidable.\nConsider a \fxed Turing machine Mand a \fxed binary string w. We\nconstruct a new Turing machine TMwthat takes as input an arbitrary binary\nstringx:\nTuring machine TMw(x):\nifx= 0n1nfor somen\u00150\nthen terminate in the accept state\nelse runMon the input string w;\nifMterminates in the accept state\nthen terminate in the accept state\nelse ifMterminates in the reject state\nthen terminate in the reject state\nendif\nendif\nendif"
            }
        ]
    },
    {
        "section": "Page 201",
        "content": [
            {
                "type": "text",
                "text": "Exercises 193\nAnswer the following two questions:\n\u000fAssume that Maccepts the string w. What is the language L(TMw) of\nthe new Turing machine TMw?\n\u000fAssume that Mdoes not accept the string w. What is the language\nL(TMw) of the new Turing machine TMw?\nThe goal is to prove that the language REG TMis undecidable. We will\nprove this by contradiction. Thus, we assume that Ris a Turing machine\nthat decides REG TM. Recall what this means:\n\u000fIfMis a Turing machine whose language is regular, then R, when\ngivenhMias input, will terminate in the accept state.\n\u000fIfMis a Turing machine whose language is not regular, then R, when\ngivenhMias input, will terminate in the reject state.\nWe construct a new Turing machine R0which takes as input an arbitrary\nTuring machine Mand an arbitrary binary string w:\nTuring machine R0(hM;wi):\nconstruct the Turing machine TMwdescribed above;\nrunRon the inputhTMwi;\nifRterminates in the accept state\nthen terminate in the accept state\nelse ifRterminates in the reject state\nthen terminate in the reject state\nendif\nendif\nProve that Macceptswif and only if R0(when givenhM;wias input),\nterminates in the accept state.\nNow \fnish the proof by arguing that the language REG TMis undecidable.\n5.11 A Java program Pis called a Hello-World-program , if the following is\ntrue: When given the empty string \u000fas input,Poutputs the string Hello\nWorld and then terminates. (We do not care what Pdoes when the input\nstring is non-empty.)"
            }
        ]
    },
    {
        "section": "Page 202",
        "content": [
            {
                "type": "text",
                "text": "194 Chapter 5. Decidable and Undecidable Languages\nConsider the language\nHW =fhPi:Pis a Hello-World-program g:\nThe questions below will lead you through a proof of the claim that the\nlanguage HW is undecidable.\nConsider a \fxed Java program Pand a \fxed binary string w. We write\na new Java program JPwwhich takes as input an arbitrary binary string x:\nJava program JPw(x):\nrunPon the input w;\nprint Hello World\n\u000fArgue that Pterminates on input wif and only ifhJPwi2HW.\nThe goal is to prove that the language HW is undecidable. We will prove this\nby contradiction. Thus, we assume that His a Java program that decides\nHW. Recall what this means:\n\u000fIfPis a Hello-World-program, then H, when givenhPias input, will\nterminate in the accept state.\n\u000fIfPis not a Hello-World-program, then H, when givenhPias input,\nwill terminate in the reject state.\nWe write a new Java program H0which takes as input the binary encoding\nhP;wiof an arbitrary Java program Pand an arbitrary binary string w:\nJava program H0(hP;wi):\nconstruct the Java program JPwdescribed above;\nrunHon the inputhJPwi;\nifHterminates in the accept state\nthen terminate in the accept state\nelse terminate in the reject state\nendif\nArgue that the following are true:"
            }
        ]
    },
    {
        "section": "Page 203",
        "content": [
            {
                "type": "text",
                "text": "Exercises 195\n\u000fFor any inputhP;wi,H0terminates.\n\u000fIfPterminates on input w, thenH0(when givenhP;wias input),\nterminates in the accept state.\n\u000fIfPdoes not terminate on input w, thenH0(when givenhP;wias\ninput), terminates in the reject state.\nNow \fnish the proof by arguing that the language HW is undecidable.\n5.12 Prove that the language Halt, see Section 5.1.5, is enumerable.\n5.13 We de\fne the following language:\nL=fu:u=h0;M;wifor somehM;wi2ATM,\noru=h1;M;wifor somehM;wi62ATMg.\nProve that neither Lnor its complement Lis enumerable.\nHint: There are two ways to solve this exercise. In the \frst solution, (i)\nyou assume that Lis enumerable, and then prove that ATMis decidable, and\n(ii) you assume that Lis enumerable, and then prove that ATMis decidable.\nIn the second solution, (i) you assume that Lis enumerable, and then prove\nthatATMis enumerable, and (ii) you assume that Lis enumerable, and then\nprove thatATMis enumerable."
            }
        ]
    },
    {
        "section": "Page 204",
        "content": [
            {
                "type": "text",
                "text": "196 Chapter 5. Decidable and Undecidable Languages"
            }
        ]
    },
    {
        "section": "Page 205",
        "content": [
            {
                "type": "text",
                "text": "Chapter 6\nComplexity Theory\nIn the previous chapters, we have considered the problem of what can be\ncomputed by Turing machines (i.e., computers) and what cannot be com-\nputed. We did not, however, take the e\u000eciency of the computations into\naccount. In this chapter, we introduce a classi\fcation of decidable languages\nA, based on the running time of the \\best\" algorithm that decides A. That\nis, given a decidable language A, we are interested in the \\fastest\" algorithm\nthat, for any given string w, decides whether or not w2A.\n6.1 The running time of algorithms\nLetMbe a Turing machine, and let wbe an input string for M. We de\fne\ntherunning time tM(w) ofMon inputwas\ntM(w) := the number of computation steps made by Mon inputw:\nAs usual, we denote by jwj, the number of symbols in the string w. We\ndenote the set of non-negative integers by N0.\nDe\fnition 6.1.1 Let \u0006 be an alphabet, let T:N0!N0be a function, let\nA\u0012\u0006\u0003be a decidable language, and let F: \u0006\u0003!\u0006\u0003be a computable\nfunction.\n\u000fWe say that the Turing machine Mdecides the language Ain timeT,\nif\ntM(w)\u0014T(jwj)\nfor all strings win \u0006\u0003."
            }
        ]
    },
    {
        "section": "Page 206",
        "content": [
            {
                "type": "text",
                "text": "198 Chapter 6. Complexity Theory\n\u000fWe say that the Turing machine Mcomputes the function Fin time\nT, if\ntM(w)\u0014T(jwj)\nfor all strings w2\u0006\u0003.\nIn other words, the \\running time function\" Tis a function of the length\nof the input , which we usually denote by n. For anyn, the value of T(n) is\nan upper bound on the running time of the Turing machine M, onanyinput\nstring of length n.\nTo give an example, consider the Turing machine of Section 4.2.1 that\ndecides, using one tape, the language consisting of all palindromes. The tape\nhead of this Turing machine moves from the left to the right, then back to\nthe left, then to the right again, back to the left, etc. Each time it reaches\nthe leftmost or rightmost symbol, it deletes this symbol. The running time\nof this Turing machine, on any input string of length n, is\nO(1 + 2 + 3 + :::+n) =O(n2):\nOn the other hand, the running time of the Turing machine of Section 4.2.2,\nwhich also decides the palindromes, but using two tapes instead of just one,\nisO(n).\nIn Section 4.4, we mentioned that all computation models listed there are\nequivalent, in the sense that if a language can be decided in one model, it\ncan be decided in any of the other models. We just saw, however, that the\nlanguage consisting of all palindromes allows a faster algorithm on a two-\ntape Turing machine than on one-tape Turing machines. (Even though we\ndid not prove this, it is true that \n( n2) is a lower bound on the running\ntime to decide palindromes on a one-tape Turing machine.) The following\ntheorem can be proved.\nTheorem 6.1.2 LetAbe a language (resp. let Fbe a function) that can be\ndecided (resp. computed) in time Tby an algorithm of type M. Then there is\nan algorithm of type Nthat decides A(resp. computes F) in timeT0, where\nM N T0\nk-tape Turing machine one-tape Turing machine O(T2)\none-tape Turing machine Java program O(T2)\nJava program k-tape Turing machine O(T4)"
            }
        ]
    },
    {
        "section": "Page 207",
        "content": [
            {
                "type": "text",
                "text": "6.2. The complexity class P 199\n6.2 The complexity class P\nDe\fnition 6.2.1 We say that algorithm Mdecides the language A(resp.\ncomputes the function F) inpolynomial time , if there exists an integer k\u00151,\nsuch that the running time of MisO(nk), for any input string of length n.\nIt follows from Theorem 6.1.2 that this notion of \\polynomial time\" does\nnot depend on the model of computation:\nTheorem 6.2.2 Consider the models of computation \\Java program\", \\ k-\ntape Turing machine\", and \\one-tape Turing machine\". If a language can\nbe decided (resp. a function can be computed) in polynomial time in one of\nthese models, then it can be decided (resp. computed) in polynomial time in\nall of these models.\nBecause of this theorem, we can de\fne the following two complexity\nclasses:\nP:=fA: the language Ais decidable in polynomial time g;\nand\nFP:=fF: the function Fis computable in polynomial time g:\n6.2.1 Some examples\nPalindromes\nLetPalbe the language\nPal:=fw2fa;bg\u0003:wis a palindromeg:\nWe have seen that there exists a one-tape Turing machine that decides Pal\ninO(n2) time. Therefore, Pal2P.\nSome functions in FP\nThe following functions are in the class FP:\n\u000fF1:N0!N0de\fned byF1(x) :=x+ 1,\n\u000fF2:N2\n0!N0de\fned byF2(x;y) :=x+y,\n\u000fF3:N2\n0!N0de\fned byF3(x;y) :=xy."
            }
        ]
    },
    {
        "section": "Page 208",
        "content": [
            {
                "type": "text",
                "text": "200 Chapter 6. Complexity Theory\nrb\nbbr\nrb\nG1G2\nFigure 6.1: The graph G1is 2-colorable; rstands for red; bstands for blue.\nThe graph G2is not 2-colorable.\nContext-free languages\nWe have shown in Section 5.1.3 that every context-free language is decid-\nable. The algorithm presented there, however, does not run in polynomial\ntime. Using a technique called dynamic programming (which you will learn\nin COMP 3804), the following result can be shown:\nTheorem 6.2.3 Let\u0006be an alphabet, and let A\u0012\u0006\u0003be a context-free\nlanguage. Then A2P.\nObserve that, obviously, every language in Pis decidable.\nThe 2-coloring problem\nLetGbe a graph with vertex set Vand edge set E. We say that Gis\n2-colorable , if it is possible to give each vertex of Va color such that\n1. for each edge ( u;v)2E, the vertices uandvhave di\u000berent colors, and\n2. only two colors are used to color all vertices.\nSee Figure 6.1 for two examples. We de\fne the following language:\n2Color :=fhGi: the graph Gis 2-colorableg;\nwherehGidenotes the binary string that encodes the graph G."
            }
        ]
    },
    {
        "section": "Page 209",
        "content": [
            {
                "type": "text",
                "text": "6.2. The complexity class P 201\nWe claim that 2 Color2P. In order to show this, we have to construct an\nalgorithm that decides in polynomial time, whether or not any given graph\nis 2-colorable.\nLetGbe an arbitrary graph with vertex set V=f1;2;:::;mg. The edge\nset ofGis given by an adjacency matrix . This matrix, which we denote by\nE, is a two-dimensional array with mrows andmcolumns. For all iandj\nwith 1\u0014i\u0014mand 1\u0014j\u0014m, we have\nE(i;j) =\u001a1 if (i;j) is an edge of G;\n0 otherwise.\nThe length of the input G, i.e., the number of bits needed to specify G, is\nequal tom2=:n. We will present an algorithm that decides, in O(n) time,\nwhether or not the graph Gis 2-colorable.\nThe algorithm uses the colors red and blue. It gives the \frst vertex the\ncolor red. Then, the algorithm considers all vertices that are connected by\nan edge to the \frst vertex, and colors them blue. Now the algorithm is done\nwith the \frst vertex; it marks this \frst vertex.\nNext, the algorithm chooses a vertex ithat already has a color, but that\nhas not been marked. Then it considers all vertices jthat are connected by\nan edge to i. Ifjhas the same color as i, then the input graph Gis not\n2-colorable. Otherwise, if vertex jdoes not have a color yet, the algorithm\ngivesjthe color that is di\u000berent from i's color. After having done this for\nall neighbors jofi, the algorithm is done with vertex i, so it marks i.\nIt may happen that there is no vertex ithat already has a color but that\nhas not been marked. (In other words, each vertex ithat is not marked does\nnot have a color yet.) In this case, the algorithm chooses an arbitrary vertex\nihaving this property, and colors it red. (This vertex iis the \frst vertex in\nits connected component that gets a color.)\nThis procedure is repeated until all vertices of Ghave been marked.\nWe now give a formal description of this algorithm. Vertex ihas been\nmarked , if\n1.ihas a color,\n2. all vertices that are connected by an edge to ihave a color, and\n3. the algorithm has veri\fed that each vertex that is connected by an edge\ntoihas a color di\u000berent from i's color."
            }
        ]
    },
    {
        "section": "Page 210",
        "content": [
            {
                "type": "text",
                "text": "202 Chapter 6. Complexity Theory\nThe algorithm uses two arrays f(1:::m ) anda(1:::m ), and a variable\nM. The value of f(i) is equal to the color (red or blue) of vertex i; ifidoes\nnot have a color yet, then f(i) = 0. The value of a(i) is equal to\na(i) =\u001a1 if vertex ihas been marked ;\n0 otherwise.\nThe value of Mis equal to the number of marked vertices. The algorithm\nis presented in Figure 6.2. You are encouraged to convince yourself of the\ncorrectness of this algorithm. That is, you should convince yourself that this\nalgorithm returns YES if the graph Gis 2-colorable, whereas it returns NO\notherwise.\nWhat is the running time of this algorithm? First we count the number\nof iterations of the outer while-loop. In one iteration, either Mincreases by\none, or a vertex i, for which a(i) = 0, gets the color red. In the latter case,\nthe variable Mis increased during the next iteration of the outer while-loop.\nSince, during the entire outer while-loop, the value of Mis increased from\nzero tom, it follows that there are at most 2 miterations of the outer while-\nloop. (In fact, the number of iterations is equal to mplus the number of\nconnected components of Gminus one.)\nOne iteration of the outer while-loop takes O(m) time. Hence, the total\nrunning time of the algorithm is O(m2), which is O(n). Therefore, we have\nshown that 2 Color2P.\n6.3 The complexity class NP\nBefore we de\fne the class NP, we consider some examples.\nExample 6.3.1 LetGbe a graph with vertex set Vand edge set E, and\nletk\u00151 be an integer. We say that Gisk-colorable , if it is possible to give\neach vertex of Va color such that\n1. for each edge ( u;v)2E, the vertices uandvhave di\u000berent colors, and\n2. at most kdi\u000berent colors are used to color all vertices.\nWe de\fne the following language:\nkColor :=fhGi: the graph Gisk-colorableg:"
            }
        ]
    },
    {
        "section": "Page 211",
        "content": [
            {
                "type": "text",
                "text": "6.3. The complexity class NP 203\nAlgorithm 2Color\nfori:= 1tomdof(i) := 0; a(i) := 0 endfor ;\nf(1) := red; M:= 0;\nwhile M6=m\ndo(\u0003Find the minimum index ifor which vertex ihas not\nbeen marked, but has a color already \u0003)\nbool:=false;i:= 1;\nwhile bool=false andi\u0014m\ndo if a(i) = 0 and f(i)6= 0then bool:=trueelsei:=i+ 1endif ;\nendwhile ;\n(\u0003Ifbool=true, then iis the smallest index such that\na(i) = 0 and f(i)6= 0.\nIfbool=false, then for all i, the following holds: if a(i) = 0, then\nf(i) = 0; because M < m , there is at least one such i.\u0003)\nifbool=true\nthen for j:= 1tom\ndo if E(i; j) = 1\nthen if f(i) =f(j)\nthen return NO and terminate\nelse if f(j) = 0\nthen if f(i) = red\nthen f(j) := blue\nelsef(j) := red\nendif\nendif\nendif\nendif\nendfor ;\na(i) := 1; M:=M+ 1;\nelsei:= 1;\nwhile a(i)6= 0doi:=i+ 1endwhile ;\n(\u0003an unvisited connected component starts at vertex i\u0003)\nf(i) := red\nendif\nendwhile ;\nreturn YES\nFigure 6.2: An algorithm that decides whether or not a graph Gis 2-\ncolorable.\nWe have seen that for k= 2, this problem is in the class P. Fork\u00153, it\nis not known whether there exists an algorithm that decides, in polynomial\ntime, whether or not any given graph is k-colorable. In other words, for"
            }
        ]
    },
    {
        "section": "Page 212",
        "content": [
            {
                "type": "text",
                "text": "204 Chapter 6. Complexity Theory\nk\u00153, it is not known whether or not kColor is in the class P.\nExample 6.3.2 LetGbe a graph with vertex set V=f1;2;:::;mgand\nedge setE. AHamilton cycle is a cycle in Gthat visits each vertex exactly\nonce. Formally, it is a sequence v1;v2;:::;vmof vertices such that\n1.fv1;v2;:::;vmg=V, and\n2.f(v1;v2);(v2;v3);:::; (vm\u00001;vm);(vm;v1)g\u0012E.\nWe de\fne the following language:\nHC:=fhGi: the graph Gcontains a Hamilton cycle g:\nIt is not known whether or not HCis in the class P.\nExample 6.3.3 The sum of subset language is de\fned as follows:\nSOS :=fha1;a2;:::;am;bi:m;a 1;a2;:::;am;b2N0and\n9I\u0012f1;2;:::;mg;P\ni2Iai=bg.\nAlso in this case, no polynomial-time algorithm is known that decides the\nlanguage SOS. That is, it is not known whether or not SOS is in the class\nP.\nExample 6.3.4 An integerx\u00152 is a prime number, if there are no a;b2N\nsuch thata6=x,b6=x, andx=ab. Hence, the language of all non-primes\nthat are greater than or equal to two, is\nNPrim :=fhxi:x\u00152 andxis not a prime number g:\nIt is not obvious at all, whether or not NPrim is in the class P. In fact, it\nwas shown only in 2002 that NPrim is in the class P.\nObservation 6.3.5 The four languages above have the following in com-\nmon: If someone gives us a \\solution\" for any given input, then we can\neasily, i.e., in polynomial time, verify whether or not this \\solution\" is a cor-\nrect solution. Moreover, for any input to each of these four problems, there\nexists a \\solution\" whose length is polynomial in the length of the input."
            }
        ]
    },
    {
        "section": "Page 213",
        "content": [
            {
                "type": "text",
                "text": "6.3. The complexity class NP 205\nLet us again consider the language kColor . LetGbe a graph with vertex\nsetV=f1;2;:::;mgand edge set E, and letkbe a positive integer. We\nwant to decide whether or not Gisk-colorable. A \\solution\" is a coloring of\nthe nodes using at most kdi\u000berent colors. That is, a solution is a sequence\nf1;f2;:::;fm. (Interpret this as: vertex ireceives color fi, 1\u0014i\u0014m). This\nsequence is a correct solution if and only if\n1.fi2f1;2;:::;kg, for alliwith 1\u0014i\u0014m, and\n2. for alliwith 1\u0014i\u0014m, and for all jwith 1\u0014j\u0014m, if (i;j)2E,\nthenfi6=fj.\nIf someone gives us this solution (i.e., the sequence f1;f2;:::;fm), then\nwe can verify in polynomial time whether or not these two conditions are\nsatis\fed. The length of this solution is O(mlogk): for eachi, we need about\nlogkbits to represent fi. Hence, the length of the solution is polynomial in\nthe length of the input, i.e., it is polynomial in the number of bits needed to\nrepresent the graph Gand the number k.\nFor the Hamilton cycle problem, a solution consists of a sequence v1,\nv2;:::;vmof vertices. This sequence is a correct solution if and only if\n1.fv1;v2;:::;vmg=f1;2;:::;mgand\n2.f(v1;v2);(v2;v3);:::; (vm\u00001;vm);(vm;v1)g\u0012E.\nThese two conditions can be veri\fed in polynomial time. Moreover, the\nlength of the solution is polynomial in the length of the input graph.\nConsider the sum of subset problem. A solution is a sequence c1;c2;:::;cm.\nIt is a correct solution if and only if\n1.ci2f0;1g, for alliwith 1\u0014i\u0014m, and\n2.Pm\ni=1ciai=b.\nHence, the set I\u0012f1;2;:::;mgin the de\fnition of SOS is the set of indices\nifor whichci= 1. Again, these two conditions can be veri\fed in polynomial\ntime, and the length of the solution is polynomial in the length of the input.\nFinally, let us consider the language NPrim . Letx\u00152 be an integer.\nThe integers aandbform a \\solution\" for xif and only if"
            }
        ]
    },
    {
        "section": "Page 214",
        "content": [
            {
                "type": "text",
                "text": "206 Chapter 6. Complexity Theory\n1. 2\u0014a<x ,\n2. 2\u0014b<x , and\n3.x=ab.\nClearly, these three conditions can be veri\fed in polynomial time. Moreover,\nthe length of this solution, i.e., the total number of bits in the binary rep-\nresentations of aandb, is polynomial in the number of bits in the binary\nrepresentation of x.\nLanguages having the property that the correctness of a proposed \\solu-\ntion\" can be veri\fed in polynomial time, form the class NP:\nDe\fnition 6.3.6 A language Abelongs to the class NP, if there exist a\npolynomial pand a language B2P, such that for every string w,\nw2A()9s:jsj\u0014p(jwj) andhw;si2B:\nIn words, a language Ais in the class NP, if for every string w,w2Aif\nand only if the following two conditions are satis\fed:\n1. There is a \\solution\" s, whose lengthjsjis polynomial in the length of\nw(i.e.,jsj\u0014p(jwj), wherepis a polynomial).\n2. In polynomial time, we can verify whether or not sis a correct \\solu-\ntion\" forw(i.e.,hw;si2BandB2P).\nHence, the language Bcan be regarded to be the \\veri\fcation language\":\nB=fhw;si:sis a correct \\solution\" for wg:\nWe have given already informal proofs of the fact that the languages\nkColor ,HC,SOS, and NPrim are all contained in the class NP. Below, we\nformally prove that NPrim2NP. To prove this claim, we have to specify\nthe polynomial pand the language B2P. First, we observe that\nNPrim =fhxi: there exist aandbinNsuch that\n2\u0014a<x; 2\u0014b<x andx=abg.(6.1)\nWe de\fne the polynomial pbyp(n) :=n+ 2, and the language Bas\nB:=fhx;a;bi:x\u00152;2\u0014a<x; 2\u0014b<x andx=abg:"
            }
        ]
    },
    {
        "section": "Page 215",
        "content": [
            {
                "type": "text",
                "text": "6.3. The complexity class NP 207\nIt is obvious that B2P: For any three positive integers x,a, andb, we\ncan verify in polynomial time whether or not hx;a;bi2B. In order to do\nthis, we only have to verify whether or not x\u00152, 2\u0014a < x , 2\u0014b < x ,\nandx=ab. If all these four conditions are satis\fed, then hx;a;bi2B. If at\nleast one of them is not satis\fed, then hx;a;bi62B.\nIt remains to show that for all x2N:\nhxi2NPrim()9a;b:jha;bij\u0014jhxij+ 2 andhx;a;bi2B: (6.2)\n(Remember thatjhxijdenotes the number of bits in the binary representation\nofx;jha;bijdenotes the total number of bits of aandb, i.e.,jha;bij=\njhaij+jhbij.)\nLetx2NPrim . It follows from (6.1) that there exist aandbinN, such\nthat 2\u0014a < x , 2\u0014b < x , andx=ab. Sincex=ab\u00152\u00012 = 4\u00152, it\nfollows thathx;a;bi2B. Hence, it remains to show that\njha;bij\u0014jhxij+ 2:\nThe binary representation of xcontainsblogxc+1 bits, i.e.,jhxij=blogxc+1.\nWe have\njha;bij=jhaij+jhbij\n= (blogac+ 1) + (blogbc+ 1)\n\u0014loga+ logb+ 2\n= logab+ 2\n= logx+ 2\n\u0014 b logxc+ 3\n=jhxij+ 2:\nThis proves one direction of (6.2).\nTo prove the other direction, we assume that there are positive integers\naandb, such thatjha;bij\u0014jhxij+ 2 andhx;a;bi2B. Then it follows\nimmediately from (6.1) and the de\fnition of the language B, thatx2NPrim .\nHence, we have proved the other direction of (6.2). This completes the proof\nof the claim that\nNPrim2NP:"
            }
        ]
    },
    {
        "section": "Page 216",
        "content": [
            {
                "type": "text",
                "text": "208 Chapter 6. Complexity Theory\n6.3.1 P is contained in NP\nIntuitively, it is clear that P\u0012NP, because a language is\n\u000finP, if for every string w, it is possible to compute the \\solution\" sin\npolynomial time,\n\u000finNP, if for every string wand for any given \\solution\" s, it is possible\ntoverify in polynomial time whether or not sis a correct solution for\nw(hence, we do not need to compute the solution sourselves, we only\nhave to verify it).\nWe give a formal proof of this:\nTheorem 6.3.7 P \u0012NP.\nProof. LetA2P. We will prove that A2NP. De\fne the polynomial p\nbyp(n) := 0 for all n2N0, and de\fne\nB:=fhw;\u000fi:w2Ag:\nSinceA2P, the language Bis also contained in P. It is easy to see that\nw2A()9s:jsj\u0014p(jwj) = 0 andhw;si2B:\nThis completes the proof.\n6.3.2 Deciding NP-languages in exponential time\nLet us look again at the de\fnition of the class NP. LetAbe a language in\nthis class. Then there exist a polynomial pand a language B2P, such that\nfor all strings w,\nw2A()9s:jsj\u0014p(jwj) andhw;si2B: (6.3)\nHow do we decide whether or not any given string wbelongs to the language\nA? If we can \fnd a string sthat satis\fes the right-hand side in (6.3), then\nwe know that w2A. On the other hand, if there is no such string s, then\nw62A. How much time do we need to decide whether or not such a string s\nexists?"
            }
        ]
    },
    {
        "section": "Page 217",
        "content": [
            {
                "type": "text",
                "text": "6.3. The complexity class NP 209\nAlgorithm NonPrime\n(\u0003decides whether or not hxi2NPrim\u0003)\nifx= 0 orx= 1 orx= 2\nthen return NO and terminate\nelsea:= 2;\nwhilea<x\ndo ifxmoda= 0\nthen return YES and terminate\nelsea:=a+ 1\nendif\nendwhile ;\nreturn NO\nendif\nFigure 6.3: An algorithm that decides whether or not a number xis contained\nin the language NPrim .\nFor example, let Abe the language\nNPrim =fhxi:x\u00152 andxis not a prime number g;\nand letx2N. The algorithm in Figure 6.3 decides whether or not hxi2\nNPrim .\nIt is clear that this algorithm is correct. Let nbe the length of the binary\nrepresentation of x, i.e.,n=blogxc+ 1. Ifx>2 andxis a prime number,\nthen the while-loop makes x\u00002 iterations. Therefore, since n\u00001 =blogxc\u0014\nlogx, the running time of this algorithm is at least\nx\u00002\u00152n\u00001\u00002;\ni.e., it is at least exponential in the length of the input.\nWe now prove that every language in NPcan be decided in exponential\ntime. LetAbe an arbitrary language in NP. Letpbe the polynomial, and\nletB2Pbe the language such that for all strings w,\nw2A()9s:jsj\u0014p(jwj) andhw;si2B: (6.4)\nThe following algorithm decides, for any given string w, whether or not\nw2A. It does so by looking at allpossible strings sfor whichjsj\u0014p(jwj):"
            }
        ]
    },
    {
        "section": "Page 218",
        "content": [
            {
                "type": "text",
                "text": "210 Chapter 6. Complexity Theory\nfor allswithjsj\u0014p(jwj)\ndo ifhw;si2B\nthen return YES and terminate\nendif\nendfor ;\nreturn NO\nThe correctness of the algorithm follows from (6.4). What is the running\ntime? We assume that wandsare represented as binary strings. Let nbe\nthe length of the input, i.e., n=jwj.\nHow many binary strings sare there whose length is at most p(jwj)? Any\nsuchscan be described by a sequence of length p(jwj) =p(n), consisting of\nthe symbols \\0\", \\1\", and the blank symbol. Hence, there are at most 3p(n)\nmany binary strings swithjsj\u0014p(n). Therefore, the for-loop makes at most\n3p(n)iterations.\nSinceB2P, there is an algorithm and a polynomial q, such that this\nalgorithm, when given any input string z, decides in q(jzj) time, whether or\nnotz2B. This input zhas the formhw;si, and we have\njzj=jwj+jsj\u0014jwj+p(jwj) =n+p(n):\nIt follows that the total running time of our algorithm that decides whether\nor notw2A, is bounded from above by\n3p(n)\u0001q(n+p(n))\u001422p(n)\u0001q(n+p(n))\n\u001422p(n)\u00012q(n+p(n))\n= 2p0(n);\nwherep0is the polynomial that is de\fned by p0(n) := 2p(n) +q(n+p(n)).\nIf we de\fne the class EXP as\nEXP :=fA: there exists a polynomial p, such that Acan be\ndecided in time 2p(n)g,\nthen we have proved the following theorem.\nTheorem 6.3.8 NP \u0012EXP ."
            }
        ]
    },
    {
        "section": "Page 219",
        "content": [
            {
                "type": "text",
                "text": "6.4. Non-deterministic algorithms 211\n6.3.3 Summary\n\u000fP\u0012NP. It is not known whether Pis a proper subclass of NP, or\nwhether P=NP. This is one of the most important open problems in\ncomputer science. If you can solve this problem, then you will get one\nmillion dollars; not from us, but from the Clay Mathematics Institute,\nsee\nhttp://www.claymath.org/prizeproblems/index.htm\nMost people believe that Pis a proper subclass of NP.\n\u000fNP\u0012EXP , i.e., each language in NPcan be decided in exponential\ntime. It is not known whether NP is a proper subclass of EXP , or\nwhether NP=EXP .\n\u000fIt follows from P\u0012NP andNP\u0012EXP , that P\u0012EXP . It can\nbe shown that Pis a proper subset of EXP , i.e., there exist languages\nthat can be decided in exponential time, but that cannot be decided in\npolynomial time.\n\u000fPis the class of those languages that can be decided e\u000eciently , i.e., in\npolynomial time. Sets that are not in P, are not e\u000eciently decidable.\n6.4 Non-deterministic algorithms\nThe abbreviation NPstands for Non-deterministic Polynomial time. The al-\ngorithms that we have considered so far are deterministic , which means that\nat any time during the computation, the next computation step is uniquely\ndetermined. In a non-deterministic algorithm, there are one or more possi-\nbilities for being the next computation step, and the algorithm chooses one\nof them.\nTo give an example, we consider the language SOS, see Example 6.3.3.\nLetm,a1,a2,:::;am, andbbe elements of N0. Then\nha1;a2;:::;am;bi2SOS() there exist c1;c2;:::;cm2f0;1g,\nsuch thatPm\ni=1ciai=b.\nThe following non-deterministic algorithm decides the language SOS:"
            }
        ]
    },
    {
        "section": "Page 220",
        "content": [
            {
                "type": "text",
                "text": "212 Chapter 6. Complexity Theory\nAlgorithm SOS (m;a 1;a2;:::;am;b):\ns:= 0;\nfori:= 1tom\ndos:=sjs:=s+ai\nendfor ;\nifs=b\nthen return YES\nelse return NO\nendif\nThe line\ns:=sjs:=s+ai\nmeans that either the instruction \\ s:=s\" or the instruction \\ s:=s+ai\" is\nexecuted.\nLet us assume that ha1;a2;:::;am;bi2SOS. Then there are c1;c2;:::;cm2\nf0;1gsuch thatPm\ni=1ciai=b. Assume our algorithm does the following, for\neachiwith 1\u0014i\u0014m: In thei-th iteration,\n\u000fifci= 0, then it executes the instruction \\ s:=s\",\n\u000fifci= 1, then it executes the instruction \\ s:=s+ai\".\nThen after the for-loop, we have s=b, and the algorithm returns YES;\nhence, the algorithm has correctly found out that ha1;a2;:::;am;bi2SOS.\nIn other words, in this case, there exists at least one accepting computation .\nOn the other hand, if ha1;a2;:::;am;bi62SOS, then the algorithm always\nreturns NO, no matter which of the two instructions is executed in each\niteration of the for-loop. In this case, there is no accepting computation .\nDe\fnition 6.4.1 LetMbe a non-deterministic algorithm. We say that M\naccepts a stringw, if there exists at least one computation that, on input w,\nreturns YES.\nDe\fnition 6.4.2 We say that a non-deterministic algorithm Mdecides a\nlanguageAin timeT, if for every string w, the following holds: w2Aif\nand only if there exists at least one computation that, on input w, returns\nYES and that takes at most T(jwj) time."
            }
        ]
    },
    {
        "section": "Page 221",
        "content": [
            {
                "type": "text",
                "text": "6.5. NP-complete languages 213\nThe non-deterministic algorithm that we have seen above decides the\nlanguage SOS in linear time: Let ha1;a2;:::;am;bi2SOS, and letnbe the\nlength of this input. Then\nn=jha1ij+jha2ij+:::+jhamij+jhbij\u0015m:\nFor this input, there is a computation that returns YES and that takes\nO(m) =O(n) time.\nAs in Section 6.2, we de\fne the notion of \\polynomial time\" for non-\ndeterministic algorithms. The following theorem relates this notion to the\nclassNPthat we de\fned in De\fnition 6.3.6.\nTheorem 6.4.3 A language Ais in the class NP if and only if there exists\na non-deterministic Turing machine (or Java program) that decides Ain\npolynomial time.\n6.5 NP-complete languages\nLanguages in the class Pare considered easy, i.e., they can be decided in\npolynomial time. People believe (but cannot prove) that Pis a proper sub-\nclass of NP. If this is true, then there are languages in NPthat are hard,\ni.e., cannot be decided in polynomial time.\nIntuition tells us that if P6=NP, then the hardest languages in NPare\nnot contained in P. These languages are called NP-complete . In this section,\nwe will give a formal de\fnition of this concept.\nIf we want to talk about the \\hardest\" languages in NP, then we have to\nbe able to compare two languages according to their \\di\u000eculty\". The idea is\nas follows: We say that a language Bis \\at least as hard\" as a language A,\nif the following holds: If Bcan be decided in polynomial time, then Acan\nalso be decided in polynomial time.\nDe\fnition 6.5.1 LetA\u0012f0;1g\u0003andB\u0012f0;1g\u0003be languages. We say\nthatA\u0014PB, if there exists a function\nf:f0;1g\u0003!f0;1g\u0003\nsuch that\n1.f2FPand"
            }
        ]
    },
    {
        "section": "Page 222",
        "content": [
            {
                "type": "text",
                "text": "214 Chapter 6. Complexity Theory\n2. for all strings winf0;1g\u0003,\nw2A()f(w)2B:\nIfA\u0014PB, then we also say that \\ Bis at least as hard as A\", or \\Ais\npolynomial-time reducible to B\".\nWe \frst show that this formal de\fnition is in accordance with the intuitive\nde\fnition given above.\nTheorem 6.5.2 LetAandBbe languages such that B2PandA\u0014PB.\nThenA2P.\nProof. Letf:f0;1g\u0003!f0;1g\u0003be the function in FPfor which\nw2A()f(w)2B: (6.5)\nThe following algorithm decides whether or not any given binary string wis\ninA:\nu:=f(w);\nifu2B\nthen return YES\nelse return NO\nendif\nThe correctness of this algorithm follows immediately from (6.5). So it\nremains to show that the running time is polynomial in the length of the\ninput string w.\nSincef2FP, there exists a polynomial psuch that the function fcan\nbe computed in time p. Similarly, since B2P, there exists a polynomial q,\nsuch that the language Bcan be decided in time q.\nLetnbe the length of the input string w, i.e.,n=jwj. Then the length\nof the string uis less than or equal to p(jwj) =p(n). (Why?) Therefore, the\nrunning time of our algorithm is bounded from above by\np(jwj) +q(juj)\u0014p(n) +q(p(n)):\nSince the function p0, de\fned by p0(n) :=p(n)+q(p(n)), is a polynomial, this\nproves that A2P.\nThe following theorem states that the relation \u0014Pis re\rexive and tran-\nsitive. We leave the proof as an exercise."
            }
        ]
    },
    {
        "section": "Page 223",
        "content": [
            {
                "type": "text",
                "text": "6.5. NP-complete languages 215\nTheorem 6.5.3 LetA,B, andCbe languages. Then\n1.A\u0014PA, and\n2. ifA\u0014PBandB\u0014PC, thenA\u0014PC.\nWe next show that the languages in Pare the easiest languages in NP:\nTheorem 6.5.4 LetAbe a language in P, and letBbe an arbitrary lan-\nguage such that B6=;andB6=f0;1g\u0003. ThenA\u0014PB.\nProof. We choose two strings uandvinf0;1g\u0003, such thatu2Bandv62B.\n(Observe that this is possible.) De\fne the function f:f0;1g\u0003!f0;1g\u0003by\nf(w) :=\u001auifw2A;\nvifw62A:\nThen it is clear that for any binary string w,\nw2A()f(w)2B:\nSinceA2P, the function fcan be computed in polynomial time, i.e.,\nf2FP.\n6.5.1 Two examples of reductions\nSum of subsets and knapsacks\nWe start with a simple reduction. Consider the two languages\nSOS :=fha1;:::;am;bi:m;a 1;:::;am;b2N0and there exist\nc1;:::;cm2f0;1g, such thatPm\ni=1ciai=bg\nand\nKS :=fhw1;:::;wm;k1;:::;km;W;Ki:\nm;w 1;:::;wm;k1;:::;km;W;K2N0\nand there exist c1;:::;cm2f0;1g,\nsuch thatPm\ni=1ciwi\u0014WandPm\ni=1ciki\u0015Kg.\nThe notation KSstands for knapsack : We have mpieces of food. The\ni-th piece has weight wiand contains kicalories. We want to decide whether\nor not we can \fll our knapsack with a subset of the pieces of food such that\nthe total weight is at most W, and the total amount of calories is at least K."
            }
        ]
    },
    {
        "section": "Page 224",
        "content": [
            {
                "type": "text",
                "text": "216 Chapter 6. Complexity Theory\nTheorem 6.5.5 SOS\u0014PKS.\nProof. Let us \frst see what we have to show. According to De\fnition 6.5.1,\nwe need a function f2FP, that maps input strings for SOS to input strings\nforKS, in such a way that\nha1;:::;am;bi2SOS()f(ha1;:::;am;bi)2KS:\nIn order for f(ha1;:::;am;bi) to be an input string for KS, this function\nvalue has to be of the form\nf(ha1;:::;am;bi) =hw1;:::;wm;k1;:::;km;W;Ki:\nWe de\fne\nf(ha1;:::;am;bi) :=ha1;:::;am;a1;:::;am;b;bi:\nIt is clear that f2FP. We have\nha1;:::;am;bi2SOS\n() there exist c1;:::;cm2f0;1gsuch thatPm\ni=1ciai=b\n() there exist c1;:::;cm2f0;1gsuch thatPm\ni=1ciai\u0014bandPm\ni=1ciai\u0015b\n() ha1;:::;am;a1;:::;am;b;bi2KS\n()f(ha1;:::;am;bi)2KS:\nCliques and Boolean formulas\nWe will de\fne two languages A= 3SAT andB=Clique that have, at\n\frst sight, nothing to do with each other. Then we show that, nevertheless,\nA\u0014PB.\nLetGbe a graph with vertex set Vand edge set E. A subsetV0ofVis\ncalled a clique , if each pair of distinct vertices in V0is connected by an edge\ninE. We de\fne the following language:\nClique :=fhG;ki:k2NandGhas a clique with kverticesg:\nWe encourage you to prove the following claim:"
            }
        ]
    },
    {
        "section": "Page 225",
        "content": [
            {
                "type": "text",
                "text": "6.5. NP-complete languages 217\nTheorem 6.5.6 Clique2NP.\nNext we consider Boolean formulas ', with variables x1;x2;:::;xm, hav-\ning the form\n'=C1^C2^:::^Ck; (6.6)\nwhere each Ci, 1\u0014i\u0014k, is of the form\nCi=`i\n1_`i\n2_`i\n3:\nEach`i\nais either a variable or the negation of a variable. An example of such\na formula is\n'= (x1_:x1_:x2)^(x3_x2_x4)^(:x1_:x3_:x4):\nA formula'of the form (6.6) is said to be satis\fable , if there exists a truth-\nvalue inf0;1gfor each of the variables x1;x2;:::;xm, such that the entire\nformula'is true. Our example formula is satis\fable: If we take x1= 0 and\nx2= 1, and give x3andx4an arbitrary value, then\n'= (0_1_0)^(x3_1_x4)^(1_:x3_:x4) = 1:\nWe de\fne the following language:\n3SAT :=fh'i:'is of the form (6.6) and is satis\fable g:\nAgain, we encourage you to prove the following claim:\nTheorem 6.5.7 3SAT2NP.\nObserve that the elements of Clique (which are pairs consisting of a graph\nand a positive integer) are completely di\u000berent from the elements of 3 SAT\n(which are Boolean formulas). We will show that 3 SAT\u0014PClique . Recall\nthat this means the following: If the language Clique can be decided in\npolynomial time, then the language 3 SAT can also be decided in polynomial\ntime. In other words, any polynomial-time algorithm that decides Clique can\nbe converted to a polynomial-time algorithm that decides 3 SAT.\nTheorem 6.5.8 3SAT\u0014PClique."
            }
        ]
    },
    {
        "section": "Page 226",
        "content": [
            {
                "type": "text",
                "text": "218 Chapter 6. Complexity Theory\nProof. We have to show that there exists a function f2FP, that maps\ninput strings for 3 SAT to input strings for Clique , such that for each Boolean\nformula'that is of the form (6.6),\nh'i23SAT()f(h'i)2Clique:\nThe function fmaps the binary string encoding an arbitrary Boolean formula\n'to a binary string encoding a pair ( G;k), whereGis a graph and kis a\npositive integer. We have to de\fne this function fin such a way that\n'is satis\fable()Ghas a clique with kvertices:\nLet\n'=C1^C2^:::^Ck\nbe an arbitrary Boolean formula in the variables x1;x2;:::;xm, where each\nCi, 1\u0014i\u0014k, is of the form\nCi=`i\n1_`i\n2_`i\n3:\nRemember that each `i\nais either a variable or the negation of a variable.\nThe formula 'is mapped to the pair ( G;k), where the vertex set Vand\nthe edge set Eof the graph Gare de\fned as follows:\n\u000fV=fv1\n1;v1\n2;v1\n3;:::;vk\n1;vk\n2;vk\n3g. The idea is that each vertex vi\nacorre-\nsponds to one term `i\na.\n\u000fThe pair (vi\na;vj\nb) of vertices form an edge in Eif and only if\n{i6=jand\n{`i\nais not the negation of `j\nb.\nTo give an example, let 'be the Boolean formula\n'= (x1_:x2_:x3)^(:x1_x2_x3)^(x1_x2_x3); (6.7)\ni.e.,k= 3,C1=x1_:x2_:x3,C2=:x1_x2_x3, andC3=x1_x2_x3.\nThe graph Gthat corresponds to this formula is given in Figure 6.4.\nIt is not di\u000ecult to see that the function fcan be computed in polynomial\ntime. So it remains to prove that\n'is satis\fable()Ghas a clique with kvertices: (6.8)"
            }
        ]
    },
    {
        "section": "Page 227",
        "content": [
            {
                "type": "text",
                "text": "6.5. NP-complete languages 219\n¬x2 ¬x3 x1\n¬x1\nx2\nx3x1\nx2\nx3\nFigure 6.4: The formula 'in (6.7) is mapped to this graph. The vertices on\nthe top represent C1; the vertices on the left represent C2; the vertices on\nthe right represent C3.\nTo prove this, we \frst assume that the formula\n'=C1^C2^:::^Ck\nis satis\fable. Then there exists a truth-value in f0;1gfor each of the variables\nx1;x2;:::;xm, such that the entire formula 'is true. Hence, for each iwith\n1\u0014i\u0014k, there is at least one term `i\nain\nCi=`i\n1_`i\n2_`i\n3\nthat is true (i.e., has value 1).\nLetV0be the set of vertices obtained by choosing for each i, 1\u0014i\u0014k,\nexactly one vertex vi\nasuch that`i\nahas value 1.\nIt is clear that V0contains exactly kvertices. We claim that this set is\na clique in G. To prove this claim, let vi\naandvj\nbbe two distinct vertices in\nV0. It follows from the de\fnition of V0thati6=jand`i\na=`j\nb= 1. Hence,\n`i\nais not the negation of `j\nb. But this means that the vertices vi\naandvj\nbare\nconnected by an edge in G.\nThis proves one direction of (6.8). To prove the other direction, we assume\nthat the graph Gcontains a clique V0withkvertices."
            }
        ]
    },
    {
        "section": "Page 228",
        "content": [
            {
                "type": "text",
                "text": "220 Chapter 6. Complexity Theory\nThe vertices of Gconsist ofkgroups, where each group contains exactly\nthree vertices. Since vertices within the same group are not connected by\nedges, the clique V0contains exactly one vertex from each group. Hence, for\neachiwith 1\u0014i\u0014k, there is exactly one a, such that vi\na2V0. Consider\nthe corresponding term `i\na. We know that this term is either a variable or\nthe negation of a variable, i.e., `i\nais either of the form xjor of the form:xj.\nIf`i\na=xj, then we give xjthe truth-value 1. Otherwise, we have `i\na=:xj,\nin which case we give xjthe truth-value 0. Since V0is a clique, each variable\ngets at most one truth-value. If a variable has no truth-value yet, then we\ngive it an arbitrary truth-value.\nIf we substitute these truth-values into ', then the entire formula has\nvalue 1. Hence, 'is satis\fable.\nIn order to get a better understanding of this proof, you should verify the\nproof for the formula 'in (6.7) and the graph Gin Figure 6.4.\n6.5.2 De\fnition of NP-completeness\nReductions, as de\fned in De\fnition 6.5.1, allow us to compare two language\naccording to their di\u000eculty. A language BinNP is called NP-complete,\nifBbelongs to the most di\u000ecult languages in NP; in other words, Bis at\nleast as hard as anyother language in NP.\nDe\fnition 6.5.9 LetB\u0012f0;1g\u0003be a language. We say that BisNP-\ncomplete , if\n1.B2NPand\n2.A\u0014PB, for every languageAinNP.\nTheorem 6.5.10 LetBbe an NP-complete language. Then\nB2P()P=NP:\nProof. Intuitively, this theorem should be true: If the language Bis inP,\nthenBis an easy language. On the other hand, since BisNP-complete,\nit belongs to the most di\u000ecult languages in NP. Hence, the most di\u000ecult\nlanguage in NP is easy. But then all languages in NP must be easy, i.e.,\nP=NP."
            }
        ]
    },
    {
        "section": "Page 229",
        "content": [
            {
                "type": "text",
                "text": "6.5. NP-complete languages 221\nWe give a formal proof. Let us \frst assume that B2P. We already\nknow that P\u0012NP. Hence, it remains to show that NP\u0012P. LetAbe an\narbitrary language in NP. SinceBisNP-complete, we have A\u0014PB. Then,\nby Theorem 6.5.2, we have A2P.\nTo prove the converse, assume that P=NP. SinceB2NP, it follows\nimmediately that B2P.\nTheorem 6.5.11 LetBandCbe languages, such that C2NP andB\u0014P\nC. IfBisNP-complete, then Cis also NP-complete.\nProof. First, we give an intuitive explanation of the claim: By assumption,\nBbelongs to the most di\u000ecult languages in NP, andCis at least as hard as\nB. SinceC2NP, it follows that Cbelongs to the most di\u000ecult languages\ninNP. Hence,CisNP-complete.\nTo give a formal proof, we have to show that A\u0014PC, for all languages A\ninNP. LetAbe an arbitrary language in NP. SinceBisNP-complete, we\nhaveA\u0014PB. SinceB\u0014PC, it follows from Theorem 6.5.3, that A\u0014PC.\nTherefore,CisNP-complete.\nTheorem 6.5.11 can be used to prove the NP-completeness of languages:\nLetCbe a language, and assume that we want to prove that CisNP-\ncomplete. We can do this in the following way:\n1. We \frst prove that C2NP.\n2. Then we \fnd a language Bthat looks \\similar\" to C, and for which\nwe already know that it is NP-complete.\n3. Finally, we prove that B\u0014PC.\n4. Then, Theorem 6.5.11 tells us that CisNP-complete.\nOf course, this leads to the question \\How do we know that the language\nBisNP-complete?\" In order to apply Theorem 6.5.11, we need a \\\frst\" NP-\ncomplete language; the NP-completeness of this language must be proven\nusing De\fnition 6.5.9.\nObserve that it is not clear at all that there exist NP-complete languages!\nFor example, consider the language 3 SAT. If we want to use De\fnition 6.5.9\nto show that this language is NP-complete, then we have to show that"
            }
        ]
    },
    {
        "section": "Page 230",
        "content": [
            {
                "type": "text",
                "text": "222 Chapter 6. Complexity Theory\n\u000f3SAT2NP. We know from Theorem 6.5.7 that this is true.\n\u000fA\u0014P3SAT, for every languageA2NP. Hence, we have to show this\nfor languages Asuch askColor ,HC,SOS,NPrim ,KS,Clique , and\nforin\fnitely many other languages.\nIn 1971, Cook has exactly done this: He showed that the language 3 SAT\nisNP-complete. Since his proof is rather technical, we will prove the NP-\ncompleteness of another language.\n6.5.3 An NP-complete domino game\nWe are given a \fnite collection of tile types . For each such type, there are\narbitrarily many tiles of this type. A tileis a square that is partitioned into\nfour triangles. Each of these triangles contains a symbol that belongs to a\n\fnite alphabet \u0006. Hence, a tile looks as follows:\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@ab\nc\nd\nWe are also given a square frame , consisting of cells. Each cell has the same\nsize as a tile, and contains a symbol of \u0006.\nThe problem is to decide whether or not this domino game has a solution.\nThat is, can we completely \fll the frame with tiles such that\n\u000ffor any two neighboring tiles sands0, the two triangles of sands0that\ntouch each other contain the same symbol, and\n\u000feach triangle that touches the frame contains the same symbol as the\ncell of the frame that is touched by this triangle.\nThere is one \fnal restriction: The orientation of the tiles is \fxed, they cannot\nbe rotated.\nLet us give a formal de\fnition of this problem. We assume that the sym-\nbols belong to the \fnite alphabet \u0006 = f0;1gm, i.e., each symbol is encoded\nas a bit-string of length m. Then, a tile type can be encoded as a tuple of\nfour bit-strings, i.e., as an element of \u00064. A frame consisting of trows andt\ncolumns can be encoded as a string in \u00064t."
            }
        ]
    },
    {
        "section": "Page 231",
        "content": [
            {
                "type": "text",
                "text": "6.5. NP-complete languages 223\nWe denote the language of all solvable domino games by Domino :\nDomino :=fhm;k;t;R;T 1;:::;Tki:\nm\u00151;k\u00151;t\u00151;R2\u00064t;Ti2\u00064;1\u0014i\u0014k;\nframeRcan be \flled using tiles of types\nT1;:::;Tk:g\nWe will prove the following theorem.\nTheorem 6.5.12 The language Domino is NP-complete.\nProof. It is clear that Domino2NP: A solution consists of a t\u0002tmatrix,\nin which the ( i;j)-entry indicates the type of the tile that occupies position\n(i;j) in the frame. The number of bits needed to specify such a solution is\npolynomial in the length of the input. Moreover, we can verify in polynomial\ntime whether or not any given \\solution\" is correct.\nIt remains to show that\nA\u0014PDomino;for every language AinNP:\nLetAbe an arbitrary language in NP. Then there exist a polynomial pand\na non-deterministic Turing machine M, that decides the language Ain time\np. We may assume that this Turing machine has only one tape.\nOn inputw=a1a2:::an, the Turing machine Mstarts in the start state\nz0, with its tape head on the cell containing the symbol a1. We may assume\nthat during the entire computation, the tape head never moves to the left of\nthis initial cell. Hence, the entire computation \\takes place\" in and to the\nright of the initial cell. We know that\nw2A() on inputw, there exists an accepting computation\nthat makes at most p(n) computation steps.\nAt the end of such an accepting computation, the tape only contains the\nsymbol 1, which we may assume to be in the initial cell, and Mis in the \fnal\nstatez1. In this case, we may assume that the accepting computation makes\nexactlyp(n) computation steps. (If this is not the case, then we extend the\ncomputation using the instruction z11!z11N.)\nWe need one more technical detail: We may assume that za!z0bRand\nza0!z00b0Lare not both instructions of M. Hence, the state of the Turing\nmachine uniquely determines the direction in which the tape head moves."
            }
        ]
    },
    {
        "section": "Page 232",
        "content": [
            {
                "type": "text",
                "text": "224 Chapter 6. Complexity Theory\nWe have to de\fne a domino game, that depends on the input string w\nand the Turing machine M, such that\nw2A() this domino game is solvable :\nThe idea is to encode an accepting computation of the Turing machine Mas\na solution of the domino game. In order to do this, we use a frame in which\neach row corresponds to one computation step. This frame consists of p(n)\nrows. Since an accepting computation makes exactly p(n) computation steps,\nand since the tape head never moves to the left of the initial cell, this tape\nhead can visit only p(n) cells. Therefore, our frame will have p(n) columns.\nThe domino game will use the following tile types:\n1. For each symbol ain the alphabet of the Turing machine M:\n\u0000\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@\n@@#a\n#\na\nIntuition: Before and after the computation step, the tape head is not\non this cell.\n2. For each instruction za!z0bRof the Turing machine M:\n\u0000\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@\n@@#(z;a)\nz0\nb\nIntuition: Before the computation step, the tape head is on this cell;\nthe tape head makes one step to the right.\n3. For each instruction za!z0bLof the Turing machine M:\n\u0000\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@\n@@z0(z;a)\n#\nb"
            }
        ]
    },
    {
        "section": "Page 233",
        "content": [
            {
                "type": "text",
                "text": "6.5. NP-complete languages 225\nIntuition: Before the computation step, the tape head is on this cell;\nthe tape head makes one step to the left.\n4. For each instruction za!z0bNof the Turing machine M:\n\u0000\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@\n@@#(z;a)\n#\n(z0;b)\nIntuition: Before and after the computation step, the tape head is on\nthis cell.\n5. For each state zand for each symbol ain the alphabet of the Turing\nmachineM, there are two tile types:\n\u0000\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@\n@@za\n#\n(z;a)\u0000\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@\n@@#a\nz\n(z;a)\nIntuition: The leftmost tile indicates that the tape head enters this cell\nfrom the left; the righmost tile indicates that the tape head enters this\ncell from the right.\nThis speci\fes all tile types. The p(n)\u0002p(n) frame is given in Figure 6.5.\nThe top row corresponds to the start of the computation, whereas the bottom\nrow corresponds to the end of the computation. The left and right columns\ncorrespond to the part of the tape in which the tape head can move.\nThe encodings of these tile types and the frame can be computed in\npolynomial time.\nIt can be shown that, for any input string w, any accepting computation\nof lengthp(n) of the Turing machine Mcan be encoded as a solution of\nthis domino game. Conversely, any solution of this domino game can be\n\\translated\" to an accepting computation of length p(n) ofM, on input\nstringw. Hence, the following holds.\nw2A() there exists an accepting computation that makes\np(n) computation steps\n() the domino game is solvable."
            }
        ]
    },
    {
        "section": "Page 234",
        "content": [
            {
                "type": "text",
                "text": "226 Chapter 6. Complexity Theory\n(z0, a1) a2 . . . an✷ . . . ✷\n#\n#\n##\n#\n...\n#...\n✷ ✷ ✷ ✷ ✷ . . . (z1,1)p(n)\np(n)\nFigure 6.5: Thep(n)\u0002p(n)frame for the domino game.\nTherefore, we have A\u0014PDomino . Hence, the language Domino isNP-\ncomplete.\nAn example of a domino game\nWe have de\fned the domino game corresponding to a Turing machine that\nsolves a decision problem. Of course, we can also do this for Turing machines\nthat compute functions. In this section, we will exactly do this for a Turing\nmachine that computes the successor function x!x+ 1.\nWe will design a Turing machine with one tape, that gets as input the\nbinary representation of a natural number x, and that computes the binary\nrepresentation of x+ 1.\nStart of the computation: The tape contains a 0 followed by the binary\nrepresentation of the integer x2N0. The tape head is on the leftmost bit\n(which is 0), and the Turing machine is in the start state z0. Here is an\nexample, where x= 431:"
            }
        ]
    },
    {
        "section": "Page 235",
        "content": [
            {
                "type": "text",
                "text": "6.5. NP-complete languages 227\n01101011112\n6\nEnd of the computation: The tape contains the binary representation of\nthe number x+ 1. The tape head is on the rightmost 1, and the Turing\nmachine is in the \fnal state z1. For our example, the tape looks as follows:\n01101100002\n6\nOur Turing machine will use the following states:\nz0: start state; tape head moves to the right\nz1: \fnal state\nz2: tape head moves to the left; on its way to the left, it has not read 0\nThe Turing machine has the following instructions:\nz00!z00R z 21!z20L\nz01!z01R z 20!z11N\nz02!z22L\nIn Figure 6.6, you can see the sequence of states and tape contents of this\nTuring machine on input x= 11.\nWe now construct the domino game that corresponds to the computation\nof this Turing machine on input x= 11. Following the general construction\nin Section 6.5.3, we obtain the following tile types:\n1. The three symbols of the alphabet yield three tile types:\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n22\n2. The \fve instructions of the Turing machine yield \fve tile types:"
            }
        ]
    },
    {
        "section": "Page 236",
        "content": [
            {
                "type": "text",
                "text": "228 Chapter 6. Complexity Theory\n(z0;0) 1 0 1 1 2\n0 (z0;1) 0 1 1 2\n0 1 ( z0;0) 1 1 2\n0 1 0 ( z0;1) 1 2\n0 1 0 1 ( z0;1)2\n0 1 0 1 1 ( z0;2)\n0 1 0 1 ( z2;1)2\n0 1 0 ( z2;1) 0 2\n0 1 ( z2;0) 0 0 2\n0 1 ( z1;1) 0 0 2\nFigure 6.6: The computation of the Turing machine on input x= 11 . The\npair (state,symbol) indicates the position of the tape head.\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z0\n0(z0;0)\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z0\n1(z0;1)\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z2 #\n2(z0;2)\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z2 #\n0(z2;1)\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n(z1;1)(z2;0)\n3. The states z0andz2, and the three symbols of the alphabet yield twelve\ntile types:\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z0\n(z0;0)0\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z0 #\n(z0;0)0\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z0\n(z0;1)1\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z0 #\n(z0;1)1\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z0\n(z0;2)2\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z0 #\n(z0;2)2\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z2\n(z2;0)0\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z2 #\n(z2;0)0\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z2\n(z2;1)1\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z2 #\n(z2;1)1\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z2\n(z2;2)2\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z2 #\n(z2;2)2\nThe computation of the Turing machine on input x= 11 consists of nine\ncomputation steps. During this computation, the tape head visits exactly\nsix cells. Therefore, the frame for the domino game has nine rows and six\ncolumns. This frame is given in Figure 6.7. In Figure 6.8, you \fnd the\nsolution of the domino game. Observe that this solution is nothing but\nan equivalent way of writing the computation of Figure 6.6. Hence, the\ncomputation of the Turing machine corresponds to a solution of the domino\ngame; in fact, the converse also holds."
            }
        ]
    },
    {
        "section": "Page 237",
        "content": [
            {
                "type": "text",
                "text": "6.5. NP-complete languages 229\n0 1 (z1;1) 0 0 2(z0;0) 1 0 1 1 2\n#########\n#########\nFigure 6.7: The frame for the domino game for input x= 11 ."
            }
        ]
    },
    {
        "section": "Page 238",
        "content": [
            {
                "type": "text",
                "text": "230 Chapter 6. Complexity Theory\n0 1 (z1;1) 0 0 2(z0;0) 1 0 1 1 2\n#########\n#########\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n(z1;1)(z2;0)\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n22\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z2\n(z2;0)0\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z2 #\n0(z2;1)\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n22\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z2\n(z2;1)1\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z2 #\n0(z2;1)\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n22\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z2\n(z2;1)1\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z2 #\n2(z0;2)\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z0\n1(z0;1)\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z0 #\n(z0;2)2\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z0\n1(z0;1)\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z0 #\n(z0;1)1\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n22\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z0\n0(z0;0)\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z0 #\n(z0;1)1\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n22\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z0\n1(z0;1)\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z0 #\n(z0;0)0\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n22\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@#z0\n0(z0;0)\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@z0 #\n(z0;1)1\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n00\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n11\n\u0000\u0000\u0000\u0000\u0000@\n@\n@\n@@# #\n22\nFigure 6.8: The solution for the domino game for input x= 11 ."
            }
        ]
    },
    {
        "section": "Page 239",
        "content": [
            {
                "type": "text",
                "text": "6.5. NP-complete languages 231\n6.5.4 Examples of NP-complete languages\nIn Section 6.5.3, we have shown that Domino isNP-complete. Using this\nresult, we will apply Theorem 6.5.11 to prove the NP-completeness of some\nother languages.\nSatis\fability\nWe consider Boolean formulas ', in the variables x1;x2;:::;xm, having the\nform\n'=C1^C2^:::^Ck; (6.9)\nwhere each Ci, 1\u0014i\u0014k, is of the following form:\nCi=`i\n1_`i\n2_:::_`i\nki:\nEach`i\njis either a variable or the negation of a variable. Such a formula '\nis said to be satis\fable , if there exists a truth-value in f0;1gfor each of the\nvariablesx1;x2;:::;xm, such that the entire formula 'is true. We de\fne the\nfollowing language:\nSAT :=fh'i:'is of the form (6.9) and is satis\fable g:\nWe will prove that SAT isNP-complete.\nIt is clear that SAT2NP. If we can show that\nDomino\u0014PSAT;\nthen it follows from Theorem 6.5.11 that SAT isNP-complete. (In Theo-\nrem 6.5.11, take B:=Domino andC:=SAT.)\nHence, we need a function f2FP, that maps input strings for Domino\nto input strings for SAT, in such a way that for every domino game D, the\nfollowing holds:\ndomino game Dis solvable()the formula encoded by the\nstringf(hDi) is satis\fable :(6.10)\nLet us consider an arbitrary domino game D. Letkbe the number of\ntile types, and let the frame have trows andtcolumns. We denote the tile\ntypes byT1;T2;:::;Tk."
            }
        ]
    },
    {
        "section": "Page 240",
        "content": [
            {
                "type": "text",
                "text": "232 Chapter 6. Complexity Theory\nWe map this domino game Dto a Boolean formula ', such that (6.10)\nholds. The formula 'will have variables\nxij`;1\u0014i\u0014t;1\u0014j\u0014t;1\u0014`\u0014k:\nThese variables can be interpretated as follows:\nxij`= 1() there is a tile of type T`at position ( i;j) of the frame :\nWe de\fne:\n\u000fFor alliandjwith 1\u0014i\u0014tand 1\u0014j\u0014t:\nC1\nij:=xij1_xij2_:::_xijk:\nThis formula expresses the condition that there is at least one tile at\nposition (i;j).\n\u000fFor alli,j,`and`0with 1\u0014i\u0014t, 1\u0014j\u0014t, and 1\u0014`<`0\u0014k:\nC2\nij``0:=:xij`_:xij`0:\nThis formula expresses the condition that there is at most one tile at\nposition (i;j).\n\u000fFor alli,j,`and`0with 1\u0014i\u0014t, 1\u0014j <t , 1\u0014`\u0014kand 1\u0014`0\u0014k,\nsuch thati < t and the right symbol on a tile of type T`is not equal\nto the left symbol on a tile of type T`0:\nC3\nij``0:=:xij`_:xi;j+1;`0:\nThis formula expresses the condition that neighboring tiles in the same\nrow \\\ft\" together. There are symmetric formulas for neighboring tiles\nin the same column.\n\u000fFor alljand`with 1\u0014j\u0014tand 1\u0014`\u0014k, such that the top symbol\non a tile of type T`is not equal to the symbol at position jof the upper\nboundary of the frame:\nC4\nj`:=:x1j`:\nThis formula expresses the condition that tiles that touch the upper\nboundary of the frame \\\ft\" there. There are symmetric formulas for\nthe lower, left, and right boundaries of the frame."
            }
        ]
    },
    {
        "section": "Page 241",
        "content": [
            {
                "type": "text",
                "text": "6.5. NP-complete languages 233\nThe formula 'is the conjunction of all these formulas C1\nij,C2\nij``0,C3\nij``0, and\nC4\nj`. The complete formula 'consists of\nO(t2k+t2k2+t2k2+tk) =O(t2k2)\nterms, i.e., its length is polynomial in the length of the domino game. This\nimplies that 'can be constructed in polynomial time. Hence, the function\nfthat maps the domino game Dto the Boolean formula ', is in the class\nFP. It is not di\u000ecult to see that (6.10) holds for this function f. Therefore,\nwe have proved the following result.\nTheorem 6.5.13 The language SAT is NP-complete.\nIn Section 6.5.1, we have de\fned the language 3 SAT.\nTheorem 6.5.14 The language 3SAT is NP-complete.\nProof. It is clear that 3 SAT2NP. If we can show that\nSAT\u0014P3SAT;\nthen the claim follows from Theorem 6.5.11. Let\n'=C1^C2^:::^Ck\nbe an input for SAT, in the variables x1;x2;:::;xm. We map', in polynomial\ntime, to an input '0for 3 SAT, such that\n'is satis\fable()'0is satis\fable : (6.11)\nFor eachiwith 1\u0014i\u0014k, we do the following. Consider\nCi=`i\n1_`i\n2_:::_`i\nki:\n\u000fIfki= 1, then we de\fne\nC0\ni:=`i\n1_`i\n1_`i\n1:\n\u000fIfki= 2, then we de\fne\nC0\ni:=`i\n1_`i\n2_`i\n2:"
            }
        ]
    },
    {
        "section": "Page 242",
        "content": [
            {
                "type": "text",
                "text": "234 Chapter 6. Complexity Theory\n\u000fIfki= 3, then we de\fne\nC0\ni:=Ci:\n\u000fIfki\u00154, then we de\fne\nC0\ni:= (`i\n1_`i\n2_zi\n1)^(:zi\n1_`i\n3_zi\n2)^(:zi\n2_`i\n4_zi\n3)^:::\n^(:zi\nki\u00003_`i\nki\u00001_`i\nki);\nwherezi\n1;:::;zi\nki\u00003are new variables.\nLet\n'0:=C0\n1^C0\n2^:::^C0\nk:\nThen'0is an input for 3 SAT, and (6.11) holds.\nTheorems 6.5.6, 6.5.8, 6.5.11, and 6.5.14 imply:\nTheorem 6.5.15 The language Clique is NP-complete.\nThe traveling salesperson problem\nWe are given two positive integers kandm, a set ofmcities, and an integer\nm\u0002mmatrixM, where\nM(i;j) = the cost of driving from city ito cityj,\nfor alli;j2f1;2;:::;mg. We want to decide whether or not there is a tour\nthrough all cities whose total cost is less than or equal to k. This problem is\nNP-complete.\nBin packing\nWe are given three positive integers m,k, and`, a set ofmobjects having\nvolumesa1;a2;:::;am, andkbins. Each bin has volume `. We want to\ndecide whether or not all objects \ft within these bins. This problem is NP-\ncomplete.\nHere is another interpretation of this problem: We are given mjobs that\nneed timea1;a2;:::;amto complete. We are also given kprocessors, and an\ninteger`. We want to decide whether or not it is possible to divide the jobs\nover thekprocessors, such that no processor needs more than `time."
            }
        ]
    },
    {
        "section": "Page 243",
        "content": [
            {
                "type": "text",
                "text": "Exercises 235\nTime tables\nWe are given a set of courses, class rooms, and professors. We want to\ndecide whether or not there exists a time table such that all courses are\nbeing taught, no two courses are taught at the same time in the same class\nroom, no professor teaches two courses at the same time, and conditions such\nas \\Prof. L. Azy does not teach before 1pm\" are satis\fed. This problem is\nNP-complete.\nMotion planning\nWe are given two positive integers kand`, a set ofkpolyhedra, and two\npointssandtinQ3. We want to decide whether or not there exists a path\nbetweensandt, that does not intersect any of the polyhedra, and whose\nlength is less than or equal to `. This problem is NP-complete.\nMap labeling\nWe are given a map with mcities, where each city is represented by a point.\nFor each city, we are given a rectangle that is large enough to contain the\nname of the city. We want to decide whether or not these rectangles can be\nplaced on the map, such that\n\u000fno two rectangles overlap,\n\u000fFor eachiwith 1\u0014i\u0014m, the point that represents city iis a corner\nof its rectangle.\nThis problem is NP-complete.\nThis list of NP-complete problems can be extended almost arbitrarily:\nFor thousands of problems, it is known that they are NP-complete. For all\nof these, it is not known , whether or not they can be solved e\u000eciently (i.e.,\nin polynomial time). Collections of NP-complete problems can be found in\nthe book\n\u000fM.R. Garey and D.S. Johnson. Computers and Intractability: A Guide\nto the Theory of NP-Completeness. W.H. Freeman, New York, 1979,\nand on the web page\nhttp://www.nada.kth.se/~viggo/wwwcompendium/"
            }
        ]
    },
    {
        "section": "Page 244",
        "content": [
            {
                "type": "text",
                "text": "236 Chapter 6. Complexity Theory\nExercises\n6.1Prove that the function F:N!N, de\fned by F(x) := 2x, is not in FP.\n6.2Prove Theorem 6.5.3.\n6.3Prove that the language Clique is in the class NP.\n6.4Prove that the language 3 SAT is in the class NP.\n6.5We de\fne the following languages:\n\u000fSum of subset:\nSOS :=fha1;a2;:::;am;bi:9I\u0012f1;2;:::;mg;X\ni2Iai=bg:\n\u000fSet partition:\nSP:=fha1;a2;:::;ami:9I\u0012f1;2;:::;mg;X\ni2Iai=X\ni62Iaig:\n\u000fBin packing: BPis the set of all strings hs1;s2;:::;sm;Bifor which\n1. 0<si<1, for alli,\n2.B2N,\n3. the numbers s1;s2;:::;sm\ft intoBbins, where each bin has size\none, i.e., there exists a partition of f1;2;:::;mginto subsets Ik,\n1\u0014k\u0014B, such thatP\ni2Iksi\u00141 for allk, 1\u0014k\u0014B.\nFor example,h1=6;1=2;1=5;1=9;3=5;1=5;1=2;11=18;3i2BP, because\nthe eight fractions \ft into three bins:\n1=6 + 1=9 + 11=18\u00141;1=2 + 1=2 = 1;and 1=5 + 3=5 + 1=5 = 1:\n1. Prove that SOS\u0014PSP.\n2. Prove that the language SOS isNP-complete. You may use the fact\nthat the language SPisNP-complete."
            }
        ]
    },
    {
        "section": "Page 245",
        "content": [
            {
                "type": "text",
                "text": "Exercises 237\n3. Prove that the language BPisNP-complete. Again, you may use the\nfact that the language SPisNP-complete.\n6.6Prove that 3 Color\u0014P3SAT.\nHint: For each vertex i, and for each of the three colors k, introduce a\nBoolean variable xik.\n6.7The (0;1)-integer programming language IPis de\fned as follows:\nIP:=fhA;ci:Ais an integer m\u0002nmatrix for some m;n2N,\ncis an integer vector of length m, and\n9x2f0;1gnsuch thatAx\u0014c(componentwise) g.\nProve that the language IPisNP-complete. You may use the fact that\nthe language SOS isNP-complete.\n6.8Let'be a Boolean formula in the variables x1;x2;:::;xm.\nWe say that 'is in disjunctive normal form (DNF) if it is of the form\n'=C1_C2_:::_Ck; (6.12)\nwhere each Ci, 1\u0014i\u0014k, is of the following form:\nCi=`i\n1^`i\n2^:::^`i\nki:\nEach`i\njis aliteral , which is either a variable or the negation of a variable.\nWe say that 'is in conjunctive normal form (CNF) if it is of the form\n'=C1^C2^:::^Ck; (6.13)\nwhere each Ci, 1\u0014i\u0014k, is of the following form:\nCi=`i\n1_`i\n2_:::_`i\nki:\nAgain, each `i\njis a literal.\nWe de\fne the following two languages:\nDNFSAT :=fh'i:'is in DNF-form and is satis\fable g;\nand\nCNFSAT :=fh'i:'is in CNF-form and is satis\fable g:"
            }
        ]
    },
    {
        "section": "Page 246",
        "content": [
            {
                "type": "text",
                "text": "238 Chapter 6. Complexity Theory\n1. Prove that the language DNFSAT is inP.\n2. What is wrong with the following argument: Since we can rewrite\nany Boolean formula in DNF-form, we have CNFSAT\u0014PDNFSAT .\nHence, since CNFSAT isNP-complete and since DNFSAT2P, we\nhaveP=NP.\n3. Prove directly that for every language AinP,A\u0014PCNFSAT . \\Di-\nrectly\" means that you should not use the fact that CNFSAT isNP-\ncomplete.\n6.91Prove that the polynomial upper bound on the length of the string y\nin the de\fnition of NPis necessary, in the sense that if it is left out, then\nany enumerable language would satisfy the condition.\nMore precisely, we say that the language Abelongs to the class E, if there\nexists a language B2P, such that for every string w,\nw2A()9y:hw;yi2B:\nProve that Eis equal to the class of all enumerable languages.\n1Thanks to Antoine Vigneron for poining out an error in a previous version of this\nexercise."
            }
        ]
    },
    {
        "section": "Page 247",
        "content": [
            {
                "type": "text",
                "text": "Chapter 7\nSummary\nWe have seen several di\u000berent models for \\processing\" languages, i.e., pro-\ncessing sets of strings over some \fnite alphabet. For each of these models,\nwe have asked the question which types of languages can be processed, and\nwhich types of languages cannot be processed. In this \fnal chapter, we give\na brief summary of these results.\nRegular languages: This class of languages was considered in Chapter 2.\nThe following statements are equivalent:\n1. The language Ais regular, i.e., there exists a deterministic \fnite au-\ntomaton that accepts A.\n2. There exists a nondeterministic \fnite automaton that accepts A.\n3. There exists a regular expression that describes A.\nThis claim was proved by the following conversions:\n1. Every nondeterministic \fnite automaton can be converted to an equiv-\nalent deterministic \fnite automaton.\n2. Every deterministic \fnite automaton can be converted to an equivalent\nregular expression.\n3. Every regular expression can be converted to an equivalent nondeter-\nministic \fnite automaton."
            }
        ]
    },
    {
        "section": "Page 248",
        "content": [
            {
                "type": "text",
                "text": "240 Chapter 7. Summary\nWe have seen that the class of regular languages is closed under the regular\noperations: If AandBare regular languages, then\n1.A[Bis regular,\n2.ABis regular,\n3.A\u0003is regular,\n4.Ais regular, and\n5.A\\Bis regular.\nFinally, the pumping lemma for regular languages gives a property that\nevery regular language possesses. We have used this to prove that languages\nsuch asfanbn:n\u00150gare not regular.\nContext-free languages: This class of languages was considered in Chap-\nter 3. We have seen that every regular language is context-free. Moreover,\nthere exist languages, for example fanbn:n\u00150g, that are context-free, but\nnot regular. The following statements are equivalent:\n1. The language Ais context-free, i.e., there exists a context-free grammar\nwhose language is A.\n2. There exists a context-free grammar in Chomsky normal form whose\nlanguage is A.\n3. There exists a nondeterministic pushdown automaton that accepts A.\nThis claim was proved by the following conversions:\n1. Every context-free grammar can be converted to an equivalent context-\nfree grammar in Chomsky normal form.\n2. Every context-free grammar in Chomsky normal form can be converted\nto an equivalent nondeterministic pushdown automaton.\n3. Every nondeterministic pushdown automaton can be converted to an\nequivalent context-free grammar. (This conversion was not covered in\nthis book.)"
            }
        ]
    },
    {
        "section": "Page 249",
        "content": [
            {
                "type": "text",
                "text": "Chapter 7. Summary 241\nNondeterministic pushdown automata are more powerful than determin-\nistic pushdown automata: There exists a nondeterministic pushdown au-\ntomaton that accepts the language\nfvbw:v2fa;bg\u0003;w2fa;bg\u0003;jvj=jwjg;\nbut there is no deterministic pushdown automaton that accepts this language.\n(We did not prove this in this book.)\nWe have seen that the class of context-free languages is closed under\nthe union, concatenation, and star operations: If AandBare context-free\nlanguages, then\n1.A[Bis context-free,\n2.ABis context-free, and\n3.A\u0003is context-free.\nHowever,\n1. the intersection of two context-free languages is not necessarily context-\nfree, and\n2. the complement of a context-free language is not necessarily context-\nfree.\nFinally, the pumping lemma for context-free languages gives a property\nthat every context-free language possesses. We have used this to prove that\nlanguages such as fanbncn:n\u00150gare not context-free.\nThe Church-Turing Thesis: In Chapter 4, we considered \\reasonable\"\ncomputational devices that model real computers. Examples of such devices\nare Turing machines (with one or more tapes) and Java programs. It turns\nout that all known \\reasonable\" devices are equivalent, i.e., can be converted\nto each other. This led to the Church-Turing Thesis:\n\u000fEvery computational process that is intuitively considered to be an\nalgorithm can be converted to a Turing machine."
            }
        ]
    },
    {
        "section": "Page 250",
        "content": [
            {
                "type": "text",
                "text": "242 Chapter 7. Summary\nDecidable and enumerable languages: These classes of languages were\nconsidered in Chapter 5. They are de\fned based on \\reasonable\" computa-\ntional devices, such as Turing machines and Java programs. We have seen\nthat\n1. every context-free language is decidable, and\n2. every decidable language is enumerable.\nMoreover,\n1. there exist languages, for example fanbncn:n\u00150g, that are decidable,\nbut not context-free,\n2. there exist languages, for example the Halting Problem, that are enu-\nmerable, but not decidable,\n3. there exist languages, for example the complement of the Halting Prob-\nlem, that are not enumerable.\nIn fact,\n1. the class of all languages is not countable, whereas\n2. the class of all enumerable languages is countable.\nThe following statements are equivalent:\n1. The language Ais decidable.\n2. BothAand its complement Aare enumerable.\nComplexity classes: These classes of languages were considered in Chap-\nter 6.\n1. The class Pconsists of all languages that can be decided in polynomial\ntime by a deterministic Turing machine.\n2. The class NP consists of all languages that can be decided in poly-\nnomial time by a nondeterministic Turing machine. Equivalently, a\nlanguageAis in the class NP, if for every string w2A, there exists a\n\\solution\"s, such that (i) the length of sis polynomial in the length\nofw, and (ii) the correctness of scan be veri\fed in polynomial time."
            }
        ]
    },
    {
        "section": "Page 251",
        "content": [
            {
                "type": "text",
                "text": "Chapter 7. Summary 243\nThe following properties hold:\n1. Every context-free language is in P. (We did not prove this).\n2. Every language in Pis also in NP.\n3. It is not known if there exist languages that are in NP, but not in P.\n4. Every language in NPis decidable.\nWe have introduced reductions to de\fne the notion of a language Bto be\n\\at least as hard\" as a language A. A language Bis called NP-complete, if\n1.Bbelongs to the class NP, and\n2.Bis at least as hard as every language in the class NP.\nWe have seen that NP-complete exist.\nThe \fgure below summarizes the relationships among the various classes\nof languages."
            }
        ]
    },
    {
        "section": "Page 252",
        "content": [
            {
                "type": "text",
                "text": "244 Chapter 7. Summary\nregularcontext-freePNPdecidableenumerableall languages"
            }
        ]
    }
]