+ {
+ "cell_type": "code",
+ "execution_count": 57,
+ "metadata": {
+ "collapsed": false,
+ "scrolled": true
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "(8177, 249224)"
+ ]
+ },
+ "execution_count": 57,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "sum(sicp_starts.values()), sum(sum(c.values()) for c in sicp_counts.values())"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 58,
+ "metadata": {
+ "collapsed": false,
+ "scrolled": true
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "(26374, 958228)"
+ ]
+ },
+ "execution_count": 58,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "sum(kjb_starts.values()), sum(sum(c.values()) for c in kjb_counts.values())"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 91,
+ "metadata": {
+ "collapsed": false,
+ "scrolled": true
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "(24560, 683961)"
+ ]
+ },
+ "execution_count": 91,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "lovecraft = unaccent(open('lovecraft-trimmed.txt').read())\n",
+ "lovecraft_starts, lovecraft_counts = find_counts(sentences(tokenise(lovecraft)), tuple_size=2)\n",
+ "sum(lovecraft_starts.values()), sum(sum(c.values()) for c in lovecraft_counts.values())"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 92,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "3.225388284211814"
+ ]
+ },
+ "execution_count": 92,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "sum(kjb_starts.values()) / sum(sicp_starts.values())"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 93,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "3.8448464032356435"
+ ]
+ },
+ "execution_count": 93,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "sum(sum(c.values()) for c in kjb_counts.values()) / sum(sum(c.values()) for c in sicp_counts.values())"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 94,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "(3.0035465329582975, 2.7443625012037365)"
+ ]
+ },
+ "execution_count": 94,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "sum(lovecraft_starts.values()) / sum(sicp_starts.values()), \\\n",
+ "sum(sum(c.values()) for c in lovecraft_counts.values()) / sum(sum(c.values()) for c in sicp_counts.values())"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 95,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "def scale_merge(left_starts, left_start_scale, left_counts, left_count_scale, \n",
+ " right_starts, right_start_scale, right_counts, right_count_scale):\n",
+ " starts = collections.Counter()\n",
+ " counts = collections.defaultdict(collections.Counter)\n",
+ " \n",
+ " for k, n in left_starts.items():\n",
+ " starts[k] = n * left_start_scale\n",
+ " for k, n in right_starts.items():\n",
+ " starts[k] += n * right_start_scale\n",
+ " \n",
+ " for k in left_counts:\n",
+ " for j in left_counts[k]:\n",
+ " counts[k][j] = left_counts[k][j] * left_count_scale\n",
+ " \n",
+ " for k in right_counts:\n",
+ " for j in right_counts[k]:\n",
+ " counts[k][j] = right_counts[k][j] * right_count_scale\n",
+ "\n",
+ " return starts, counts"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 96,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "sk_starts, sk_counts = scale_merge(sicp_starts, 3, sicp_counts, 4, kjb_starts, 1, kjb_counts, 1)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 97,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<p>The voltage response v of the circuit ( summarized by v C, the Lisp evaluator would be to process the query ( and ( supervisor ? x ? middle- manager in the wheel rule of section 4.4.</p><p>3: 11 Therefore thy gates shall be burned therein.</p><p>36: 3 And say to the forest of the south shall be strong, and eat bread.</p><p>The servants said unto him, My LORD, if thou go with us; and he gave him, and saith unto them, ready to depart on the morrow, which was set over the affairs of this life, and out of the hand that wrote.</p><p>5: 6 And the sword shall abide on his cities, and with an extreme burning, and the vintage shall reach unto Azal: yea, though many false witnesses came, yet found they none.</p><p>He adds all the weekly meetings of the firm to the Microshaft data base by expressions of the query system repeatedly reads input expressions.</p><p>Specifically: The evaluator enables us to increase the modularity of our systems by encapsulating, or hiding,'' parts of the earth, that the LORD cast out before the LORD, as the LORD commanded to give unto her the cup of the Lord Jesus Christ, was in number three hundred thousand choice men, able to deliver you from his iniquities.</p><p>Cy agrees that Ben is right about the behavior of a quantity x as a function of n, to be applied in the same way as in the night visions, and your goodliest young men, and they slew all the males from a month old and upward, from the east side unto the west side, a portion for Asher.</p><p>13: 1 And the LORD said, Bring the portion which I gave thee, of the children of Israel before their idols, and all that is in use, existing programs that define procedures with these names bound as local variables.</p><p>For example, the complex number z = x + iy ( where i 2 =- 1) can be expressed as sequence operations.</p>"
+ ],
+ "text/plain": [
+ "<IPython.core.display.HTML object>"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "display(HTML(cat('<p>' + sentence_join(markov_item(sk_starts, sk_counts, 500)) + '</p>' for _ in range(10))))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 98,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "sl_starts, sl_counts = scale_merge(sicp_starts, 3, sicp_counts, 2, lovecraft_starts, 1, lovecraft_counts, 1)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 99,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<p>Akeley had been a nightmare of buzzing voices, and a partial relief from the second signal ordering a general plan was to see him.</p><p>From the handwriting ? Do you wonder how many farmhouses burnt to ashes.</p><p>1 that the interpreter runs, it follows a process that must be made to a system in which complex numbers are naturally represented as ordered pairs.</p><p>Then the GCD is the other argument for * ( assign proc ( op ( opcompiled- procedure- entry) ( let (( segments ( segments agenda) ( cdr z)) ( append (( save, first- reg) ( modifies- register ? modularity, [ 2 ] unify- match query- pattern frame) ( let (( t1 -> t2 ( apply- generic op ( t1 -> t2 ( get- register- contents < machine- model > < register- name > ( op vector- set!) ( reg new- cdrs) ( reg env));; begin actual procedure body ( save continue) ( goto ( reg val)) Now we can try our rational- number operations in terms of abstract selectors and constructors for this notation such that our derivative program still works ? 2.3.</p><p>And yet it was, or that those horrible cylinders and machines- and the desolate salt marshes, desolate and unpeopled, chaos was complete long before I die I should be as it shines on certain others- even a perfect solution, because of a long and familiar doom.</p><p>He defines the following two rules: For any list y, the empty list, then the smaller number in the sequence.</p><p>Of all its curious influence call up the sixty- three at the cryptic parchment; but upon examining the bank as if to accentuate by their fellow- men.</p><p>When he read this, but kept it in the hideous culmination of events that depopulated the whole scene of such data as my torch could not doubt the power of its unfamiliar temporary form, was of little quaint fishing towns that climbed from the Rowley road drew so close to the fourth floor, and likewise any metal case- resulted in a vast Italian quarter, and a feeling that some of our camel drivers older than Memphis and mankind.</p><p>\" Ye see all too close to the sight.</p><p>In the roseal dawn the burghers of Milwaukee rose to panic.</p>"
+ ],
+ "text/plain": [
+ "<IPython.core.display.HTML object>"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "display(HTML(cat('<p>' + sentence_join(markov_item(sl_starts, sl_counts, 500)) + '</p>' for _ in range(10))))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 100,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "lk_starts, lk_counts = scale_merge(lovecraft_starts, 1, lovecraft_counts, 1, kjb_starts, 1, kjb_counts, 1)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 101,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<p>I say harass, because they have when they tore open the massive pre- Revolutionary homes with their nails, so I think almost hope that no man has yet dared not seem to have the ether- resisting wings characteristic of him, and many things.</p><p>2: 31 And he said unto him, Caesar's.</p><p>33: 9 Nevertheless the priests of the high places of Baal, which are called might receive the fruits of your ground; neither shall thy tears run down like a river, and every tree therein: for then should ye go after vain things, which thou hast redeemed.</p><p>He could scarcely decipher what they portrayed, and almost ghastly results, since it could be done.</p><p>27: 12 These shall stand upon the mount of Olives, then sent Jesus two disciples, 21: 6 And Ezra blessed the LORD God, even the porch of the house of the LORD was with Jehoshaphat, because he had heard therefore that he was Rebekah's son: and I will remove far off from my presence: 23: 11 And they found written in the law.</p><p>Was he just come up.</p><p>Lanterns that shudder and death are penned.</p><p>But this was a cross of new developments in those fields where the west window, and with haste To get this impetus, though to my flagging quest; for thousands and tens of thousands of light which would warn the waiting horse gave a thrill that a small man probably having a kind of a chaotic pronunciation.</p><p>Beneath him dozens of queer things I had not traversed before; and though progress was very close indeed.</p><p>52: 23 And they were exceeding many: neither was the word of the LORD came again unto Jerusalem and Judah, Hanani, Eliathah, Giddalti, and Romamtiezer, Joshbekashah, Mallothi, Hothir, and Mahazioth: 25: 27 Over against the border were at the north bay of the salt sea: this shall not be ashamed of Chemosh, as the manner of all the tribes of the children of Ammon said to Hanun, Thinkest thou that David doth honour thy father, that he may die: because she hath not sinned.</p>"
+ ],
+ "text/plain": [
+ "<IPython.core.display.HTML object>"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "display(HTML(cat('<p>' + sentence_join(markov_item(lk_starts, lk_counts, 500)) + '</p>' for _ in range(10))))"
+ ]
+ },