Done challenge 4

diff --git a/2020/2020-a-challenge4.ipynb b/2020/2020-a-challenge4.ipynb
new file mode 100644 (file)
index 0000000..19f01c1
--- /dev/null
+++ b/2020/2020-a-challenge4.ipynb
@@ -0,0 +1,434 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [],
+   "source": [
+    "from szyfrow.keyword_cipher import *\n",
+    "from szyfrow.column_transposition import *\n",
+    "from szyfrow.support.text_prettify import *"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [],
+   "source": [
+    "challenge_number = 4\n",
+    "plaintext_a_filename = f'plaintext.{challenge_number}a.txt'\n",
+    "plaintext_b_filename = f'plaintext.{challenge_number}b.txt'\n",
+    "ciphertext_a_filename = f'ciphertext.{challenge_number}a.txt'\n",
+    "ciphertext_b_filename = f'ciphertext.{challenge_number}b.txt'"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [],
+   "source": [
+    "ca = open(ciphertext_a_filename).read()\n",
+    "sca = sanitise(ca)\n",
+    "cb = open(ciphertext_b_filename).read()\n",
+    "scb = sanitise(cb)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "akkad KeywordWrapAlphabet.from_largest \n",
+      "\n",
+      "i have drawn a blank in my enqciries aboct uaistor st edmcnd and even harry is getting nowhere with his uontauts in the mod. we both agree that the silenue is very worrying. this is ulearly an area of interest as it presents a urcuial front in any potential invasion so we wocld have expeuted some interest in ocr intelligenue, or at least some pcsh bauk warning cs to keep away from basiu defenue installations like listening posts. the faut that no-one wants to talk aboct the fauilities in the area scggests that it is all very hcsh hcsh. something important is happening there and we probably want to find oct what.\n",
+      "\n",
+      "ocr agents in london kept watuh for the tocring party and reuorded them as arriving at liverpool street. the grocp uonsisted of twenty three uyulists who split cp and stayed with members of loual suoct grocps while their leader was hosted by a member of the loual ortsgrcppe. we reueived reports from k's network that the grocp has been invited to stay with the spalding rotary ulcb in linuoln and we need yoc to investigate any reports yoc uan find aboct that visit. i will uontince to press for information aboct strategiu developments in norwiuh and norfolk.\n",
+      "\n",
+      "the akela initiative has been given approval and i will be setting that cp this week. the intention is to enroll leaders and members of loual suoct grocps, train them in uocnter-intelligenue and cse them to host the spyulist tocring grocps in the hope of getting inside their operation. sinue fctcre uommcniuations may inulcde sensitive operational details of this initiative i intend to inurease seucrity by moving to a blouked keyword uipher.\n",
+      "\n",
+      "the attauhed doucment was interuepted by ocr london agents when the party posted it on arrival at liverpool street. as cscal it was addressed to tirpitzcfer. primary analysis scggests that it does not cse a scbstitction uipher.\n",
+      "\n",
+      "more to follow.\n",
+      "\n",
+      "pearl.\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "(word_a, wrap_a), score_a = keyword_break_mp(sca, fitness=Ptrigrams)\n",
+    "print(word_a, wrap_a, '\\n')\n",
+    "pa = keyword_decipher(ca, word_a, wrap_a)\n",
+    "print(pa)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "akelmnopqrstuvwxyzbcdfghij \n",
+      "\n",
+      "i have drawn a blank in my enquiries about caistor st edmund and even harry is getting nowhere with his contacts in the mod. we both agree that the silence is very worrying. this is clearly an area of interest as it presents a crucial front in any potential invasion so we would have expected some interest in our intelligence, or at least some push back warning us to keep away from basic defence installations like listening posts. the fact that no-one wants to talk about the facilities in the area suggests that it is all very hush hush. something important is happening there and we probably want to find out what.\n",
+      "\n",
+      "our agents in london kept watch for the touring party and recorded them as arriving at liverpool street. the group consisted of twenty three cyclists who split up and stayed with members of local scout groups while their leader was hosted by a member of the local ortsgruppe. we received reports from k's network that the group has been invited to stay with the spalding rotary club in lincoln and we need you to investigate any reports you can find about that visit. i will continue to press for information about strategic developments in norwich and norfolk.\n",
+      "\n",
+      "the akela initiative has been given approval and i will be setting that up this week. the intention is to enroll leaders and members of local scout groups, train them in counter-intelligence and use them to host the spyclist touring groups in the hope of getting inside their operation. since future communications may include sensitive operational details of this initiative i intend to increase security by moving to a blocked keyword cipher.\n",
+      "\n",
+      "the attached document was intercepted by our london agents when the party posted it on arrival at liverpool street. as usual it was addressed to tirpitzufer. primary analysis suggests that it does not use a substitution cipher.\n",
+      "\n",
+      "more to follow.\n",
+      "\n",
+      "pearl.\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "word_a, score_a = simulated_annealing_break(sca, fitness=Ptrigrams,\n",
+    "        plain_alphabet=string.ascii_lowercase, cipher_alphabet=keyword_cipher_alphabet_of('akkad', wrap_alphabet=KeywordWrapAlphabet.from_largest))\n",
+    "print(word_a, '\\n')\n",
+    "pa = keyword_decipher(ca, word_a)\n",
+    "print(pa)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "i have drawn a blank in my enquiries about caistor st edmund and even harry is getting nowhere with his contacts in the mod. we both agree that the silence is very worrying. this is clearly an area of interest as it presents a crucial front in any potential invasion so we would have expected some interest in our intelligence, or at least some push back warning us to keep away from basic defence installations like listening posts. the fact that no-one wants to talk about the facilities in the area suggests that it is all very hush hush. something important is happening there and we probably want to find out what.\n",
+      "\n",
+      "our agents in london kept watch for the touring party and recorded them as arriving at liverpool street. the group consisted of twenty three cyclists who split up and stayed with members of local scout groups while their leader was hosted by a member of the local ortsgruppe. we received reports from k's network that the group has been invited to stay with the spalding rotary club in lincoln and we need you to investigate any reports you can find about that visit. i will continue to press for information about strategic developments in norwich and norfolk.\n",
+      "\n",
+      "the akela initiative has been given approval and i will be setting that up this week. the intention is to enroll leaders and members of local scout groups, train them in counter-intelligence and use them to host the spyclist touring groups in the hope of getting inside their operation. since future communications may include sensitive operational details of this initiative i intend to increase security by moving to a blocked keyword cipher.\n",
+      "\n",
+      "the attached document was intercepted by our london agents when the party posted it on arrival at liverpool street. as usual it was addressed to tirpitzufer. primary analysis suggests that it does not use a substitution cipher.\n",
+      "\n",
+      "more to follow.\n",
+      "\n",
+      "pearl.\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "pa = keyword_decipher(ca, 'akela', wrap_alphabet=KeywordWrapAlphabet.from_largest)\n",
+    "print(pa)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "1883"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "open(plaintext_a_filename, 'w').write(pa)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [],
+   "source": [
+    "clue_dict = ['alerting', 'altering', 'integral', 'relating', 'triangle', 'angriest', 'gantries', 'granites', 'ingrates', 'rangiest', 'tangiers', \n",
+    "             'dniester', 'inserted', 'nerdiest', 'resident', 'trendies', 'respects', 'scepters', 'sceptres', 'specters', 'spectres', 'restrain', \n",
+    "             'retrains', 'strainer', 'terrains', 'trainers']\n",
+    "clue_trans = [transpositions_of(w) for w in clue_dict]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "(1, 2, 3, 5, 0, 4) True False \n",
+      "\n",
+      "REERNLNHNRTAANNBSCROIAUSHOALISICTETSIEAARASTSOSLNOIOOWESEIOGMHISTXDIEGEAGATTADRVSICAYSITREARESOFNCFFWPNSPPTNTAELHYBNRRAMNSMSXOENGPNINFAFOGDLOMENOORHNTNERMTATFTWATFAESOGRYNCSHSENTEDAOTRKIBPPARHCERRTTHNNTGIREITMVHEDCONTCTUWPDOIROENHEGIKIATNCWLNODRSSNTEOGAFREBTAEHOLWHOSSITPTRNCUNLECLOUTDSTHNTSLLHTRTGNDCRERSESALLBEOCOAUEOEOUAEHIDVADSAHETISIVTGLNNESTTPCIWUFRAETIRHIIWFLEREYTEEOKHYRFOTIRERNERTSFIILFCINYTSALGAHHSDIVSLLSUTWSTELNPOOSISOSHAYNEGNLOGTORDTHTRRORRSDISUEOEIALSTKGASPUNBNLUMYIIBSIELHEEAIGIRISSCRCVNATGTYIOIAIUTTEANNNURSCMMHEEAPEOIRIRTESOODTHILWCESTTYSRALHUROOSAAGRRIOEIAHHFSOYTRMATSEAUSNKVVSWTLSNTDONOENFTYMRHTCNETHLDEWWHSPHNSTALNIIEEETTAFTASSFHASRAEOSEPUEOXAOOHHTTTNIEENYAIOYATVIEIISEOLIYGEIEIROTPUTASARUETHONOIPNIHUUTOEHIUNTUWISTNRYNSREAOUNCTEMFEIFGEISNRHOCYORTRLANATASOYHIFTHHHAGMNTNMWOICTPIRAEEAIARNNLDUNYFEHRYURTOONDIONTAOOIEUWLJEOWODMHITGGRHKOPENNIEAYMMANETIRPDRBYACCRPUTOHRSRSIMOBSUMEOWPHTATORENNBHHNTAIHNALIDCOOLWOPUFODIFAEOIFELITHNBEEAEOSTNIDIERILBDOAQTEVGEUEWLUEIDSRBEDEDAETHVBOTISHTTNTMEWOSTFEEDCODTDUMEATLISLOETVOLSHIJGETTASAHEIEBIHFSERHNHREROOUDDTTTTYGNATOHAGWIVISGTACENOIRUHRNAUDAEUTPNHYPETSRCY                                                                                                                                                                                                                                                                                     \n",
+      "DNIMOYNARCIENTATITSUELDTDIDBOPBPSHRAREYLTVROYERSNAGEUTTBFRHTONUIEEFYHRUXGEOEEDHCTETPCTIUTUVAMLRNLTTERRRTONOOHEDUOCSRFAAOTINWENEETSNUISIEASSEURHNOUTRRSRTSTHEEERELSESYTEETIUTIOIAIARMHISEMBIECISSSREMIEOAOLNOIDPLESCYUNATMROETIOIMNUTANSHHHOHTKOUENONOIAUTHRHSNRTATEEPPCFRMRSAGASOSEESE\n"
+     ]
+    }
+   ],
+   "source": [
+    "(trans_b, fill_b, empty_b), score_b = column_transposition_break(scb, translist=clue_trans, fitness=Ptrigrams)\n",
+    "print(trans_b, fill_b, empty_b, '\\n')\n",
+    "pb = column_transposition_decipher(cb, trans_b, \n",
+    "    fillcolumnwise=fill_b,\n",
+    "    emptycolumnwise=empty_b)\n",
+    "print(pb)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "(1, 2, 4, 0, 3) False False \n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "(trans_b, fill_b, empty_b), score_b = column_transposition_break(scb, fitness=Ptrigrams)\n",
+    "print(trans_b, fill_b, empty_b, '\\n')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "dear uncle wilhelm our journey to london was short and charming the trains rattling along the\n",
+      "british tracks through open fields and countryside their roads maybe almost impassable in places but\n",
+      "these iron tracks provide a very usable way to travel across country at speed our hosts in london\n",
+      "have organised for us to stay with members of the british scouting movement though i will spend the\n",
+      "next few days with the orts gruppe exchanging news about the fatherland and their activities herein\n",
+      "the capital city it was difficult to tear ourselves away from the pleasures of norfolk and the city\n",
+      "offers far fewer opportunities for open exploration on the other hand the bustle of the city as in\n",
+      "berlin offers a certain anonymity and it is in some ways relaxing to leave behind the gossip and\n",
+      "inquisitiveness of village life as you suggested we will use our time herein london to support our\n",
+      "brothers and sisters and to reassure them that they have not been forgotten i will pass on the gifts\n",
+      "that you entrusted to me together with your instructions for their use as we did in caistor st\n",
+      "edmund the radio you sent me is remarkable it is able to pickup signals across the spectrum even\n",
+      "from berlin itself though i am enjoying listening to british radio especially the home service they\n",
+      "are building a chain of transmitters across the country which provide information throughout the day\n",
+      "and night something that i think you might want to take into account when planning our own radio\n",
+      "services i am assuming that there are no changes planned for our itinerary but if there are then\n",
+      "perhaps you could forward them to our hosts in spalding with a copy sent post restante to cse yours\n",
+      "sincerely\n"
+     ]
+    }
+   ],
+   "source": [
+    "pb = prettify(column_transposition_decipher(scb, trans_b, \n",
+    "    fillcolumnwise=fill_b,\n",
+    "    emptycolumnwise=empty_b))\n",
+    "print(pb)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "['fable',\n",
+       " 'gable',\n",
+       " 'nacre',\n",
+       " 'nahum',\n",
+       " 'oakum',\n",
+       " 'odium',\n",
+       " 'padre',\n",
+       " 'pinto',\n",
+       " 'salvo',\n",
+       " 'santo',\n",
+       " 'scour',\n",
+       " 'tabus',\n",
+       " 'tagus',\n",
+       " 'talus',\n",
+       " 'thous',\n",
+       " 'timur',\n",
+       " 'torus',\n",
+       " 'torys',\n",
+       " 'whizs',\n",
+       " 'gabble',\n",
+       " 'kabuki',\n",
+       " 'lactic',\n",
+       " 'ladoga',\n",
+       " 'leftie',\n",
+       " 'madrid',\n",
+       " 'megohm',\n",
+       " 'oberon',\n",
+       " 'raisin',\n",
+       " 'refuel',\n",
+       " 'refuge',\n",
+       " 'salvos',\n",
+       " 'santos',\n",
+       " 'scours',\n",
+       " 'tabbys',\n",
+       " 'tabula',\n",
+       " 'taffys',\n",
+       " 'taguss',\n",
+       " 'taiwan',\n",
+       " 'tallys',\n",
+       " 'tammys',\n",
+       " 'tanyas',\n",
+       " 'telexs',\n",
+       " 'tenure',\n",
+       " 'terrys',\n",
+       " 'tethys',\n",
+       " 'thrush',\n",
+       " 'thrust',\n",
+       " 'timmys',\n",
+       " 'toruss',\n",
+       " 'velezs',\n",
+       " 'vilyui',\n",
+       " 'weepys',\n",
+       " 'whizzs',\n",
+       " 'willys',\n",
+       " 'worrys',\n",
+       " 'lactate',\n",
+       " 'macrame',\n",
+       " 'palazzo',\n",
+       " 'raisins',\n",
+       " 'raritan',\n",
+       " 'refresh',\n",
+       " 'refugee',\n",
+       " 'sabbath',\n",
+       " 'salazar',\n",
+       " 'sinuous',\n",
+       " 'tactual',\n",
+       " 'tarawas',\n",
+       " 'telexes',\n",
+       " 'tethyss',\n",
+       " 'thrushs',\n",
+       " 'thrusts',\n",
+       " 'maharaja',\n",
+       " 'nakayama',\n",
+       " 'ramayana',\n",
+       " 'sabbaths',\n",
+       " 'salazars',\n",
+       " 'semester',\n",
+       " 'sensuous',\n",
+       " 'teletype',\n",
+       " 'tortuous',\n",
+       " 'maharajah',\n",
+       " 'refresher',\n",
+       " 'refreshes',\n",
+       " 'semesters',\n",
+       " 'refreshers',\n",
+       " 'sensuousness',\n",
+       " 'sensuousnesss']"
+      ]
+     },
+     "execution_count": 20,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "transpositions[trans_b]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "1685"
+      ]
+     },
+     "execution_count": 21,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "open(plaintext_b_filename, 'w').write(pb)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "jupytext": {
+   "formats": "ipynb,md"
+  },
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.4"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

diff --git a/2020/2020-a-challenge4.md b/2020/2020-a-challenge4.md
new file mode 100644 (file)
index 0000000..032af74
--- /dev/null
+++ b/2020/2020-a-challenge4.md
@@ -0,0 +1,99 @@
+---
+jupyter:
+  jupytext:
+    formats: ipynb,md
+    text_representation:
+      extension: .md
+      format_name: markdown
+      format_version: '1.2'
+      jupytext_version: 1.3.4
+  kernelspec:
+    display_name: Python 3
+    language: python
+    name: python3
+---
+
+```python Collapsed="false"
+from szyfrow.keyword_cipher import *
+from szyfrow.column_transposition import *
+from szyfrow.support.text_prettify import *
+```
+
+```python Collapsed="false"
+challenge_number = 4
+plaintext_a_filename = f'plaintext.{challenge_number}a.txt'
+plaintext_b_filename = f'plaintext.{challenge_number}b.txt'
+ciphertext_a_filename = f'ciphertext.{challenge_number}a.txt'
+ciphertext_b_filename = f'ciphertext.{challenge_number}b.txt'
+```
+
+```python Collapsed="false"
+ca = open(ciphertext_a_filename).read()
+sca = sanitise(ca)
+cb = open(ciphertext_b_filename).read()
+scb = sanitise(cb)
+```
+
+```python Collapsed="false"
+(word_a, wrap_a), score_a = keyword_break_mp(sca, fitness=Ptrigrams)
+print(word_a, wrap_a, '\n')
+pa = keyword_decipher(ca, word_a, wrap_a)
+print(pa)
+```
+
+```python Collapsed="false"
+word_a, score_a = simulated_annealing_break(sca, fitness=Ptrigrams,
+        plain_alphabet=string.ascii_lowercase, cipher_alphabet=keyword_cipher_alphabet_of('akkad', wrap_alphabet=KeywordWrapAlphabet.from_largest))
+print(word_a, '\n')
+pa = keyword_decipher(ca, word_a)
+print(pa)
+```
+
+```python Collapsed="false"
+pa = keyword_decipher(ca, 'akela', wrap_alphabet=KeywordWrapAlphabet.from_largest)
+print(pa)
+```
+
+```python Collapsed="false"
+open(plaintext_a_filename, 'w').write(pa)
+```
+
+```python Collapsed="false"
+clue_dict = ['alerting', 'altering', 'integral', 'relating', 'triangle', 'angriest', 'gantries', 'granites', 'ingrates', 'rangiest', 'tangiers', 
+             'dniester', 'inserted', 'nerdiest', 'resident', 'trendies', 'respects', 'scepters', 'sceptres', 'specters', 'spectres', 'restrain', 
+             'retrains', 'strainer', 'terrains', 'trainers']
+clue_trans = [transpositions_of(w) for w in clue_dict]
+```
+
+```python Collapsed="false"
+(trans_b, fill_b, empty_b), score_b = column_transposition_break(scb, translist=clue_trans, fitness=Ptrigrams)
+print(trans_b, fill_b, empty_b, '\n')
+pb = column_transposition_decipher(cb, trans_b, 
+    fillcolumnwise=fill_b,
+    emptycolumnwise=empty_b)
+print(pb)
+```
+
+```python Collapsed="false"
+(trans_b, fill_b, empty_b), score_b = column_transposition_break(scb, fitness=Ptrigrams)
+print(trans_b, fill_b, empty_b, '\n')
+```
+
+```python Collapsed="false"
+pb = prettify(column_transposition_decipher(scb, trans_b, 
+    fillcolumnwise=fill_b,
+    emptycolumnwise=empty_b))
+print(pb)
+```
+
+```python Collapsed="false"
+transpositions[trans_b]
+```
+
+```python Collapsed="false"
+open(plaintext_b_filename, 'w').write(pb)
+```
+
+```python Collapsed="false"
+
+```

diff --git a/2020/challenge3-clue.ipynb b/2020/challenge3-clue.ipynb
index ada44458733e900e1bbc15a38639b68bc86524c0..a79b1990ce9e58ecc29a5651e5d264c72ea36d97 100644 (file)
--- a/2020/challenge3-clue.ipynb
+++ b/2020/challenge3-clue.ipynb
@@ -3,7 +3,9 @@
   {
    "cell_type": "code",
    "execution_count": 1,
-   "metadata": {},
+   "metadata": {
+    "Collapsed": "false"
+   },
    "outputs": [],
    "source": [
     "from szyfrow.support.language_models import *\n",
@@ -14,7 +16,9 @@
   {
    "cell_type": "code",
    "execution_count": 2,
-   "metadata": {},
+   "metadata": {
+    "Collapsed": "false"
+   },
    "outputs": [
     {
      "data": {
@@ -34,8 +38,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 4,
-   "metadata": {},
+   "execution_count": 3,
+   "metadata": {
+    "Collapsed": "false"
+   },
    "outputs": [],
    "source": [
     "anags8 = collections.defaultdict(list)\n",
@@ -45,8 +51,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
-   "metadata": {},
+   "execution_count": 4,
+   "metadata": {
+    "Collapsed": "false"
+   },
    "outputs": [
     {
      "data": {
@@ -54,7 +62,7 @@
        "['aegilnrt', 'aeginrst', 'deeinrst', 'ceeprsst', 'aeinrrst']"
       ]
      },
-     "execution_count": 5,
+     "execution_count": 4,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -66,8 +74,10 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
-   "metadata": {},
+   "execution_count": 5,
+   "metadata": {
+    "Collapsed": "false"
+   },
    "outputs": [
     {
      "name": "stdout",
@@ -86,10 +96,34 @@
     "    print(c, anags8[c])"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "\"alerting', 'altering', 'integral', 'relating', 'triangle', 'angriest', 'gantries', 'granites', 'ingrates', 'rangiest', 'tangiers', 'dniester', 'inserted', 'nerdiest', 'resident', 'trendies', 'respects', 'scepters', 'sceptres', 'specters', 'spectres', 'restrain', 'retrains', 'strainer', 'terrains', 'trainers\""
+      ]
+     },
+     "execution_count": 13,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "\"', '\".join(w for c in cands for w in anags8[c])"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
-   "metadata": {},
+   "metadata": {
+    "Collapsed": "false"
+   },
    "outputs": [],
    "source": []
   }

diff --git a/2020/ciphertext.4a.txt b/2020/ciphertext.4a.txt
new file mode 100644 (file)
index 0000000..0d2f887
--- /dev/null
+++ b/2020/ciphertext.4a.txt
@@ -0,0 +1,11 @@
+Q PAFM LZAGV A KTAVS QV UI MVYDQZQMB AKWDC EAQBCWZ BC MLUDVL AVL MFMV PAZZI QB OMCCQVO VWGPMZM GQCP PQB EWVCAECB QV CPM UWL. GM KWCP AOZMM CPAC CPM BQTMVEM QB FMZI GWZZIQVO. CPQB QB ETMAZTI AV AZMA WN QVCMZMBC AB QC XZMBMVCB A EZDEQAT NZWVC QV AVI XWCMVCQAT QVFABQWV BW GM GWDTL PAFM MHXMECML BWUM QVCMZMBC QV WDZ QVCMTTQOMVEM, WZ AC TMABC BWUM XDBP KAES GAZVQVO DB CW SMMX AGAI NZWU KABQE LMNMVEM QVBCATTACQWVB TQSM TQBCMVQVO XWBCB. CPM NAEC CPAC VW-WVM GAVCB CW CATS AKWDC CPM NAEQTQCQMB QV CPM AZMA BDOOMBCB CPAC QC QB ATT FMZI PDBP PDBP. BWUMCPQVO QUXWZCAVC QB PAXXMVQVO CPMZM AVL GM XZWKAKTI GAVC CW NQVL WDC GPAC.
+
+WDZ AOMVCB QV TWVLWV SMXC GACEP NWZ CPM CWDZQVO XAZCI AVL ZMEWZLML CPMU AB AZZQFQVO AC TQFMZXWWT BCZMMC. CPM OZWDX EWVBQBCML WN CGMVCI CPZMM EIETQBCB GPW BXTQC DX AVL BCAIML GQCP UMUKMZB WN TWEAT BEWDC OZWDXB GPQTM CPMQZ TMALMZ GAB PWBCML KI A UMUKMZ WN CPM TWEAT WZCBOZDXXM. GM ZMEMQFML ZMXWZCB NZWU S'B VMCGWZS CPAC CPM OZWDX PAB KMMV QVFQCML CW BCAI GQCP CPM BXATLQVO ZWCAZI ETDK QV TQVEWTV AVL GM VMML IWD CW QVFMBCQOACM AVI ZMXWZCB IWD EAV NQVL AKWDC CPAC FQBQC. Q GQTT EWVCQVDM CW XZMBB NWZ QVNWZUACQWV AKWDC BCZACMOQE LMFMTWXUMVCB QV VWZGQEP AVL VWZNWTS.
+
+CPM ASMTA QVQCQACQFM PAB KMMV OQFMV AXXZWFAT AVL Q GQTT KM BMCCQVO CPAC DX CPQB GMMS. CPM QVCMVCQWV QB CW MVZWTT TMALMZB AVL UMUKMZB WN TWEAT BEWDC OZWDXB, CZAQV CPMU QV EWDVCMZ-QVCMTTQOMVEM AVL DBM CPMU CW PWBC CPM BXIETQBC CWDZQVO OZWDXB QV CPM PWXM WN OMCCQVO QVBQLM CPMQZ WXMZACQWV. BQVEM NDCDZM EWUUDVQEACQWVB UAI QVETDLM BMVBQCQFM WXMZACQWVAT LMCAQTB WN CPQB QVQCQACQFM Q QVCMVL CW QVEZMABM BMEDZQCI KI UWFQVO CW A KTWESML SMIGWZL EQXPMZ.
+
+CPM ACCAEPML LWEDUMVC GAB QVCMZEMXCML KI WDZ TWVLWV AOMVCB GPMV CPM XAZCI XWBCML QC WV AZZQFAT AC TQFMZXWWT BCZMMC. AB DBDAT QC GAB ALLZMBBML CW CQZXQCJDNMZ. XZQUAZI AVATIBQB BDOOMBCB CPAC QC LWMB VWC DBM A BDKBCQCDCQWV EQXPMZ.
+
+UWZM CW NWTTWG.
+
+XMAZT.

diff --git a/2020/ciphertext.4b.txt b/2020/ciphertext.4b.txt
new file mode 100644 (file)
index 0000000..84feab4
--- /dev/null
+++ b/2020/ciphertext.4b.txt
@@ -0,0 +1 @@
+EAUDR CLWNE LHLIE OUJMR UREON TOOYL DOWNN SSOAH TADRN HAMCR NGHIT TRIEA SRTNA LIGTN LOGAN HERTB TIHIS RAKTC THOSR GHPUO NFEEI DSNLA CONDU RYITS ETEDH RRAIO SMYDA EAMBL STMOI ASAPS LENBI LAEPC BUTST ESIHE ONRRT CKPAS OVDRI AVREE USBYA EWYLA OTATR ELCVA OSCRS UNROT ATPYS EDUEO HOTRS INOSL DOHNN VERAO ANSGI DFREO STSUO AYITW HMMTE EROBS THBFE ITSRI SCUHO INMTG VEEOM TTONH GHWUI LLPIS NDHET NETEX EWAFD SWTYI THOHE TSRRG PPEUE CHNXA INNGG WSBEA UTHOT FAHET RLNEA ANTDD EIAHR TIICV IEHTS RENEI HEATC ITLPA ITICY WADTS FFCII LTOUT EAOTR RSLUE ESWVA YFOAR THPME EAULS ESFRO ORONF KADLN HEITC YOFTF RSAEF FEERW OPORP TUIRN IEFTS ROEOP EXLNP RAIOT NOTON EOHHT RHNEA THBDE STEUL FTEOH ITACY INESB LIORN FESFR CETAR INNAA NYIOM YADTN TIIIS SOENM AYRWS LAIEX GTLNO AVBEE HIDEN HEOTG SIASP DIQNN ISTUI VEEIN SOVSF LLGIA LIEEF SYUAO UGESG TEWSD WILEL SEUUO TIERM ERIHE LODNN NTSOO PPRUO OUBTR OTERH SADRN ISEST SADRN ORATE SUESR HETTM ATHHT YHVEA NOBET ENOEF GOTRT NIIEW LPSLA ONHST GITEF THTSA OUNYE RUTTS DTMEO TOEEG HEWTR THOIY RISUN RUTTC ONFIS RTEOH RUEIS SWDAE DICIN ISOAT STDRE UNTMD ERDHA OYUIO ENMST ISEER ARAMK LETBI SALIB TOIEP KUSCP GNLIA ACOSR STESH PETSC UMVRE NFOER BELMR NISIT LFHET UGIOH MEJAN YIGON ISELT INTNG BRTOI SHAIR IOSDE ECAPI LYHLT HOEEM ERISV ETECH ARBYE ILIUD GAHNC INFAO RASTN ITEMT SARRC SSHOT CONEU RYHTW CHRIP VIEOD NFRIO ATOMI THONR GHUUO THDTE YADAN IGTNH OMTSE INTHG ATTHI INYHK UMGOI TWNHA TOATT EITKN ACOOC NTHUW NPAEL NIGNN URWOO RAIND SEVOR CEIIS MASAS MIGUN HATTT ERAHE ENCRO ANEHG PLNSA EDONF OUIRR INRTE RYUAB IFHTT REREA THNEE ERAPH SYUPO OUDCL ORAFW DTERH TOUMO HOTRS INPSS LDNAI WIHGT COYAP ENPST STEOR TATSN TOSEC YOREU SICSN REYEL

diff --git a/2020/plaintext.4a.txt b/2020/plaintext.4a.txt
new file mode 100644 (file)
index 0000000..2de18c2
--- /dev/null
+++ b/2020/plaintext.4a.txt
@@ -0,0 +1,11 @@
+i have drawn a blank in my enquiries about caistor st edmund and even harry is getting nowhere with his contacts in the mod. we both agree that the silence is very worrying. this is clearly an area of interest as it presents a crucial front in any potential invasion so we would have expected some interest in our intelligence, or at least some push back warning us to keep away from basic defence installations like listening posts. the fact that no-one wants to talk about the facilities in the area suggests that it is all very hush hush. something important is happening there and we probably want to find out what.
+
+our agents in london kept watch for the touring party and recorded them as arriving at liverpool street. the group consisted of twenty three cyclists who split up and stayed with members of local scout groups while their leader was hosted by a member of the local ortsgruppe. we received reports from k's network that the group has been invited to stay with the spalding rotary club in lincoln and we need you to investigate any reports you can find about that visit. i will continue to press for information about strategic developments in norwich and norfolk.
+
+the akela initiative has been given approval and i will be setting that up this week. the intention is to enroll leaders and members of local scout groups, train them in counter-intelligence and use them to host the spyclist touring groups in the hope of getting inside their operation. since future communications may include sensitive operational details of this initiative i intend to increase security by moving to a blocked keyword cipher.
+
+the attached document was intercepted by our london agents when the party posted it on arrival at liverpool street. as usual it was addressed to tirpitzufer. primary analysis suggests that it does not use a substitution cipher.
+
+more to follow.
+
+pearl.

diff --git a/2020/plaintext.4b.txt b/2020/plaintext.4b.txt
new file mode 100644 (file)
index 0000000..51f55ce
--- /dev/null
+++ b/2020/plaintext.4b.txt
@@ -0,0 +1,18 @@
+dear uncle wilhelm our journey to london was short and charming the trains rattling along the
+british tracks through open fields and countryside their roads maybe almost impassable in places but
+these iron tracks provide a very usable way to travel across country at speed our hosts in london
+have organised for us to stay with members of the british scouting movement though i will spend the
+next few days with the orts gruppe exchanging news about the fatherland and their activities herein
+the capital city it was difficult to tear ourselves away from the pleasures of norfolk and the city
+offers far fewer opportunities for open exploration on the other hand the bustle of the city as in
+berlin offers a certain anonymity and it is in some ways relaxing to leave behind the gossip and
+inquisitiveness of village life as you suggested we will use our time herein london to support our
+brothers and sisters and to reassure them that they have not been forgotten i will pass on the gifts
+that you entrusted to me together with your instructions for their use as we did in caistor st
+edmund the radio you sent me is remarkable it is able to pickup signals across the spectrum even
+from berlin itself though i am enjoying listening to british radio especially the home service they
+are building a chain of transmitters across the country which provide information throughout the day
+and night something that i think you might want to take into account when planning our own radio
+services i am assuming that there are no changes planned for our itinerary but if there are then
+perhaps you could forward them to our hosts in spalding with a copy sent post restante to cse yours
+sincerely
\ No newline at end of file