Converting English text to Shavian script

To learn about the Shavian Alphabet, check out Shavian.info.

To transliterate English-language text files and web pages into Shavian script, I've written a C program, a Python program, a dictionary used by both, and a Firefox extension that transports you into a bizarro alternate universe where everyone already uses Shavian.

To use shaw.c, do:

bunzip2 dave.dict.bz2
gcc -o shaw shaw.c
shaw dave.dict

Now just type stuff on the screen and press Enter.

For shaw.py you'll need Python version 3.5 or later and the Natural Language Toolkit:

sudo apt install bunzip2 wget python3
wget dechifro.org/shavian/{shaw.py,dave.dict.bz2}
bunzip2 dave.dict.bz2
pip3 install --user nltk
python3
>>> import nltk
>>> nltk.download('averaged_perceptron_tagger')
>>> exit()

wget -k -O normal.html en.wikipedia.org/wiki/George_Bernard_Shaw
cat normal.html | uconv -x Latin-ASCII | python3 shaw.py dave.dict > shavian.html
firefox shavian.html &

The result should look like this. If some text appears in a low-quality bitmap font, it's because you don't have a Serif font for Shavian. Either go get one, or go to Preferences/Fonts/Advanced and set your Serif font to "Noto Sans Shavian".

Because shaw.py runs the entire text through NLTK's part-of-speech tagger in one piece, it outputs nothing until receiving end-of-file on input. You can do this from the keyboard by starting a new line with Ctrl-D, or Ctrl-Z in Windows. The benefit of NLTK is that part-of-speech tagging resolves most heteronyms, e.g. "lives" as a noun is "𐑤𐑲𐑝𐑟" while as a verb it is "𐑤𐑦𐑝𐑟". shaw.c can't do this because NLTK doesn't support C.

If "uconv -x Latin-ASCII" doesn't work, try "iconv -t ASCII//TRANSLIT", or "recode -f utf8..flat" instead. If you skip this step, accented words like "résumé" won't translate.

I haven't found a great solution for PDF files. The best I have so far is:

pdftohtml -stdout file.pdf | sed 's/&#160;/ /g' | grep -v '<hr/>' | sed 's@<br/>@@' > normal.html

Then edit normal.html and insert <p> on a line by itself wherever a paragraph break is called for.

On Android 7.0 or later, you can install Termux with Storage permission and run shaw.py there. The .bashrc file on my smartphone contains:

shave()
{
  wget -N dechifro.org/shavian/{shaw.py,dave.dict.bz2}
  [ dave.dict.bz2 -nt dave.dict ] && bunzip2 -kf dave.dict.bz2
  wget -k -O normal.html $1
  echo Converting to Shavian...
  uconv -x Latin-ASCII normal.html | python3 shaw.py dave.dict \
	> /storage/emulated/0/Documents/shavian.html
  echo All done!
}

I copy a URL, open Termux, type "shave ", paste the URL, press Enter, wait for it to finish, then open Documents/shavian.html.

The Firefox extension currently only works in Linux. Once you have shaw.py working, do this at the command line:

sudo bash
apt install xxd
cd /usr/local/share
wget dechifro.org/shavian/extension.tgz
tar xvf extension.tgz
cd shavian
mv shavian.json /usr/lib/mozilla/native-messaging-hosts
wget dechifro.org/shavian/{shaw.py,dave.dict.bz2}
bunzip2 dave.dict.bz2
exit
firefox about:debugging\#/runtime/this-firefox &

In Firefox, click "Load Temporary Add-on" and choose the file /usr/local/share/shavian/manifest.json. Every website you load will display normally for a second, then switch to Shavian. This continues until you either quit Firefox or right-click the "𐑖𐑷" button and choose "Remove Extension".

There are many ways to input Shavian text, but Gaspar Sinai's Yudit lets users enter non-Latin text without learning a new keyboard layout. Here's my keymap file. To install it, do:

mytool -convert my -write ~/.yudit/data/Shavian.my -mys Shavian.mys -rmys Shavian.mys

If you don't like my Latin-to-Shavian mapping, you can very easily change it.

When I come across missing words or spellings I don't like, I fix them in dave.dict and reload the page. You should keep your changes in a separate file so they don't get wiped out by new versions of dave.dict.

> cat addendum.dict 
Lilithferagar 𐑤𐑦𐑤𐑦𐑔𐑓𐑧𐑮𐑩𐑜𐑸
schedule 𐑖𐑧𐑡𐑵𐑤
erase 𐑦𐑮𐑱𐑟
shone 𐑖𐑪𐑯
> echo The schedule for Lilithferagar was shone to be wrong, and must be erased. | python3 shaw.py dave.dict 
 𐑞 𐑕𐑒𐑧𐑡𐑵𐑤 𐑓 Lilithferagar 𐑢𐑪𐑟 𐑖𐑴𐑯 𐑑 𐑚𐑰 𐑮𐑪𐑙, 𐑯 𐑥𐑳𐑕𐑑 𐑚𐑰 𐑦𐑮𐑱𐑕𐑑. 
> echo The schedule for Lilithferagar was shone to be wrong, and must be erased. | python3 shaw.py dave.dict addendum.dict 
 𐑞 𐑖𐑧𐑡𐑵𐑤 𐑓 ·𐑤𐑦𐑤𐑦𐑔𐑓𐑧𐑮𐑩𐑜𐑸 𐑢𐑪𐑟 𐑖𐑪𐑯 𐑑 𐑚𐑰 𐑮𐑪𐑙, 𐑯 𐑥𐑳𐑕𐑑 𐑚𐑰 𐑦𐑮𐑱𐑟𐑛.

This feature is not available in shaw.c.

To display Shavian text in Xterm, I designed some free bitmap glyphs. Here's a specimen in 6x13 fixed, which has been X11's default font since the 1980s because of its compactness and superior readability:

Xterm defaults to black-on-white, but I much prefer white on dark slate gray. The same text in your web browser's default font looks like this:

𐑞 𐑿𐑯𐑨𐑯𐑦𐑥𐑩𐑕 𐑛𐑧𐑒𐑤𐑼𐑱𐑖𐑩𐑯 𐑝 𐑞 𐑔𐑻𐑑𐑰𐑯 𐑿𐑯𐑲𐑑𐑩𐑛 𐑕𐑑𐑱𐑑𐑕 𐑝 ·𐑩𐑥𐑧𐑮𐑦𐑒𐑩, 𐑢𐑧𐑯 𐑦𐑯 𐑞 𐑒𐑹𐑕 𐑝 𐑣𐑿𐑥𐑩𐑯 𐑦𐑝𐑧𐑯𐑑𐑕, 𐑦𐑑 𐑚𐑦𐑒𐑳𐑥𐑟 𐑯𐑧𐑕𐑩𐑕𐑺𐑦 𐑓 𐑢𐑳𐑯 𐑐𐑰𐑐𐑩𐑤 𐑑 𐑛𐑦𐑟𐑪𐑤𐑝 𐑞 𐑐𐑩𐑤𐑦𐑑𐑦𐑒𐑩𐑤 𐑚𐑨𐑯𐑛𐑟 𐑢𐑦𐑗 𐑣𐑨𐑝 𐑒𐑩𐑯𐑧𐑒𐑑𐑩𐑛 𐑞𐑧𐑥 𐑢𐑦𐑞 𐑩𐑯𐑳𐑞𐑼, 𐑯 𐑑 𐑩𐑕𐑿𐑥 𐑩𐑥𐑳𐑙 𐑞 𐑐𐑬𐑼𐑟 𐑝 𐑞 𐑻𐑔, 𐑞 𐑕𐑧𐑐𐑼𐑩𐑑 𐑯 𐑰𐑒𐑢𐑩𐑤 𐑕𐑑𐑱𐑖𐑩𐑯 𐑑 𐑢𐑦𐑗 𐑞 𐑤𐑷𐑟 𐑝 𐑯𐑱𐑗𐑼 𐑯 𐑝 𐑯𐑱𐑗𐑼'𐑟 𐑜𐑪𐑛 𐑦𐑯𐑑𐑲𐑑𐑩𐑤 𐑞𐑧𐑥, 𐑩 𐑛𐑰𐑕𐑩𐑯𐑑 𐑮𐑦𐑕𐑐𐑧𐑒𐑑 𐑑 𐑞 𐑩𐑐𐑦𐑯𐑘𐑩𐑯𐑟 𐑝 𐑥𐑨𐑯𐑒𐑲𐑯𐑛 𐑮𐑦𐑒𐑢𐑲𐑼𐑟 𐑞𐑨𐑑 𐑞𐑱 𐑖𐑫𐑛 𐑛𐑦𐑒𐑤𐑺 𐑞 𐑒𐑷𐑟𐑩𐑟 𐑢𐑦𐑗 𐑦𐑥𐑐𐑧𐑤 𐑞𐑧𐑥 𐑑 𐑞 𐑕𐑧𐑐𐑼𐑱𐑖𐑩𐑯.

𐑢𐑰 𐑣𐑴𐑤𐑛 𐑞𐑰𐑟 𐑑𐑮𐑵𐑞𐑟 𐑑 𐑚𐑰 𐑕𐑧𐑤𐑓-𐑧𐑝𐑦𐑛𐑩𐑯𐑑, 𐑞𐑨𐑑 𐑷𐑤 𐑥𐑧𐑯 𐑸 𐑒𐑮𐑦𐑱𐑑𐑩𐑛 𐑰𐑒𐑢𐑩𐑤, 𐑞𐑨𐑑 𐑞𐑱 𐑸 𐑦𐑯𐑛𐑬𐑛 𐑚𐑲 𐑞𐑺 𐑒𐑮𐑦𐑱𐑑𐑼 𐑢𐑦𐑞 𐑕𐑻𐑑𐑩𐑯 𐑳𐑯𐑱𐑤𐑾𐑯𐑩𐑚𐑩𐑤 𐑮𐑲𐑑𐑕, 𐑞𐑨𐑑 𐑩𐑥𐑳𐑙 𐑞𐑰𐑟 𐑸 𐑤𐑲𐑓, 𐑤𐑦𐑚𐑼𐑑𐑦 𐑯 𐑞 𐑐𐑼𐑕𐑿𐑑 𐑝 𐑣𐑨𐑐𐑦𐑯𐑩𐑕.--𐑞𐑨𐑑 𐑑 𐑕𐑦𐑒𐑘𐑫𐑼 𐑞𐑰𐑟 𐑮𐑲𐑑𐑕, 𐑜𐑳𐑝𐑼𐑯𐑥𐑩𐑯𐑑𐑕 𐑸 𐑦𐑯𐑕𐑑𐑦𐑑𐑿𐑑𐑩𐑛 𐑩𐑥𐑳𐑙 𐑥𐑧𐑯, 𐑛𐑦𐑮𐑲𐑝𐑦𐑙 𐑞𐑺 𐑡𐑳𐑕𐑑 𐑐𐑬𐑼𐑟 𐑓𐑮𐑪𐑥 𐑞 𐑒𐑩𐑯𐑕𐑧𐑯𐑑 𐑝 𐑞 𐑜𐑳𐑝𐑼𐑯𐑛, --𐑞𐑨𐑑 𐑢𐑧𐑯𐑧𐑝𐑼 𐑧𐑯𐑦 𐑓𐑹𐑥 𐑝 𐑜𐑳𐑝𐑼𐑯𐑥𐑩𐑯𐑑 𐑚𐑦𐑒𐑳𐑥𐑟 𐑛𐑦𐑕𐑑𐑮𐑳𐑒𐑑𐑦𐑝 𐑝 𐑞𐑰𐑟 𐑧𐑯𐑛𐑟, 𐑦𐑑 𐑦𐑟 𐑞 𐑮𐑲𐑑 𐑝 𐑞 𐑐𐑰𐑐𐑩𐑤 𐑑 𐑷𐑤𐑑𐑼 𐑹 𐑑 𐑩𐑚𐑪𐑤𐑦𐑖 𐑦𐑑, 𐑯 𐑑 𐑦𐑯𐑕𐑑𐑦𐑑𐑿𐑑 𐑯𐑿 𐑜𐑳𐑝𐑼𐑯𐑥𐑩𐑯𐑑, 𐑤𐑱𐑦𐑙 𐑦𐑑𐑕 𐑓𐑬𐑯𐑛𐑱𐑖𐑩𐑯 𐑪𐑯 𐑕𐑳𐑗 𐑐𐑮𐑦𐑯𐑕𐑩𐑐𐑩𐑤𐑟 𐑯 𐑹𐑜𐑩𐑯𐑲𐑟𐑦𐑙 𐑦𐑑𐑕 𐑐𐑬𐑼𐑟 𐑦𐑯 𐑕𐑳𐑗 𐑓𐑹𐑥, 𐑨𐑟 𐑑 𐑞𐑧𐑥 𐑖𐑨𐑤 𐑕𐑰𐑥 𐑥𐑴𐑕𐑑 𐑤𐑲𐑒𐑤𐑦 𐑑 𐑦𐑓𐑧𐑒𐑑 𐑞𐑺 𐑕𐑱𐑓𐑑𐑦 𐑯 𐑣𐑨𐑐𐑦𐑯𐑩𐑕. 𐑐𐑮𐑵𐑛𐑩𐑯𐑕, 𐑦𐑯𐑛𐑰𐑛, 𐑢𐑦𐑤 𐑛𐑦𐑒𐑑𐑱𐑑 𐑞𐑨𐑑 𐑜𐑳𐑝𐑼𐑯𐑥𐑩𐑯𐑑𐑕 𐑤𐑪𐑙 𐑦𐑕𐑑𐑨𐑚𐑤𐑦𐑖𐑑 𐑖𐑫𐑛 𐑯𐑪𐑑 𐑚𐑰 𐑗𐑱𐑯𐑡𐑛 𐑓 𐑤𐑲𐑑 𐑯 𐑑𐑮𐑨𐑯𐑟𐑾𐑯𐑑 𐑒𐑷𐑟𐑩𐑟; 𐑯 𐑩𐑒𐑹𐑛𐑦𐑙𐑤𐑦 𐑷𐑤 𐑦𐑒𐑕𐑐𐑽𐑾𐑯𐑕 𐑣𐑨𐑔 𐑖𐑴𐑯, 𐑞𐑨𐑑 𐑥𐑨𐑯𐑒𐑲𐑯𐑛 𐑸 𐑥𐑹 𐑛𐑦𐑕𐑐𐑴𐑟𐑛 𐑑 𐑕𐑳𐑓𐑼, 𐑢𐑲𐑤 𐑰𐑝𐑩𐑤𐑟 𐑸 𐑕𐑳𐑓𐑼𐑩𐑚𐑩𐑤, 𐑞𐑨𐑯 𐑑 𐑮𐑲𐑑 𐑞𐑧𐑥𐑕𐑧𐑤𐑝𐑟 𐑚𐑲 𐑩𐑚𐑪𐑤𐑦𐑖𐑦𐑙 𐑞 𐑓𐑹𐑥𐑟 𐑑 𐑢𐑦𐑗 𐑞𐑱 𐑸 𐑩𐑒𐑳𐑕𐑑𐑩𐑥𐑛. 𐑚𐑳𐑑 𐑢𐑧𐑯 𐑩 𐑤𐑪𐑙 𐑑𐑮𐑱𐑯 𐑝 𐑩𐑚𐑿𐑕𐑩𐑟 𐑯 𐑿𐑟𐑻𐑐𐑱𐑖𐑩𐑯𐑟, 𐑐𐑼𐑕𐑿𐑦𐑙 𐑦𐑯𐑝𐑺𐑾𐑚𐑤𐑦 𐑞 𐑕𐑱𐑥 𐑪𐑚𐑡𐑧𐑒𐑑 𐑦𐑝𐑦𐑯𐑕𐑩𐑟 𐑩 𐑛𐑦𐑟𐑲𐑯 𐑑 𐑮𐑦𐑛𐑿𐑕 𐑞𐑧𐑥 𐑳𐑯𐑛𐑼 𐑨𐑚𐑕𐑩𐑤𐑵𐑑 𐑛𐑧𐑕𐑐𐑩𐑑𐑦𐑟𐑩𐑥, 𐑦𐑑 𐑦𐑟 𐑞𐑺 𐑮𐑲𐑑, 𐑦𐑑 𐑦𐑟 𐑞𐑺 𐑛𐑿𐑑𐑦, 𐑑 𐑔𐑮𐑴 𐑪𐑓 𐑕𐑳𐑗 𐑜𐑳𐑝𐑼𐑯𐑥𐑩𐑯𐑑, 𐑯 𐑑 𐑐𐑮𐑩𐑝𐑲𐑛 𐑯𐑿 𐑜𐑸𐑛𐑟 𐑓 𐑞𐑺 𐑓𐑿𐑗𐑼 𐑕𐑦𐑒𐑘𐑫𐑼𐑦𐑑𐑦.--𐑕𐑳𐑗 𐑣𐑨𐑟 𐑚𐑰𐑯 𐑞 𐑐𐑱𐑖𐑩𐑯𐑑 𐑕𐑳𐑓𐑼𐑩𐑯𐑕 𐑝 𐑞𐑰𐑟 𐑒𐑪𐑤𐑩𐑯𐑦𐑟; 𐑯 𐑕𐑳𐑗 𐑦𐑟 𐑯𐑬 𐑞 𐑯𐑦𐑕𐑧𐑕𐑦𐑑𐑦 𐑢𐑦𐑗 𐑒𐑩𐑯𐑕𐑑𐑮𐑱𐑯𐑟 𐑞𐑧𐑥 𐑑 𐑷𐑤𐑑𐑼 𐑞𐑺 𐑓𐑹𐑥𐑼 𐑕𐑦𐑕𐑑𐑩𐑥𐑟 𐑝 𐑜𐑳𐑝𐑼𐑯𐑥𐑩𐑯𐑑. 𐑞 𐑣𐑦𐑕𐑑𐑼𐑦 𐑝 𐑞 𐑐𐑮𐑧𐑟𐑩𐑯𐑑 𐑒𐑦𐑙 𐑝 𐑜𐑮𐑱𐑑 ·𐑚𐑮𐑦𐑑𐑩𐑯 𐑦𐑟 𐑩 𐑣𐑦𐑕𐑑𐑼𐑦 𐑝 𐑮𐑦𐑐𐑰𐑑𐑩𐑛 𐑦𐑯𐑡𐑼𐑦𐑟 𐑯 𐑿𐑟𐑻𐑐𐑱𐑖𐑩𐑯𐑟, 𐑷𐑤 𐑣𐑨𐑝𐑦𐑙 𐑦𐑯 𐑛𐑦𐑮𐑧𐑒𐑑 𐑪𐑚𐑡𐑧𐑒𐑑 𐑞 𐑦𐑕𐑑𐑨𐑚𐑤𐑦𐑖𐑥𐑩𐑯𐑑 𐑝 𐑩𐑯 𐑨𐑚𐑕𐑩𐑤𐑵𐑑 𐑑𐑦𐑮𐑩𐑯𐑦 𐑴𐑝𐑼 𐑞𐑰𐑟 𐑕𐑑𐑱𐑑𐑕. 𐑑 𐑐𐑮𐑵𐑝 𐑞𐑦𐑕, 𐑤𐑧𐑑 𐑓𐑨𐑒𐑑𐑕 𐑚𐑰 𐑕𐑩𐑚𐑥𐑦𐑑𐑩𐑛 𐑑 𐑩 𐑒𐑨𐑯𐑛𐑦𐑛 𐑢𐑻𐑤𐑛.