Glyph GPT |

𓉡𓀬𓉢

Balancing weights of Anubis...

How does it work?

Glyph GPT begins with nanoGPT, Andrej Karpathy's GPT architecture implementation, trained from scratch exclusively on hieroglyphic texts painstakingly prepared by Mattia De Cao. This generates streams of hieroglyphic sequences, which we then translate into English using Mattia's hiero-transformer - which itself is based on Facebook's M2M-100 translation model, fine-tuned on the new hieroglyphics dataset. Essentially a mini AI learns to babble Middle and Old Egyptian, and big AI does its best to translate that into English. Both models have some room for improvement, but what could go wrong? All the training/babbling/translating was done in Gardner notation which is more extensive than unicode hence you may occasionally see things like [Aa41] in the output because that glyph is not in Unicode.

Could King Tut read Glyph GPT?

Real Egyptian scribes would have been very impressed with Glyph GPT, or very horrified - possibly both! We hope that you are too. For some qualitative sense, check out NanoGPT's performance when trained on Shakespeare:
"And cowards it be strawn to my bed, And thrust the gates of my threats"
And if you can, read this ancient Hebrew produced by the same model architecture but trained on the Hebrew Bible. Results might be comparable because Hebrew and Egyptian were contemporaneous and hail from the same language family:
ועתה אמרת אלי בן אדם התקדש תתקדש הרבה לפניך
Both those NanoGPT outputs contain their fair share of weirdness, but we are many orders of magnitude better than a million monkeys on typewriters and saving a lot of bananas in the process.

Contact

jonathan@lansey.net

Thanks to Andrej Karpathy's for NanoGPT, and Mattia De Cao and team (Nicola De Cao, Angelo Colonna, Alessandro Lenci) for releasing the translation model and dataset.