On Automated Philosophy
This text, initially written in the summer of 2021, is part of a lecture series about language and AI at ToftH School. The original quotes can be found in the German version.
There’s a slightly yellowed book sitting on my desk. The cover is square and kept entirely in lowercase letters. It reads: “max bense theorie der texte [theory of texts] kiepenheuer & witsch”. Almost sixty years after its publication, I’m looking at the screen in front of me and read the words “OpenAI Beta Playground”.
The company OpenAI offers an experimental service to interact with GPT-3, their much discussed speech-based artificial intelligence. Given an arbitrary text, GPT-3 can predict which tokens are most likely to follow next. The results are surprisingly coherent sentences, which can even take more abstract context into account.
The technology is based on the Transformer architecture for artificial neural networks developed at Google a few years ago. What makes GPT-3 special is the extraordinary amount of training data. This AI has read virtually the whole Internet and, through this, independently learned to credibly imitate natural language. Here, language is to be understood as a generalized concept. The text can be composed in English or German, or even in a programming…
