Some musings on AI creative writing
Every so often someone (e.g. Scott Alexander here) posts an 'AI Art Turing Test' where people guess whether a picture was made by a human or by a Stable Diffusion-esque generative model. A little while ago fantasy author Mark Lawrence did the same1 for creative writing with GPT-5. The results seem to show that on average people get identify about half the pieces correctly, a.k.a they don't do any better than random chance2.
So does this mean human art is cooked? Not so, I would say.
These tests don't really prove that 'most people cannot tell AI and human made art apart' or 'people who say they hate AI art actually would like it if they didn't know its source' because the AI-generated art in these tests are implicitly human curated and usually generated with the knowledge that they will be used in a test like this, but the vast majority of AI content you see around the web (and real life) is made with no or minimal effort to distinguish itself from the now very over-familiar 'house styles' of the most popular models.
At best, they show a decontextualised selection of AI art curated by someone with decent taste can fool people into thinking it's passably human. Even if the capabilities of these models given careful prompting and selection are pretty high, the sheer ease of making something that sort of looks appropriate to the untrained eye means most of what you actually see is unapologetic slop. And it's perfectly fine to not want to see or interact with slop, and to make rules to keep them out of your spaces.
But what creative writing can AI models do with discerning prompting and curation? It seems like the best sources for figuring out about capabilities are random RPers on Reddit. I think these models can do somewhat more than what you would expect from the low-effort AI content polluting the space, and I am honestly pretty curious to see what they can produce3.
Found via the Thing of Things newsletter. TBH I'm with the author of that piece that the AI pieces are pretty bad but the human ones aren't much better.↩
Though notably people with knowledge and experience of both human and AI-made art do much better than chance.↩
Of course I won't post anything AI generated any place where people expecting human content could come across it.↩