Writing like a chatbot
I’ve done an audio recording of this text to make it a bit more accessible, and also because reading it out loud was a good way to proofread it. I know a lot of people don’t enjoy being given big long paragraphs to read so if you appreciate this please let me know and I’ll keep doing it when I write in future!
Max Read highlighted in a recent newsletter the contrast between OpenAI treating AI chatbots as first and foremost entertainment against many companies’ earnest attempts at asserting that this is real and extremely useful information technology. It’s an important thing to note, and something that I think will make understanding and predicting the role of these technologies in future more accurate and helpful. For context icymi Read is talking about OpenAI’s showcase of their latest GPT version, which can emulate sarcasm among other things, like repeatedly mention the stylish and modern industrial lighting. Read:
…many of OpenAI’s applications “feel like magic,” in the sense of a particularly amazing card trick. The first time you see it performed it’s absolutely astounding, but with every subsequent performance you become slightly more aware of the patter and the occlusions hiding the sleight of hand. Eventually the trick is explained to you; even if you yourself couldn’t pull it off, you can see its structure and its weaknesses… chatbots like GPT-4o may be incidentally informative or productive, but they are chiefly meant to be entertaining, hence the focus on spookily impressive but useless frippery like emotional affect. OpenAI’s insistence on pursuing A.I. that is, in Altman’s words, “like in the movies” is a smart marketing tactic, but it’s also the company meeting consumer demand… it seems to me that for OpenAI [the ‘actual’ use cases] are something like legitimizing or world-building supplements to the core product, which is the experience of talking with a computer.
Reading this piece felt timely; last night talking with some friends I began lamenting chatbots’ tendency to be gratuitously verbose for any useful tasks. Rarely do I feel like a chatbot – especially one of the big versions like from Google or OpenAI or Anthropic – is efficiently helping me in learning something but instead creates an experience akin to that egregious modern ritual of scanning through 300 words of long-winded fictional culinary narrative SEO optimisation to reach a recipe. Someone pointed out that there’s something really identifiable in ChatGPT’s writing; even without the linguistics expertise to be able to articulate its exact qualities, when something has been written by a chatbot you can feel it, emanating from the text and putting you at unease.
However this is a feeling that doesn’t necessarily come from AI-generated text, and it’s entirely possible that when you sense this in the text that in fact it has been written by a human. This is important to establish, because it suggests that rather than a reader superstitiously divining some ethereal machine-like quality in the text through an extra-sensory organic instinct, the reader is in fact simply noticing some bad writing. More specifically I’d say it’s bad writing dressed in good grammar, and this is something that can come from any source. So what makes this writing so poor? Obviously its fault isn’t its grammar or clarity, which are unfaltering, perfectly honed.
I suspect that it’s to do with precision, which is a function of the holistic purpose of the text, and a misunderstanding of why sometimes we write long things instead of short things. When a user wants an LLM to write something fairly subjective, say, a 500-word summary of their artistic practice1, it normally involves the user inputting a few highlights of information, which the AI links together, cushioning it in verbosity and grammatically linking each bit of data together. Anything substantial that it adds which wasn’t given as input is likely a vapid trope, a hallucination, or irrelevant. You end up with some long text output that contains more content, but roughly the same amount of accurate information.2
Communication is a pursuit of precision: the reader or listener is at the wide base of a cone, and the apex, the literal point, is what I’m trying to say. If I’m in possession of a wealth of data and my audience isn’t, the writing is an attempt to move the viewer from the broad end of the cone towards the point. Sentences, like the two preceding this one and the one following, might notionally repeat a concept, but each iteration hones the idea, clarifying and adding information. This can continue over and over virtually indefinitely, each iteration adding more precision, and hopefully by the end of the piece the audience is buried somewhere useful deep inside the cone.
I can summarise all of the information into a short abstract, which itself might not move the viewer very far through the cone. However by stripping out lots of the precision, it works to aim them towards the point; to position them to discover more. LLMs are often very good at this, which gets to the crux of the issue I think! This is true for both humans and AI: you can start with a large dataset and refine it to a summary, but if you start with only a summary and try to transform it into a large dataset, you end up with extremely imprecise, estimated values. Every sentence in which you iterate another repetition of the concept, it becomes no more precise than the previous iterations; you’re just rephrasing the exact same information.
Writing like this rapidly becomes tiring to read. Your mind’s performing triage on each phrase, trying to efficiently find new things to understand and disregard old data that it doesn’t need to waste energy processing. You’re trying to read while your brain is screaming at you to give up because it thinks there’s nothing here, and it’s right!
As I’ve mentioned before, I love Raymond Queneau’s Exercises in Style, yet I’ve never really been able to read much of it in a single sitting. Queneau retells a short two-paragraph story over and over through 99 different narrative styles, and despite each being fascinating it still numbs my mind receiving no new information each time around. In 2020’s underrated Palm Springs, it never really made sense to me that Andy Samberg’s character recedes into a sort of blissful apathy while trapped eternally in a time loop, a situation that I think would be extremely fatiguing. More reasonable is Cristin Miliotti’s character’s reaction, where she spends all her time studying, a situation that sounds almost ideal? In the absence of any information changing or developing around her, in my imagination I feel like your mind would be able to develop a hyper-focus on any new info you’d offer it. But that’s just my pipe dream because I’m tired of studying atm!
Concluding this, I think in the past then longer writing has carried the significance of important writing, simply by virtue of suggesting that the text might have deserved the labour involved in writing it. Now however, since the length of a text is distended from the labour that was involved in generating it, we can better define the purpose of the communication: a piece of text increasing in length can only aim to be useful if it is progressively gaining greater precision. If you give an AI 1,000 words of good information you can probably get it to return a clearly-written 300 word summary. If you give it 300 words of information, you can probably get your 300 words back with better grammar. If you only have 100 words of information to give it, you probably only have about 100 words to say and that’s good! Everyone can understand it really quickly!
