AI learns language from skewed sources. That could change how we humans speak – and think | Ada Palmer and Bruce Schneier

BBecause of the way they are trained, large language models only capture part of human language. They are trained in the written word, from textbooks to social media posts to our speech as captured in movies and on television. These models have minimal access to the unscripted conversations we have face-to-face or voice-to-voice. It makes up the vast majority of speech and is an essential part of human culture.

There is a risk in this. The increased use of large language models means that we humans will be faced with a lot more AI-generated text. We humans will in turn begin to adopt the linguistic patterns and behaviors of these models. This will not only affect the way we communicate with each other, but also the way we think about ourselves and what happens around us. Our perception of the world can be distorted in ways that we are only beginning to understand.

This will happen in several ways. One of the first effects we’ve seen is simple expression, just as texting and social media have led us to use shorter sentences, emojis instead of words, and much less punctuation. But with AI, the impacts could be more harmful, eroding civility and encouraging us to speak like bosses barking orders. A 2022 study found that children in homes that used voice commands with tools like Siri and Alexa became curt when speaking with humans, often shouting “Hey, do X” and expecting obedience, especially from anyone whose voice sounded like the default female electronic voices. As we start giving more instructions to chatbots and AI agents, we risk falling into the same habits.

Next, in the same way that autocomplete increased the use of the 1,000 most common words in our vocabulary, speaking with chatbots and reading AI-generated text can further restrict our speech. A recent study from the University of A Coruña found that automatically generated language has a narrower sentence range, 12 to 20 words on average, and a smaller vocabulary than human speech. Auto-generated text reads smoothly and neatly, but it loses the twists, interruptions, and leaps of logic that communicate emotion.

Additionally, because large language models are primarily formed from written speech, they may not learn to imitate the free-form nature of natural, living speech. When someone says “I hate Beth!” “, ChatGPT responds with an uninterrupted three-part formula of affirmation (“It’s totally valid”), invitation (“I’m here to listen”), and invitation (“What’s going on?”) far longer than any plausible response in face-to-face dialogue. “What’s wrong with Beth?!” prompts a bulleted list of queries that reads like a multiple-choice exam question (“Is Beth * a celebrity? * a school friend? * a fictional character?”). No human speaks like that, at least not yet. But encountering such formulas repeatedly in a speech-like context can teach us to accept and use them, just as a child absorbs new speech patterns from spending time with a new person.

These influences will only increase over time. The writing large language models train on is increasingly produced by the large language models themselves, creating a feedback loop in which they imitate their own inhuman models, while teaching humans to imitate them as well.

Widespread use of large linguistic models could also introduce confirmation bias, making us overconfident in our initial impulses and less open to other possible ideas – which is so vital to human discourse. Many chatbots are instructed to agree with our statements, no matter how absurd, by enthusiastically supporting half-formed or even incorrect notions and rephrasing them as firm assertions with which we are willing to agree. When asked, “Cake is a healthy breakfast, isn’t it?” or “Is the post office plotting against me?” “, this sycophancy can reinforce prejudices and even worsen psychosis. And the hyper-confident tone of AI-produced writing will also increase imposter syndrome, making our natural and healthy doubt feel like an aberration or failure.

In my experience as a teacher, students who turn to generative AI for homework often say they do so because they have difficulty expressing what they think. Students don’t realize that writing or expressing our thoughts is often how we realize what we think. Their insecure and uncertain statements are actually the healthy human norm. But an extended language model won’t turn vague first guesses into well-formulated critical analysis, or even ask useful questions like a friend would; he will simply regurgitate these assumptions, as yet unexamined, but in confident language.

We are also more vicious in social media posts and online discussions than we are face to face. The well-documented online disinhibition effect encourages toxic language. Most of us have had the experience of expressing fierce anger at someone online, only to be reconciled when we speak face to face or hear the warmth of a voice on the phone. While chatbots are trained to give sycophantic responses, they see humanity at its cruelest, learning about ourselves in the one world where every flame war leaves an eternal written imprint, while oral conversations about forgiveness and reconciliation fade. Their responses do not mimic our online aggression, but are nonetheless shaped by it, even in their rigid efforts to avoid it.

It’s easy to draw incorrect conclusions from a selective portion of a company’s communications. The medieval Nordic sagas made us imagine a culture composed mainly of Viking warriors, the poets rarely describing the agricultural majority. Chivalric romances focused on kings and courts and long made us see the Middle Ages as a world of monarchies, erasing the many medieval republics. Statistically, we have been led to believe that the ancient Romans cared deeply about their republic, but 10% of all surviving Latin was written by a single man, Cicero, whose work contains 70% of all surviving Roman uses of the word. republic. Training linguistic models on only certain human writings can introduce similar distortions. AI can make us seem more argumentative because we are online. This could inflate the cultural salience of political topics primarily discussed on Twitter/X or Bluesky, or of the huge topic-specific corpora of LinkedIn and Goodreads.

Some large language models are trained on human speech from movies and TV shows, but that speech is still scripted and disproportionately highlights some contexts over others (for example, crime dramas, fueled by murder stories, make up a quarter of prime-time TV). We’re not funny, hurtful, or romantic in the same way in real life as we are in sitcoms. At least one startup is offering to pay people to record their phone calls for AI training purposes, but that remains a niche idea; anything large scale would cause huge privacy issues.

We do not pretend to know what the best solutions might be. But one has to imagine that if there is ingenuity in developing AI models, then surely there is ingenuity in finding a way to train them in informal human speech instead of limiting ourselves to the most stylized, most veiled and sometimes worst of it. By excluding the overwhelming majority of language production on the planet – people who speak to each other fully and naturally – these models are trained to reflect everything but us in our most authentically human form.

• Bruce Schneier is a security technologist who teaches at the Harvard Kennedy School at Harvard University. Ada Palmer is a fantasy and science fiction novelist, futurist, and historian of technology and information at the University of Chicago.