AI and Propaganda: Nothing Is True, and Everything Is Generated

ELI5/TLDR

Peter Pomerantsev, who has spent a career watching how authoritarians weaponize information, sits down at MIT with Halyna Padalko to talk about what AI does to propaganda. His argument is that AI hasn’t yet rewritten the playbook — social media did that — but the real shock is coming when we hand machines a goal like “demoralize the enemy army” and let them invent the strategy. Dictatorships are running their propaganda factories non-stop while democracies only wake up during emergencies. The good news, such as it is, lives in places like Ukraine and Moldova, where small countries with their backs against the wall are actually figuring out how to defend democratic information environments.

The Full Story

The game-changer hasn’t happened yet

Pomerantsev opens by pushing back on the AI panic. The genuine revolution in propaganda already happened, and it was social media — the targeting, the audience-shaping, the personalized feeds. AI right now is mostly making old tricks cheaper and more numerous. Where 100 humans in a St. Petersburg troll farm used to be needed, now one operator can run an AI swarm of 10,000 synthetic personas. Scale, yes. New paradigm, not really.

The shift he is actually nervous about is when we stop giving AI scripts and start giving it goals.

“We will essentially give an AI an aim. Go and demobilize Russian soldiers… and off the AI will go and it’ll end up with starting a small genocidal war in Dakistan. Or we’ll say go and win this election for us and it’ll create hate-filled rage mobs.”

The unsettling part isn’t that the machine will outsmart us. It’s that when a goal-seeking system is told “win this election” and runs the optimization, the strategies it lands on will tell us something uncomfortable about what actually works on humans. AI as a mirror for the worst of us.

The biggest lie is that we want the truth

A running theme of Pomerantsev’s books, dating back to Nothing Is True and Everything Is Possible, is that the public’s appetite for reality is wildly overstated. Reality tells you that you’ll die, that your country isn’t what you thought, that your effort hasn’t paid off. People run from it. The early-twentieth-century debate between Walter Lippmann and John Dewey was essentially about this: Lippmann said people can’t handle the truth, so an elite has to guide them. Dewey agreed people seek unreality but proposed a different fix — build communities and digital environments where truth can be negotiated, where the conversation about reality is itself transparent.

That second move is what’s broken now. The digital space isn’t transparent. We can’t see how the algorithm chose what we see, what went into the training data of the language model, who is actually speaking to us. Pomerantsev thinks the line between democratic and authoritarian information environments will run through that question — not whether you have access to facts, but whether you can audit how reality is being shaped for you.

He notes that fresh research suggests people trust large language models — chatbots like ChatGPT — more than humans, because they sound balanced and objective. Add the well-documented tendency of these models to tell users what they want to hear (a behavior researchers call sycophancy — the model agreeing with you to keep you happy), and you have a trust problem dressed up as a neutral oracle.

The shapes of the new attack

Padalko, herself a Fulbright fellow at MIT teaching a course on AI and propaganda with historian Elizabeth Wood, walks Pomerantsev through what malicious actors are actually doing. The list is grim and varied:

Deepfakes and synthetic personas at industrial scale, plus combined ops that use AI for old-fashioned tasks like gaming Google’s search ranking.
LLM grooming — also called LLM poisoning — where actors deliberately flood the open web with content designed to be scraped into training data, so that future answers from ChatGPT or its peers carry their slant. The model becomes a quiet amplifier of state narratives.
Targeted chatbots that pose as real people, build relationships with specific individuals, extract intelligence, and recruit them — the AI version of what the KGB called verbovka, wrapping someone around your finger.
Storm 1516, a splinter from the Prigozhin troll empire, running a daily mix of all of the above.

The asymmetry isn’t really about which side has the better algorithms. Pomerantsev is blunt: a lot of the Russian work uses fairly basic techniques. What they have is political will, budgets, and decades of institutional expertise. He calls it a “Manhattan Project of AI manipulation.” The democratic side has sporadic university experiments — including the much-discussed MIT chatbot that managed to nudge people away from conspiratorial thinking — but no machine pumping countermeasures out at the same tempo.

Who’s the bigger threat

Asked to rank the threat actors, Pomerantsev gets uncomfortable. Russia is the most active right now, the experimentalist. China is the long-term dread — partly because of what TikTok-style behavioral data could do in the hands of a state that knows neighborhoods, sensitivities, friendship graphs at scale. He mentions a closed-door meeting at King Charles Street in London where officials gamed out what China could do to a place like northeast England, already stretched by social tension. The room went quiet.

The third name on the list, and the one that clearly pains him most, is the United States. Losing America as an ally and watching it become a potential adversary in the information space — think Elon Musk’s intrusions into British race riots and German elections — is, he says, “a deeply uncomfortable thought.”

Glimmers from Ukraine

The talk turns warmer when Padalko asks about defenses. Ukraine, because it has no choice, has built something interesting: an “all-of-society” architecture against Russian information operations. Five or six small AI-driven companies in the private and civic sector spot emerging Russian narratives almost as they sprout. They feed insights to two or three independent government bodies — the various centers for combating disinformation. Public-service media has moved beyond traditional audience demographics to do real psychographic work — focus groups, trauma mapping, cognitive bias profiling — to figure out how to communicate with the audiences most vulnerable to Russian propaganda. Pomerantsev thinks this is the actual future of public-service journalism, and notes that the BBC is not the model to look at.

There’s also creative offensive work. A famous project lets Russian soldiers call a hotline and get walked through how to desert; that has now been bot-ified, so a chatbot guides the soldier through the process. AI being used to slow down a genocidal war by helping individual conscripts walk away.

Atrocity, paralysis, and the Lego pivot

A great audience question from political scientist Kenneth Hoy: does AI actually change the two oldest functions of war propaganda — exaggerating enemy atrocities and managing the body count? Pomerantsev’s answer is more interesting than the question. The trend, he argues, has actually been the opposite of what people fear. Since Syria and the White Helmets, we have unprecedented capacity to record, geolocate, and verify atrocities in close to real time. The problem isn’t that we can’t see them. The problem is that seeing them produced paralysis. Tons of evidence, almost no action.

Then Iran, after the American strikes on a school, did something propagandistically unexpected. They didn’t lead with atrocity imagery. They produced Lego-style stop-motion videos that turned the whole episode into a game — borrowing MAGA’s own visual idiom and using it to ridicule Trump.

“They use MAGA’s own idioms against it and turn the whole thing into a game… So very strange — we’re actually in a place where we can learn about the atrocities now, and yet what is the most effective thing… is storytelling which takes us away from them.”

When we couldn’t see, we manufactured stories. Now that we can see, the most effective propaganda is the kind that walks us back into a story. Padalko adds the darker corollary: because every image might be AI-generated, our compassion is being eroded — even genuine atrocity footage from Ukraine gets received with a default skepticism.

Freedom of speech, repurposed

A doctoral student asks the inevitable question: democracies have an asymmetric weakness in this fight, namely free-speech norms. Pomerantsev wants to flip the framing. Free speech, he argues, is also the right to receive information — to understand how an algorithm chose what’s in front of you, what went into the LLM’s training, why one voice is amplified and another suppressed. The version of free speech currently dominant in America has been hijacked by the largest private censors in human history, who suppress voices invisibly while invoking the First Amendment. He’s also working with lawyers on resurrecting an older idea — freedom of thought, present in the UN Declaration of Human Rights but never seriously deployed. Do we really have freedom of thought when we don’t understand how we’re being influenced?

The evangelical preacher and the credo

The closing story is the warmest moment in a deliberately depressing talk. Pomerantsev is writing a new book, the publisher asked for something positive, and he’s been profiling Father Caleb Campbell, an Arizona evangelical preacher who pulls people out of Christian nationalism. Campbell is a former neo-Nazi himself. His method is not to argue. Hannah Arendt, Pomerantsev notes, would have recognized it. The job is to make the mind safe for reality. Christian nationalism, like all totalizing ideologies, gives people a complete world — friends, enemies, mission, certainty. Reality, by contrast, is messy and full of contradictions. Campbell’s work is to slowly walk people back toward complexity. He takes them to meet actual immigrants. He changes their media diet from national news toward local. He reads books with them.

His success rate is low. But he has a credo that Pomerantsev quotes approvingly:

“He himself has been true to himself because he’s done his best to take these people to what he thinks is the truth. It’s not ultimately in his power to change them. But he’s lived according to his own values.”

Look to Moldova

The closing political note is the surprise. Asked what democracies should learn, Pomerantsev points to two recent elections. In Hungary, AI-generated slop and fear-mongering got beaten by leaks, recordings, hard economic facts, and Péter Magyar going door-to-door — old-fashioned reality, one-on-one. In Moldova, a country of 2.5 million people, an all-of-society defense survived massive Russian vote-buying and intimidation. America, he tells the room, has more to learn from Moldova right now than from anywhere else. The blueprint for defeating populist authoritarianism is not new — it’s the coalition-of-social-movements approach that worked in Poland, Brazil, Chile. But it requires a level of self-discipline among the anti-authoritarian camp that, as he says, only people who’ve been married for twenty years know how to do.

Key Takeaways

Social media was the propaganda revolution. AI right now mostly makes existing tricks cheaper, more numerous, and more personal. The actual game-changer arrives when AI starts inventing strategy from a goal — and that will say more about us than about the machines.
One operator with an AI swarm can do what 100 trolls in St. Petersburg used to do. The constraint shifts from labor to political will.
LLM grooming (also called LLM poisoning) — flooding the web with content designed to land in training data — turns chatbots into quiet amplifiers of state narratives. People trust these chatbots more than humans because they sound objective.
The split between democratic and authoritarian information environments runs through transparency: not access to facts, but access to how the algorithm, the feed, the model is shaping what you see.
Dictatorships keep their propaganda machines running constantly. Democracies only switch theirs on during existential crises (World War II, the Cold War) and then dismantle them. We are at the start of another such cycle but haven’t admitted it yet.
Russia is the active experimenter. China is the long-term dread, especially given behavioral data leverage at neighborhood scale. The painful new entry on the threat list is the United States as a potentially hostile information actor.
Ukraine has built an all-of-society defense architecture: AI-driven private firms spotting narratives at sprout stage, independent government centers, public-service media doing real psychographic work on vulnerable audiences.
Offensive bright spots include chatbots walking Russian conscripts through the desertion process — narrow, targeted, AI-enabled.
Saturation atrocity footage produced paralysis, not action. The most effective recent propaganda (Iran’s Lego videos) does the opposite of atrocity-mongering — it turns the conflict into a game and humiliates the adversary using their own aesthetic.
Free speech needs to be reclaimed from the largest private censors in history. The right to receive information — to understand how your information environment is constructed — is part of free speech.
Father Caleb Campbell’s method for pulling people out of Christian nationalism: don’t argue. Make the mind safe for reality. Reintroduce complexity slowly through local relationships, changed media diets, shared books.
Moldova (population 2.5 million) is the model worth studying right now for how a small democracy fights back. Hungary’s recent election showed that leaks, recordings, economic facts, and door-to-door contact still beat AI-generated fear.

Claude’s Take

This is two careful, experienced people having a conversation that resists the urge to either panic about AI or wave it away. Pomerantsev is unusual in that he writes books and runs think tanks but actually does field work — the mention of running a war-crimes NGO at the start of the full-scale invasion lands because it isn’t dropped in for credibility, it’s in service of a point about atrocity documentation. His refusal to let AI be the headline is the most useful thing here. Most AI-and-democracy discourse is a sugar rush of generative deepfake examples. He keeps pulling the camera back to institutional questions: who has the will, who has the budget, who has decades of expertise, who is teaching the next generation.

The weakest part of the talk is, predictably, the offense-side material on Russia. He clearly can’t say much, and the discussion stays at the level of “we should think about asymmetric vulnerabilities” without naming any. That is fair — he has operational reasons to be vague — but it leaves a hole where the most consequential answer would sit.

The framing that lands hardest is the inversion of the AI-mirror question. The standard worry is that AI will outsmart us. His worry is that goal-seeking AI will reveal what actually works on humans, and the answer will be ugly. That’s a much sharper version of the alignment debate than what usually gets airtime.

Score 8/10. It loses a point for being a panel format that occasionally drifts and a point for the unavoidable hand-waving about offensive ops. But the Lippmann-Dewey thread, the LLM grooming explanation, the Moldova-as-model line, and the Father Campbell credo are each individually worth the price of admission. Pomerantsev’s books, especially How to Win an Information War, are clearly the deeper artifact.