“Predictions sound like facts, but they are not.”
That line, repeated by Carissa Véliz, an AI researcher at Oxford, in her new work Prophecy: Prediction, Power, and the Fight for the Future, from Ancient Oracles to AI and interviews, names a quiet but profound shift in how our society decides: we are increasingly governed not by debates about values but by forecasts that present themselves as inevitabilities.
Tech CEOs, policy advocates, and funders frame futures as though they’re already written—“you’ll be using AI for everything tomorrow”—and because those forecasts wear the cloak of certainty, they begin to function as prescriptions. The result is predictable: power consolidates around those who produce the predictions, and democratic accountability frays.
This article weaves Véliz’s central arguments from Prophecy into a policy-minded narrative and practical roadmap. It explains why forecasts are political, how predictive systems narrow freedom and amplify bias, why “super intelligent AI” distractions matter less than everyday errors, and what democratic societies can do to reclaim agency.
Why predictions act like power
A forecast delivered with confidence performs more than an epistemic claim; it shapes expectations, mobilizes investment, and channels regulation. When a widely respected figure or a well-funded company declares a certain future “inevitable,” that declaration alters incentives and constrains the range of policy options and public imagination. Véliz calls this “obeying in advance”—an apt echo of Timothy Snyder’s concept that people can be induced to accept a path before it’s enacted.
“Tech leaders have financial incentives to present desirable futures as inevitable,” Véliz notes. Companies profit when users, regulators, and investors treat their preferred future as the only realistic path. The rhetorical power of prediction thus becomes an economic lever. Marketing, hype, and technical forecasting blur into one another; and the public, seeking certainty in anxious times, often confers undue authority on those who promise it.
The psychology and language of prognostication
Why do people accept predictions? Fear and uncertainty do much of the work. Asking “what will happen?” often masks a deeper plea: “tell me what to do.” Forecasts are attractive because they offer guidance in the face of contingency. Language amplifies the effect: predictive statements are frequently phrased like neutral descriptions, so our cognitive systems treat them as information rather than proposals.
“Predictions sound like facts, but they are not,” Véliz repeats. The rhetorical form of many forecasts disguises the fact that they are built on assumptions, selective data, and normative choices. Once a forecast takes hold, it becomes a self‑fulfilling instrument: institutions act on it, producing the very data that seem to confirm it.
Narrowing opportunities and reinforcing bias
One of the most immediate harms from predictive systems is the subtle but steady narrowing of human opportunity. Models trained on historical outcomes encourage institutions to “play it safe.” In hiring, that can mean preferring candidates resembling past “successful” employees—commonly advantaging white men. In lending and insurance, it can mean excluding groups with poor historical metrics, making some people effectively “uninsurable.”
Véliz explains how the mechanism works: biased decisions generate biased data, which then train future models to repeat those decisions. That feedback loop institutionalizes inequality and reduces social mobility. Predictions thus don’t just reflect past patterns; they amplify them.
The turkey problem: why past data misleads
Véliz invokes the classic “turkey problem” to illustrate how dangerous it is to draw conclusions solely from historical data. A turkey that is fed every day develops a blind trust in its safety. Ironically, on the day it is slaughtered, that trust reaches its consequential peak.
Similarly, models trained on historical regularities assume stability. When the world changes—because of climate shocks, structural economic shifts, or technological novelty—predictive systems can fail spectacularly.
“AI predictions come from past data, not future data,” Véliz warns. Overreliance on such predictions creates collective blind spots, especially when many actors use the same datasets and architectures.
Systemic fragility through synchronization
There’s another layer: if regulators, insurers, investors, and firms all rely on similar models, society risks synchronized errors. Véliz compares this to the 2008 financial crisis, where homogenous risk models produced a shared miscalculation with catastrophic outcomes. Today’s predictive monocultures can similarly generate systemic fragility: a single model failure or incorrect assumption can cascade across industries and institutions.
She cited a UK insurance supervisor’s warning that AI is already making some groups “uninsurable”—a case where model‑driven conservatism shifts risk onto individuals least able to bear it.
Everyday stupidity beats fantastical panic
Public discourse often fixates on apocalyptic visions of superintelligent AI. Véliz pushes back: those fears often function as marketing and status signaling for insiders. The far more immediate risk is mundane: models making stupid mistakes, producing false positives, and enabling opaque institutional moves that reduce accountability.
High-profile warnings—like Geoffrey Hinton’s resignation and subsequent alarmism—raise genuine ethical questions, but Véliz cautions we should examine motives and conflicts of interest. “Dramatic warnings from high-profile insiders can be self-serving,” she notes; they may amplify fear while leaving structural incentives intact.
Democracy is the real stake
Véliz’s core political insight is stark: predictive systems let institutions avoid accountability. It’s relatively straightforward to contest a factual error—if your loan was denied because your income was misrecorded, you can challenge the record. But how do you contest a predictive score that labels you “high risk”? Forecasts are often opaque, hard to inspect, and even harder to rebut.
“Predictions let companies and governments avoid accountability,” Véliz observes. That erosion of contestability threatens democratic norms. Democracies depend on transparency, public reasoning, and remedies; opaque prediction systems bypass these mechanisms and shift power to those who control data and models.
Medicine: promise, caution, rigour
Véliz acknowledges that predictive systems can produce genuine medical benefits—early detection of difficult cancers is a notable example. Yet history counsels caution. During COVID‑19, many AI tools were rushed into clinical practice without randomized trials; later reviews showed most to be ineffective and some harmful.
Predictive medicine demands the highest evidentiary bar: peer review, randomized controlled trials, long-term impact assessment, and tight protocols to avoid overdiagnosis and harm. “Promising clinical results exist, but history shows the need for skepticism and rigorous trials,” she says.
Practical rules for responsible prediction
Véliz doesn’t leave us only with diagnosis; she offers concrete rules:
- Predict things, not people. Aggregate forecasts (disease prevalence, demand surges, weather) pose fewer moral hazards than individualized risk scores.
- Prefer population‑level models over individualized profiling when human lives and livelihoods are at stake.
- Institute ethical guardrails: transparency requirements, independent audits, model cards, dataset disclosures, contestability mechanisms, and legal limits on predictive uses.
These are not mere technicalities; they are democratic safeguards that preserve human agency and contestability.
Why predictions go unquestioned
Two forces make predictive claims stubbornly resilient. First is human psychology: constant exposure normalizes predictions—they become invisible background assumptions. Second is structural interest: governments and corporations benefit when decisions are framed as following models because doing so reduces scrutiny and diffuses responsibility.
“Predictions are so common they become invisible, like water to a fish,” Véliz notes. That combination of cognitive blind spots plus institutional incentives makes meaningful critique difficult—unless citizens, journalists, and regulators push back.
Predictions as tools of empire
Véliz situates prediction in a history of informational domination. Scholars like Karen Hao have argued that the problem is not technology alone but the corporate and state systems that wield it: surveillance capitalism, militarization, and extractive corporate power. Véliz agrees: prediction is an enabling instrument for these systems.
“Predictions only have power if people believe them,” she says. Undermining unwarranted predictive claims thus weakens the institutions that depend on them.
The paradise future is a sales pitch
Technology firms frequently sell a vision of liberation: less work, universal basic income, frictionless optimization. Véliz is skeptical. Promises of a “paradise future” can pacify dissent and justify the transfer of judgment to centralized systems. Real-world trends—rising workloads, growing precarity—often contradict these rosy narratives.
Véliz makes the following policy recommendations: a one-page action plan:
- Ban or strictly regulate individualized predictive scores in high‑stakes domains (credit, insurance, hiring, criminal justice) unless systems are transparent, auditable, and contestable.
- Require model cards, dataset disclosure, and independent audits for models used in public policy or regulated industries.
- Mandate randomized controlled trials and peer review before clinical deployment of predictive medical tools.
- Enforce data minimization rules to limit large-scale personal data hoarding that powers intrusive prediction.
- Preserve human-in-the-loop decision-making with legal rights to human review and appeal of algorithmic decisions.
- Invest in public education on probability, uncertainty, and the limits of forecasting
- Support public or cooperative alternatives to corporate prediction monopolies—open datasets, civic forecasting platforms, and community-driven risk pools.
Forcing Forecasts Into the Light: A Politics of Skepticism, Transparency, and Contestability
We do not need to reject prediction wholesale—the practice of forecasting has undeniable uses. The task is to constrain how prediction is used in decisions that shape people’s lives, and to restore democratic contestability where forecasts now function as excuses for authority. As Véliz warns, “If we do not question predictive claims, we risk surrendering collective agency and the health of our democracies.” The antidote is a politics of skepticism, institutional design that insists on transparency and contestability, and a public ready to treat predictions as proposals to argue about—not decrees to obey.