The probability of what, exactly?
p(doom), in plain English.
The p stands for probability. In AI debates, p(doom) is shorthand for someone's estimated chance of a catastrophic outcome from advanced AI. Depending on the speaker, that might mean human extinction, permanent loss of human control or another severe outcome. The time horizon and assumptions vary too.
A personal estimate is a judgement under uncertainty, not a measured failure rate or a single scientific consensus. Before comparing numbers, ask: which outcome, by when, under which development path, with what safeguards? What evidence would change the estimate?
Katja Grace and colleagues' survey of 2,778 AI researchers, first published in 2024 and revised in 2025, found wide uncertainty and disagreement about the best pace of progress. It records researchers' forecasts; it does not measure the planet's odds of survival.
p(doom) / p(protopia) / p(fair go) / p(Joyful Responsible Abundance)
Surviving is a start.
The other three p(...) labels turn the same question towards futures worth building. They are prompts for comparing choices, not established numerical metrics. The outcomes overlap: fewer catastrophic risks should go alongside wider opportunity and a better everyday life.
p(doom)
What could go catastrophically wrong? Consider loss of control, deliberate misuse and concentration of power. Look for independent tests, reported incidents, working limits on what systems may do, and evidence that somebody can stop a dangerous deployment.
Ask who carries the risk when a builder gets the upside.
p(protopia)
Are things getting better, with room to change course? Protopia means ongoing improvement, with new problems and trade-offs to work through, rather than a perfect final society. Look for useful trials, published failures, repair and choices that stay open.
Ask whether the next step leaves people more capable than the last.
p(fair go)
Who gets opportunity, benefit and a say? A fair go means meaningful access and voice, including for people outside the big labs, wealthy cities and comfortable jobs. Compare who owns the tools, gets paid, can challenge a decision and can leave a provider.
Ask whether rural, disabled, low-income and First Nations participants help define success.
p(Joyful Responsible Abundance)
Is the extra capability making life worth living? Joyful means time and room for lives people value; responsible means answerable for consequences; abundance means enough useful resources and opportunities to share. Hold the three together.
Look for time returned, needs met, harms repaired and benefits reaching others. More engagement or a higher company valuation is a different result.
Primary statements / checked 4 October 2026
The warning is coming from the workshop.
2023: the extinction-risk statement
The Center for AI Safety statement calls for AI extinction risk to be treated as a global priority alongside pandemics and nuclear war. Its signatories include Sam Altman, Demis Hassabis, Dario Amodei, Geoffrey Hinton and Yoshua Bengio. It gives no agreed percentage or deadline.
2 April 2025: DeepMind's risk categories
Anca Dragan, Rohin Shah, Four Flynn and Shane Legg set out misuse, misalignment, accidents and structural risks. Their paper accompanies continued AGI development and proposes technical safeguards. It is an approach to managing risks, not a claim that testing has eliminated them.
January 2026: power and control
In The Adolescence of Technology, Dario Amodei discusses autonomous-system risks, destructive misuse, authoritarian power and economic disruption. He argues against treating catastrophe as inevitable while taking severe risks seriously. These are his analysis and forecasts.
September 2026: a call to pace the frontier
Amodei calls for slower capability advancement, embedded third-party evaluators and wider coordination. He describes Anthropic's commitment to the evaluator step and argues for continued development with more time for safeguards. That is a published commitment to examine against what happens next.
If the people building the engine warn that it might take the wheels off civilisation, the public deserves more than a seatbelt slogan. Builders argue that benefits and strategic competition favour continuing. The question is whether commercial incentives, independent scrutiny and enforceable limits actually match the scale of their warnings.
Superintelligence / very human power
A clever machine is not a mandate.
Superintelligence means AI surpassing human cognitive capabilities on a broad scale. In The Gentle Singularity, Sam Altman forecasts digital superintelligence and abundant intelligence and energy. It is a forecast about technology and society, not a declaration that a deity has arrived.
“AI gods” is a useful provocation: are we handing something enormous power because it sounds wise? Who chooses its goals, owns its infrastructure and decides whose values count? A machine might outperform us at many tasks without earning the right to choose our future. And a company should not acquire that right by owning the machine.
Australian choices / proposals to compare
What changes the incentives?
Keep developing, with outside scrutinyRequire independent evaluations, incident reporting and named responsibility for deployment. This preserves useful experimentation, but tests may miss new behaviour and reviewers need real access and independence.
Hold high-risk capability at checkpointsMake further development or deployment conditional on defined safety evidence. This buys time and creates consequences, but thresholds are contested, verification is difficult and rules should not simply protect the largest incumbents.
Build public and cooperative capabilityInvest in shared infrastructure, skills and useful bounded applications, with local control and an exit from providers. This spreads agency and benefits, but needs paid operators, maintenance and honest evaluation; local ownership alone does not solve frontier-model risk.
These paths address different problems and can be combined. Australian procurement, research funding and infrastructure agreements should publish who benefits, who bears failures, what independent review occurs and who has authority to stop or change the work. Faster capability is not the same measure as a fairer outcome.
From an outcome question to useful work
Put a fair go to work.
GAJRA is the Global Association for Joyful Responsible Abundance on Earth. Its existing public project asks how AI alignment and the way people live and work should fit together. Its open questions invite contributions about care, joy, accountability and whose abundance is being created. Start with one question and bring an example that others can examine.
The missing middle proposal puts serious computing under towns, libraries, universities, hospitals and cooperatives. Ready S.E.T. develops the people and permissions around a proposed node: paid operations, learning, equipment access and locally chosen workloads. Its browser-based brief builder already lets a group write and download its own plan; the proposed hardware is a separate step.
For a library, school or cooperative, compare a cloud service with a shared local option. Record total cost, people and training, custody of information, benefits, failure response and how to leave. Publish the result even if the smaller or simpler option wins.
Ask for the whole bet.
Take one AI proposal to your representative, workplace, university or council. Ask what it is trying to improve, which failures matter, who has a say and what evidence would change the decision. Set a review date and keep the answer beside the result.
For each of the four outcome questions, write one hoped-for benefit, one risk, one observable sign and the person or body responsible for responding. No invented odds required: just a better account of the choices being made in our name.
