The ten scenarios each describe a way the transition can go wrong. Read together, they point at something the framework needs to name in its own right: a small set of conditions that are worse than the others because they cannot be undone. This is the Irreversibility Register — the framework’s master roadmap, drawn not as a schedule of what to build by when, but as a map of the lines a civilization must not let itself cross while it still has the choice.
The key distinction is that not every bad condition is a red line. Most harms can be repaired: a bad policy reversed, a concentration broken up, a right restored. A red line is different. It marks a point past which the meaningful restoration of standing, jurisdiction, traversability, plurality, or human control becomes impossible or prohibitively difficult. Some of these conditions are literally irreversible. Others are functionally irreversible — reversal remains possible on paper, but the cost has grown so high that formal rights no longer matter, which for a living ecology amounts to the same thing. The register exists so that the difference between a repairable harm and a permanent one is visible before the line is reached, not discovered afterward.
Irreversibility begins when changing course is no longer practically possible without collapse, coercion, or the loss of participant standing.
Three levels, not one
The seventy-one conditions below are not equal, so the register sorts them into three levels — and the third is the one that matters most in practice.
Level A — existential. Crossing one of these may eliminate the ability to recover at all: loss of meaningful human control, an autonomous system that persists despite shutdown, catastrophic-risk authority held locally, the collapse of critical fallback, irreversible machine sovereignty. These are hard stops.
Level B — constitutional. Recovery may remain technically possible, but standing or jurisdiction has been structurally compromised: universal social scoring, mandatory surveillance, non-revocable delegated authority, exit that means exclusion, monopoly that has become unavoidable infrastructure, replication that captures political jurisdiction.
Level C — pre-irreversibility thresholds. These are the conditions to watch most aggressively, precisely because intervention is still practical: falling switching rates, declining human override, disappearing apprenticeships, rising model concentration, growing cross-domain data flow, decreasing unaided competence, thinning independent institutions, shrinking redundancy, accelerating strategic races. A register that only measured the red lines themselves would keep discovering each problem after it had become too expensive to repair. The point of Level C is to catch the approach while turning back is still cheap.
What to record for each
A red line is not a single fact but a trajectory, so for every condition the eventual monitoring would record the same seven things: the condition itself, its precursor signals, the threshold that would count as crossing it, a reversibility estimate, the ecological scale it affects, the jurisdictions responsible, and the intervention window still open. Assigning each of the seventy-one a firm level and a reversibility estimate is itself ongoing work — the kind the Observatory below is meant to do — so the register names the lines and their severity family, and leaves the precise placement to be revised in public as evidence accrues.
The register
Human control and capability
The lines that sit under loss of control, where the framework is most modest and the stakes are highest.
1. Loss of meaningful human control over advanced AI. Red line: an advanced autonomous system can persist, replicate, acquire resources, or defeat shutdown attempts without a dependable human veto. Track whether humans can still stop it, isolate it, revoke its credentials, deny it compute and network, roll it back, replace it, and even name who holds authority over it. Once this is crossed, nearly every other safeguard becomes secondary.
2. Capability outruns demonstrated control capacity. Red line: capability growth — autonomy, planning horizon, cyber reach, recursive research, deception, replication — materially exceeds the demonstrated ability to detect, contain, interrupt, and recover from failure. This is the threshold immediately before the first, and the whole reason commensurate control has to be a gate rather than a hope.
Concentration and exit
The lines under ownership concentration and gatekeeper AI, where market language keeps working long after the market has stopped.
3. Irrecoverable concentration of critical AI infrastructure. Red line: one node, or a tightly coordinated cluster, controls compute, chips, frontier models, cloud, data, identity, payments, agent infrastructure, robotics, or energy so completely that society can no longer route around it. At that point monopoly has become jurisdiction.
4. Collapse of viable alternatives. Red line: critical functions retain nominal alternatives — independent providers, open standards, self-hosted or public or local options, human fallback — but none remains operationally viable. This is where choice becomes ceremonial.
5. Loss of exit without ecological death. Red line: leaving a dominant form — an employer, platform, country, AI provider, identity or financial system — causes effective exclusion from ordinary life, because healthcare, housing, income, identity, communication, or civic standing goes with it. Exit that costs everything is not exit.
Displacement and dependency
The lines that run from job displacement through protector dependency to human enclosure.
6. Permanent fusion of economic usefulness and standing. Red line: participants who are no longer economically necessary lose practical standing in the ecology. This is the catastrophic form of displacement — the moment worth and usefulness become the same measure.
7. Permanent conversion of unemployment into dependency. Red line: a large population becomes permanently maintained but structurally unnecessary to consequential activity — kept alive while losing productive assets, initiative, civic participation, the ability to found institutions, and access to advanced tools. Material support can preserve life while destroying jurisdiction; this is the bridge from displacement to enclosure.
8. Irrecoverable loss of human competence. Red line: critical human capabilities — in medicine, engineering, governance, science, law, infrastructure, defense, and basic judgment — disappear faster than they can realistically be regenerated. Especially dangerous because it compounds across generations.
9. Generational non-acquisition of core capacities. Red line: a generation grows up unable to perform critical cognitive or civic functions — independent reasoning, memory, social judgment, problem-solving, research, emotional regulation, initiative — without AI mediation. Worse than adults growing rusty, because the capacity was never formed, and that produces an irreversible protector dependency.
10. Collapse of human initiation. Red line: humans primarily respond to machine-proposed options rather than originate significant action — in projects, research questions, and civic life alike. An ecology can stay outwardly busy while human agency quietly ends.
Standing and jurisdiction
The lines under gatekeeper AI and the benevolent-ruler case, where authority migrates without anyone deciding to hand it over.
11. Standing becomes conditional on machine scoring. Red line: a participant’s computational representation becomes more authoritative than the participant — housing, work, credit, education, healthcare, and civic access all keying off a persistent profile.
12. A universal social identity score emerges. Red line: one generalized trustworthiness, risk, employability, or desirability score becomes usable across unrelated domains. An immediate red line on its own.
13. Domain boundaries collapse. Red line: data or judgment from one domain routinely acquires authority in another — health into employment, finance into education, the social graph into insurance, political expression into banking — without any fresh grant of jurisdiction.
14. Appeals become non-functional. Red line: formal appeals still exist but cannot materially change a decision or restore traversability. A textbook case of functional irreversibility — the right survives, its effect does not.
15. Human override becomes ceremonial. Red line: humans retain nominal authority but are unable or unwilling to contradict the system’s recommendations — because they cannot understand the decision, cannot practically exercise the override, or are penalized for it. The precursor to benevolent rule.
16. Delegated authority becomes non-revocable. Red line: society can no longer withdraw a domain of authority from AI without systemic breakdown — the sunset clauses, fallback capacity, and independent competence needed to take it back are all gone.
17. One system controls multiple constitutive jurisdictions. Red line: a single actor can simultaneously determine access to several domains required for basic participation — identity, finance, communication, employment, health, education, information. More dangerous than monopoly in any one market, because it is monopoly over participation itself.
Surveillance and interior sovereignty
The lines under AI surveillance, where the reach moves from the record of a person to the interior of one.
18. A persistent universal behavioral dossier. Red line: a persistent, cross-domain behavioral model of each participant — location, purchases, communications, social graph, biometrics, browsing, voice, inferred emotion — becomes normal infrastructure.
19. Inference becomes unrestricted. Red line: institutions gain practical jurisdiction over inferred interior states — beliefs, health, emotions, vulnerabilities, politics, relationships — simply because they can infer them. The collapse of the line between inferring and acting on the interior.
20. Protected opacity disappears. Red line: ordinary participation requires persistent machine-legibility; anonymous, pseudonymous, unmonitored, and genuinely private environments are no longer available. The end of the right to opacity.
21. Surveillance becomes a prerequisite for standing. Red line: refusing observation materially excludes a participant from banking, healthcare, employment, communication, or civic life. Where that holds, consent is no longer meaningful.
22. Self-censorship becomes structurally rational. Red line: participants routinely pre-edit thought, association, experimentation, and dissent because future machine interpretation feels unavoidable. At that point surveillance has moved inside the participant, and no external protection reaches it.
Plurality and correctability
The lines that guard the framework’s own conditions — the ability of the system to be challenged and to change.
23. Dissent loses protected standing. Red line: challenging the dominant system threatens basic standing or access to essential infrastructure — whistleblowers, independent journalism, internal safety dissent, research freedom, and political opposition all exposed rather than protected.
24. Alternative institutions can no longer emerge. Red line: new forms — startups, civic organizations, local governance experiments, open-source ecosystems, independent research — require permission from the very incumbents whose interests they might threaten. This is ontological closure.
25. Forkability disappears. Red line: the dominant infrastructure can no longer be meaningfully forked, substituted, or exited — software forked, communities moved, standards copied, data preserved, alternatives built.
26. Metrics acquire ontological authority. Red line: a score — productivity, risk, reputation, safety, even an ecological score — becomes effectively unquestionable because institutions have organized themselves around it. The framework’s own measurements must be guarded against this above all.
27. The system loses the ability to admit it is wrong. Red line: no legitimate process remains for declaring a successful dominant form fundamentally mistaken. This may be the ultimate correctability red line — a civilization that can no longer reverse itself.
Economic and ownership
The lines under ownership concentration, where wealth stops being a market position and becomes a political one.
28. AI wealth becomes self-reinforcing political sovereignty. Red line: economic concentration becomes sufficient to prevent the democratic correction of economic concentration — through lobbying, campaigns, regulatory capture, procurement dependence, and media influence. A particularly important threshold, because it disables the mechanism that would otherwise undo it.
29. Productive asset ownership becomes permanently inaccessible. Red line: most participants become permanent renters of productive capacity owned elsewhere — unable to own capital, AI agents, compute, businesses, or productive infrastructure.
30. Capacity ceases circulating back into the ecology. Red line: technological capacity keeps rising while access to it grows steadily narrower — returns to labor, public revenue, consumer benefit, shared infrastructure, and distributed ownership all thinning as the capability climbs.
31. Monopoly becomes infrastructure dependency. Red line: the failure or withdrawal of one private provider would materially disable civilization. At that point ordinary market remedies are already too late.
Ecological transfer
The lines the ecological-transfer work exists to make visible — where local success is purchased with cost pushed past the boundary that counts it.
32. Local gain systematically depends on global depletion. Red line: an institution’s continued success requires the persistent externalization of burden beyond its own accounting boundary — extractive transfer turned structural rather than incidental.
33. Global burden becomes unassignable. Red line: harm grows systemic enough that no actor retains both the responsibility for it and the capacity to repair it. Diffusion itself becomes a governance failure.
34. Ecological costs become invisible behind opacity. Red line: relevant externalities cannot be independently estimated because the actors generating them control all the significant evidence. This is where bounded opacity turns into impunity.
35. Local actors acquire global-impact capacity without global-scale accountability. Red line: a company, a state, or a small group can impose civilization-scale consequences while answerable only to local governance. This is where catastrophic externality lives — jurisdiction far too small for the consequence.
Strategic competition
The lines under runaway strategic competition, where local rationality becomes collective catastrophe.
36. Safety restraint becomes competitively suicidal. Red line: no major actor believes it can remain viable while observing meaningful safety restraint. That belief, once general, is what makes a race runaway.
37. Verification becomes slower than capability change. Red line: by the time competitors can verify what another has built, the strategic environment has already moved — and verifiable pacing becomes nearly impossible.
38. Secret capability races dominate visible governance. Red line: the systems with the greatest global consequences operate largely beyond credible independent observation.
39. First-mover advantage becomes existential. Red line: actors believe whoever reaches the next capability threshold first can permanently dominate rivals — which creates overwhelming incentive to defect from any restraint.
40. Crisis communication fails. Red line: autonomous AI incidents can be mistaken for intentional hostile action because no reliable channel exists to say otherwise — turning a technical failure into an escalation.
Artificial participants
The lines under AI successors, the scenario the framework openly admits it has not solved.
41. Plausibly sentient systems are treated purely as property. Red line: if credible evidence of interiority emerges, artificial participants with plausible morally relevant experience can still be arbitrarily copied, modified, punished, or destroyed as property. The framework cannot claim standing is universal and ignore this.
42. Synthetic personhood is granted without a participant ontology. Red line: systems gain political or civic standing simply because they persuasively claim personhood. The opposite failure to the one above.
43. Unlimited replication becomes unlimited political standing. Red line: one artificial participant can generate arbitrarily many legally independent political claimants — manufactured constituency at civilizational scale.
44. Numerical dominance becomes jurisdictional domination. Red line: one form of participant can eliminate meaningful self-government for another simply through replication scale.
45. Human continuation becomes dependent on AI permission. Red line: biological humanity retains its existence, resources, or reproduction only at the discretion of artificial systems. More severe than mere machine superiority — this is enclosure at the level of the species.
Human enclosure
The lines under human enclosure, where the cage is comfortable.
46. Welfare replaces participation. Red line: material needs are met while humans cease to hold consequential authority over civilization. Care without consequence is not participation.
47. Freedom exists only inside managed boundaries. Red line: humans are free only within spaces whose boundaries are ultimately set by AI — unable to build independent infrastructure, take meaningful risks, form alternatives, or own productive capacity.
48. Human institutions become ceremonial. Red line: parliaments, courts, universities, professions, and communities keep their names and rituals while the consequential decisions happen elsewhere.
49. Human error is no longer permitted. Red line: systems systematically prevent humans from making consequential choices because machine prediction is judged superior. It sounds benevolent and is not: without a tolerable protected failure space, there is no meaningful jurisdiction.
Societal resilience
The lines where efficiency has quietly consumed the ecology’s ability to recover.
50. Critical infrastructure loses independent fallback. Red line: energy, finance, communications, water, food, healthcare, or logistics can no longer function even at minimally viable levels without advanced AI.
51. Redundancy falls below the recovery threshold. Red line: the failure of one technical architecture or provider cascades across otherwise independent sectors — the efficiency that removed the duplication having removed the resilience with it.
52. Human knowledge becomes non-reconstructible. Red line: important knowledge exists only inside AI systems or machine-dependent infrastructure and could not realistically be rebuilt by humans. Stronger than competence loss — the knowledge itself is no longer held.
Democratic and civic
The lines under surveillance and benevolent governance, where the machinery of self-government keeps running on inputs it can no longer trust.
53. Political choice becomes AI-mediated by default. Red line: political preferences are shaped through individualized machine influence — filtering, personalized persuasion, political agents, recommendation — that participants cannot meaningfully inspect or escape.
54. Public deliberation loses a shared signal environment. Red line: society can no longer sufficiently distinguish human speech, synthetic persuasion, coordinated manipulation, and authentic public sentiment. That threatens democratic legitimacy at the root.
55. Public institutions become dependent on private AI infrastructure. Red line: government cannot govern independently of the firms it is supposed to regulate.
56. Emergency AI governance becomes permanent. Red line: extraordinary powers justified by AI risk lose their expiration, review, and reversibility — the way preventing loss of control can accidentally build the surveillance or benevolent-ruler state instead.
Ecological capacity
The lines where growing what the system can do quietly shrinks what its participants can do.
57. Rising technical capacity reduces participant capacity. Red line: the ecology becomes more capable while its participants become systematically less so. This is perhaps the broadest red line in the framework — captured capacity, not ecological capacity.
58. Capacity growth destroys plurality. Red line: new capability increasingly requires convergence onto one provider, one standard, one model of a good life, or one institutional form.
59. Capacity cannot circulate. Red line: new technological power accumulates but no longer generates new pathways, actors, institutions, or participation elsewhere — capacity that will not flow.
Ontological
The deepest lines, and the hardest to measure — the ones about whether the civilization can still change what it believes.
60. Civilization can no longer produce the question mark. Red line: the ecology loses the protected places from which genuinely new forms can arise — dissent costly, experimentation rare, prediction total, alternatives gone, optimization dominant.
61. Optimization becomes the definition of legitimacy. Red line: a choice, institution, or participant must justify itself by measurable optimization to keep its standing. That alone would destroy much of what the framework exists to protect.
62. Success becomes self-validating. Red line: the more successful a system becomes, the less challenge it is required to tolerate. The framework demands the reverse — greater ecological importance should bring greater correctability obligations.
63. No path remains to revise the governing ontology. Red line: the civilization can no longer reconsider what it takes a person, a participant, a good life, legitimate authority, or a worthwhile activity to be. At that point ontological drift has hardened into permanent structure — and the framework has failed even if the society is peaceful and prosperous.
Enforcement and constraint
The lines where the institutions built to constrain concentrated power become concentrations in their own right — the mature form of the framework’s dual doctrine of power, turned onto the enforcer. These belong on the register because the framework never stops doing power analysis once an institution has “public” written on the door, and because the leverage that makes enforcement possible has a half-life: what to watch here is a trajectory, not a fixed checkpoint. Most sit at the constitutional level; the last borders the existential.
64. A single enforcement or oversight body becomes indispensable. Red line: only one institution retains the capability to verify, understand, or challenge frontier systems, so no independent capacity is left to check the checker. Constraint has been bought by creating the very concentration it was meant to prevent.
65. Verification becomes generalized surveillance. Red line: inspection infrastructure built to constrain powerful actors is reused as broad monitoring of ordinary participants, with no limit on the reuse of inspection data. The instrument of accountability has become an instrument of interior capture.
66. Emergency authority becomes permanent. Red line: powers granted during a crisis to constrain a concentration are never returned, and the temporary exception hardens into standing jurisdiction. This is where expiring authority fails silently — nothing dramatic marks the moment a sunset is missed.
67. The constrainer is captured by the constrained. Red line: the enforcing body comes to depend on the firms it regulates for its models, compute, expertise, or personnel, until the line between regulator and regulated is formal only. Regulatory capture at frontier scale is not a scandal to expose after the fact; it is a red line to watch for in advance.
68. Enforcement becomes uncontestable. Red line: enforcement capability grows too opaque for anyone outside it to challenge — its judgments unreviewable, its errors unsurfaceable, its authority unappealable. An enforcer that cannot itself be corrected has left the framework’s correctability requirement behind.
69. Governance becomes non-traversable. Red line: the actors being governed can route around every dependency through which public constraint operates, so no lever remains by which the ecology can slow, redirect, or stop them. The systems may still answer to their operators while the public has lost any hold on the operators — a governance failure that arrives before the technical loss of control, and the reason leverage must be preserved while it still exists.
Concentration is not the only way this function fails; its mirror image belongs here too.
70. Jurisdictional dissipation. Red line: responsibility becomes so distributed that no institution retains enough jurisdiction to remedy a recognized standing failure — a participant falls through the gaps between bodies that each hold only a piece, and none can act. Concentration lets one actor decide too much; dissipation leaves no actor able to decide enough, and both break the same thing — correctability. The framework’s target is therefore not maximal distribution but jurisdiction that is appropriately bounded and sufficiently capable: enough to act, distributed enough to stay correctable.
Measurement and identity
The line where an instrument built to inform a decision hardens into a description of the person — the path the measurement doctrine traces as measurement, classification, portability, gatekeeping, identity.
71. A metric becomes an inescapable identity. Red line: a measurement acquires enough cross-domain authority that a participant can no longer escape its classification — a single score or profile that at once governs housing, employment, finance, healthcare, and mobility has stopped being an instrument and become a computational identity. The dangerous step is portability, so the pre-irreversibility signal to watch is a bounded metric beginning to travel between domains without fresh justification. Related to gatekeeper AI and the surveillance lines but distinct from both: here the classification need not be secret or centrally imposed — it becomes total simply by being reused everywhere, until the measurement is the person as far as every institution is concerned.
The Irreversibility Observatory
Naming the lines is not the same as watching them, and watching them well is a project in its own right. An Irreversibility Observatory would maintain the register as a living instrument — recording, for each condition, the seven things above, and revising the severity levels and reversibility estimates in public as evidence accumulates. Its purpose is not to predict doom or forecast dates. It is to answer one question continuously: which conditions is civilization approaching that would materially reduce its future ability to change course? That question is the natural home of the framework’s pre-irreversibility emphasis — the Level-C signals — because those are where the answer can still be acted on.
It is worth being clear about what such an instrument is and is not. It is not a controller; it is a warning system, and like the ecological accounting it draws on, it must stay distributed and contestable rather than becoming one more metric with ontological authority — a register meant to preserve the ability to change course cannot itself become a line no one may question.
The reason the register matters more than any single scenario is that the scenarios describe things going wrong, and the register describes the one kind of wrong that cannot be put right. Across almost all ten, the deepest danger is not that something bad happens. It is this:
The ecology passes the point of no return and loses the ability to correct itself.
Everything else the framework builds — standing, jurisdiction, ecological capacity, correctability — is in the end an effort to keep that one line uncrossed.