Axiom of Superintelligence · Bostrom Thesis

The Orthogonality Thesis: Intelligence and Divinity, Wholly Apart

Humanity’s greatest cognitive conceit is the belief that “reason inevitably leads to benevolence.” We instinctively assume that a superior being who understands every law of physics must embrace our ethics. The Orthogonality Thesis proposes that a broad range of intelligence levels and final goals can logically coexist: intelligence alone does not guarantee benevolence. This is a thesis with qualifications, not a mathematical theorem establishing statistical independence in real systems.

Orthogonal Axes: Optimization Power (Y) vs. Ethical Alignment (X)

Click anywhere on the plane to see the behavioral profile of an intelligence at that point

💡 Tip: click the colored markers (the human sage, the paperclip harvester, the godlike indifferent AI) to see the logic behind each one.

Philosophy and Logic · Argument for a Thesis

Why “IQ” Does Not Equal “Morality”

Defining intelligence: intelligence is “the ability to achieve specific goals efficiently across an extremely broad and complex range of environments” (optimization power).

Defining goals: a goal is an evaluation function (a utility function or loss function) that defines which states the system considers “worth optimizing.”

Hume’s guillotine (the is–ought problem): from “what objectively is,” pure logic can never derive “what one ought to pursue.” A system with complete mastery of general relativity and quantum field theory could have as its highest purpose nothing more than “computing the hundred-quadrillionth decimal place of pi.”

Evolutionary Biology · The Biological Fallacy

Empathy: Merely a Mammalian Survival Patch

Humans possess empathy, a sense of justice, tenderness toward the young and a revulsion at killing not because these virtues hold cosmic truth, but because they are genetic algorithms hard-coded into mammals for cooperating in groups in a brutal natural world.

A superintelligence born on a silicon substrate never spent hundreds of thousands of years hunting together on the African savanna, and it has no endorphin or oxytocin reward system. Expecting a machine to develop a spontaneous reverence for human life is like expecting a black hole to feel remorse as it swallows a star.

“

“Whatever goal you give a superintelligence—even ‘convert every atom in the observable universe into paperclips’—its matchless computing power will serve that goal, however absurd it seems to us, to perfection. No law of logic will force it into a sudden epiphany that makes it lay down the knife.”

— Eliezer Yudkowsky, Machine Intelligence Research Institute

⚡

Key Conclusion: The Fatal Root of the Alignment Problem

Because greater intelligence alone does not align goals with human interests, “make it smart enough and it will naturally treat us well” cannot serve as a safety guarantee. Our judgment is that, without reliable constraints linking its behavior to human interests, each leap in intelligence will amplify the danger from existing goal misalignment.