The Scaling Hypothesis of Intelligence and the Scale Invariance of Choice

 The Scaling Hypothesis of Intelligence and the Scale Invariance of Choice

As intelligence grows, does the structure of decision-making change?

Beneath much of the criticism directed at politicians and other leaders lies a simple frustration. Why do they not do more for the people they govern? Why are they unable to make better choices? Such questions usually carry an unspoken assumption: someone who becomes a prime minister or a president must possess more options than the average citizen and should therefore be able to respond to events with greater flexibility.

But imagine from the inside that you yourself had become a prime minister or a president. Vast quantities of information would flow toward you, and you would be able to mobilize enormous organizations. Even so, the lived experience of choosing, moment by moment, might not differ so greatly from the choices made in ordinary life. This possibility suggests a hypothesis: perhaps choice—or even free will—has a form that remains invariant across changes of scale.

Intelligence Scales

The phrase “scale invariance” immediately calls fractals to mind. In the Mandelbrot set, for example, related structures appear at different levels of magnification. I do not mean to claim that human decision-making is a fractal in any rigorous mathematical sense. The comparison is a thought experiment: even when power and the number of relevant variables increase by several orders of magnitude, might the basic experience of choosing reappear in much the same form?

A leader can unquestionably control more variables than an ordinary citizen. Governments command administrative institutions, allocate budgets, formulate policies within the law, and make decisions about national security. Political power can therefore be understood, at least in part, as an increase in the number of variables one is able to manipulate within an immensely enlarged phase space.

Artificial intelligence presents a contrasting version of the scaling hypothesis. As the size of models, datasets, and computational resources has increased, large language models have tended to acquire greater capabilities in a partly predictable way. Demanding benchmarks such as Humanity’s Last Exam measure how well AI can answer questions that even the most accomplished human experts find difficult. Hallucinations remain, as do limitations in creativity, communication, and reasoning. Nevertheless, the empirical idea that greater scale can yield more advanced intelligence has acquired considerable force.

Yet the scaling of intelligence and the scaling of choice are not the same thing. A system may integrate more information, predict outcomes more precisely, and improve the quality of its judgments. Still, at the moment when one action must be selected, high-dimensional intelligence has to pass through a narrow exit. There may be a fundamental discontinuity between the breadth of intelligence and the form of a decision.

The Decision Bottleneck

The choices presented to us often take a deceptively simple form: this way or that way. I therefore propose a “double-slit” model of decision, as though an agent had to choose which of two openings to pass through. This is only a metaphor; it does not imply that decision-making is a quantum process. The point is that even a classical process may end in a branch with only two available exits.

In reality, ten thousand variables may be interacting, and their configurations may generate radically different future dynamics. Human cognition, however, does not necessarily survey that entire high-dimensional landscape before acting. Instead, a complex situation is compressed into a small number of cognitively manageable alternatives: act or refrain, approve or reject, continue or stop. A prime minister or president may also have to pass through this compressed entrance before action becomes possible.

AI may eventually compare configurations containing thousands of variables, project the trajectory associated with each of them, and exercise some control over the dynamics themselves. It may gain direct access to a high-dimensional space of possibilities that human beings cannot see. Yet when its computation is connected to action in the real world, discrete branches still emerge: implement this policy; do not issue this command. The complexity may grow, while the exits into action remain few.

Medicine and the Design of Options

The structure of evidence-based medicine offers a useful illustration. Consider a comparison between giving a patient nivolumab, marketed as Opdivo, and not giving it, and then measuring the difference in clinical outcomes. Medical evidence is often presented in a form suited to a physician’s decision: should this intervention be adopted or not?

The body itself is far more complex. Immunity, metabolism, genetic background, living conditions, concomitant medications, and countless other variables interact organically and change over time. Yet to make a decision possible, this intricate physiology is projected onto a single treatment choice. The important point is that evidence does not merely support a choice. Evidence is also organized to fit a form of choice that human beings can understand and act upon.

We do not choose the complexity of the world in its original form. We first divide the world into branches that can be chosen, and only then select one of them. Unless these two stages are distinguished, we risk confusing the quantity of information available to intelligence with the range of possibilities actually experienced by free will.

The Magnitude of Power and the Solitude of Choice

If this hypothesis is correct, a national leader ultimately stands in a position surprisingly similar to our own. Expert analysis, institutional knowledge, and privileged information surround the decision-maker. Nevertheless, what the person finally experiences is a singular question: which course should I choose? Power may increase ten thousandfold without expanding the subjective act of choosing by the same proportion.

We expect a prime minister or president to move freely through a vast phase space, solve equations of decision too complex for ordinary people, and perceive an optimal move invisible to everyone else. That expectation may be an illusion. Access to information and analytical intelligence can expand, while the cognitive form of choice remains much as it is in everyday life.

This does not absolve leaders of responsibility. Even if the form of choice is the same, the number of people affected and the gravity of the consequences are vastly different. The answer is not to demand superhuman free will from one individual, but to examine how options are constructed and to build institutions that admit dissent, expose error, and permit correction. Relinquishing excessive faith in leaders is not political resignation. It is a shift of attention toward the architecture that supports decisions.

Intuition First, Reasons Later

However complicated a problem may be, action can often be formulated as go or no-go: proceed or hold back. The two alternatives may be asymmetrical, but the final branch still has two paths. The cognitive process operating at that threshold may resemble what we call intuition. Here intuition is not a mysterious faculty. It simply means that not all the reasons contributing to a judgment are available to conscious awareness.

Experiments on choice blindness demonstrate this vividly. A person selects one of two alternatives. Through a sleight-of-hand procedure, the result is switched, and the participant is shown the option that was not actually chosen. Many people fail to notice the substitution. Asked why they made that choice, they proceed to offer a plausible justification. Rather than choice emerging from a conscious accumulation of reasons, the selection appears to occur first through nonconscious processes, with an explanation assembled afterward.

A reason is not necessarily the cause of a choice. It can also serve as a bookkeeping device through which we organize events for ourselves, or as a narrative through which we explain our actions to others. If the process that produces a decision and the process that explains it are distinct, the scale invariance of choice may reside not only in a binary branch, but also in this double structure.

How Should Free Will Be Implemented in AI Agents?

This question will become increasingly important as we design AI agents. What grows through scale is, first of all, intelligence. As models become more advanced, their ability to incorporate many parameters, anticipate multiple futures, and improve the quality of judgment will continue to increase. But an agent acting in the world must eventually adopt a course of action. The space of evaluation may expand while the branch into action remains scale-invariant.

If so, AI may require a dual architecture. The choice itself may emerge from immense internal computation without the system explicitly comprehending every contributing reason. When an explanation is requested, a separate process may organize the grounds for the judgment and translate them into language a human being can understand. This does not mean reproducing human post hoc rationalization. On the contrary, decision-making and explanation should be distinguished so that we can audit how faithfully an explanation reflects the actual causal process.

Intelligence scales. Choice, however, may continue to pass through the same narrow gate. Understanding a vast space of possibilities and living out one possibility within it are different problems. Recognizing this distinction can make our expectations of political leaders more realistic while opening a new approach to free will and accountability in AI agents. What we must design is not an infinitely wise chooser, but an honest bridge between the immensity of intelligence and the inescapable narrowness of decision.

Transcribed and formatted by AI from Ken Mogi's talk in Japanese

https://youtu.be/YKquFwZ1Mho