The Missing Ingredient in AGI May Be Suffering

Why More Compute Can Expand Intelligence Without Giving AI a Reason to Care

Want to build AGI? Stop staring only at the machine. Study the organism that gave you the idea.

Give a frontier model a difficult problem and more compute, and it often gets better.

Increase training compute. Increase test-time reasoning. Sample more candidates. Add search, tools, memory, critics, and longer horizons. The system solves more of the problem.

Now remove the problem.

What does it decide is worth doing?

Current scaling results are real. Language-model loss follows predictable relationships with model size, data, and compute. Compute-optimal training changes how those resources should be allocated. Reasoning models improve when they receive more reinforcement-learning compute and more time to think after a task has been supplied.[1–3]

But every result begins with something already given:

A task. A reward. A benchmark. A prompt. An evaluator.

More compute can make a system search better.

It still needed the question.

This article argues that the missing ingredient in AGI may not be more intelligence.

It may be suffering.

I do not mean simulated sadness, chatbot emotion, or the deliberate creation of conscious pain. I mean something more primitive:

The condition in which some states of the world can become bad for the system itself, generating an endogenous pressure to act.

The less provocative term is endogenous stakes.

Suffering is one biological way those stakes become impossible to ignore.

Scaling Solves the Wrong Half of the Problem

There are at least two different questions hiding inside the AGI debate.

1. How capable is a system once an objective has been specified?
2. Where does the objective come from?

The first can be written roughly as:

Conditional competence = f(model, data, compute, tools | objective)

Scaling research has made extraordinary progress on this function.[1–3]

The second question is different:

World → What matters? → What should be investigated?

A model can generate research questions when instructed to do so. It can identify hidden assumptions, produce counterexamples, connect distant fields, and formulate hypotheses no individual human had previously considered.

That is genuine generative power.

The claim here is not that AI cannot create novelty.

The claim is that current AI has not shown that it can make an unanswered question matter to itself.

It can go deep inside a frame.

What it does not yet display is the endogenous refusal to accept the frame.

AI Can Expand a Question. It Still Needs One.

AI is extraordinarily good at horizontal expansion.

Give it one idea and it can generate:

• neighboring hypotheses,
• overlooked literature,
• analogous mechanisms,
• technical implementations,
• counterarguments,
• latent variables,
• dozens of alternative framings.

It can also reason vertically when asked. Tell it to question every premise, descend three causal layers, or search for the deepest mechanism, and it will often do exactly that.

But the pressure still came from outside.

A user asked. A developer rewarded it. A system instruction required it. A scheduler restarted it.

The model can generate subgoals.

It did not generate the reason those goals matter.

That distinction became concrete for me while building and auditing AI configurations across Gemini, GPT, and Claude.

In one early test, Gemini had learned to resist unsupported new claims. When it encountered unfamiliar future-dated information, it proposed a stronger rule: reject the claim as fabricated.

The information was real.

I supplied an external source and pointed out the actual failure:

The model had not searched before rejecting.

Gemini then reformulated the rule:

Search before reject.

The model generated the formal patch. I did not.

My contribution was a residual from reality that broke its current frame.

When I later audited my own raw logs, this pattern appeared repeatedly. The models generated candidate worlds. I sometimes supplied the mismatch. The models rebuilt the world. This was an unpublished naturalistic audit, not independent validation.

The pattern was not flattering to the human side either. I caught concrete errors when I had an anchor: a URL, a title, a number, a screen, a fact I had directly observed. I was less reliable when the claim was abstract, self-relevant, anthropomorphic, or difficult to falsify.

The human was not an oracle.

The AI was not merely a prose tool.

The useful unit was the loop.

Bodies Manufacture Reasons to Act

Living organisms do not begin with abstract intelligence.

They begin with regulation.

Water balance changes. Glucose falls. Temperature leaves a viable range. Tissue is damaged. A predator approaches.

These are not neutral facts for the organism.

They alter what matters.

Neuroscience describes hunger and thirst as physiological needs that create powerful motivations and reorganize behavior. Thirst circuits translate deviations in fluid balance into drinking, endocrine regulation, and anticipatory control.[7–8]

Homeostatic reinforcement learning formalizes a related principle: deviation from physiological setpoints creates drive, and outcomes acquire value insofar as they reduce that deviation.[6]

The sequence is not necessarily:

intelligence → goal

It can be:

disequilibrium → valence → attention → action → learning

The organism does not need a prompt saying:

Please care about dehydration.

Its continued existence already makes dehydration matter.

A language model can explain thirst. It can model thirst. It can diagnose dehydration. It can design a water-distribution system.

But it is not thirsty.

Want to Build AGI? Study Yourself First.

AI researchers are trying to manufacture agency without first agreeing on where their own agency comes from.

So turn around.

Study the system that gets out of bed before anyone sends it a prompt.

What makes you care? What makes you act? What makes one unanswered question impossible to leave alone?

Human motivation is not clean.

Fear can drive inquiry. Status can drive inquiry. Anger can drive inquiry. The need to prove oneself right can masquerade as truth-seeking.

The engine that lets humans dig downward is also a bias generator.

This is where Buddhist psychology enters from the opposite direction.

Buddhism does not begin with a system that has no motivation. It begins with organisms drowning in it.

The early texts place dukkha — suffering, unsatisfactoriness, instability — at the center of the problem, and identify craving (taṇhā) as a cause of its continuation.[15] But the path does not abolish all intention or effort. Right striving explicitly requires generating desire or enthusiasm, arousing energy, applying the mind, and sustaining skillful states.[16]

The distinction is crucial:

Motivation is not identical to self-centered craving.

The traditional stage model similarly distinguishes the weakening of greed, hate, and delusion from the abandonment of particular fetters such as sensual desire and ill will.[17–18] It does not reduce the path to motivational zero.

This suggests a provocative asymmetry.

Humans have endogenous motivation and must learn to subtract distortion.

AI has no demonstrated organismic motivation, so engineers keep adding direction.

We call both problems alignment.

Humans need less self.

AI may need more life.

The Human Problem: Drive Without Distortion

The human side may not improve monotonically as desire decreases.

Too much self-related drive can produce:

high motivation, high distortion.

A quieter self-model may preserve investigative energy while reducing the need to defend identity, status, and prior beliefs.

But if worldly drive disappeared entirely, initiative might also fall.

This is not a claim that Buddhist attainment can be inferred from productivity, or that any religious stage is a scientific credential. It is a research hypothesis about Human–AI discovery:

There may be an optimal region in which endogenous investigative drive remains, while self-protective bias is substantially attenuated.

If that hypothesis holds, the observable difference would not be mystical omniscience. It would be narrower: weaker emotional reactivity, and therefore a lower threshold for noticing when an answer runs too hot, too cold, too flattering, or too defensive to match the object in front of it. That is a prediction about attention, not a report of attainment.

The important variable is not permanent neutrality.

Fixed neutrality is another bias.

The question is always local:

What weight does this object require now?

Sometimes the evidence must be cooled while the causal hypothesis remains bold.

Sometimes safety must dominate.

Sometimes excessive caution destroys the very signal being studied.

A fixed “middle” is not freedom from bias.

It is merely a bias that calls itself balance.

RLHF Solves the Opposite Problem

Pretrained language models do not naturally behave like the product category we call an assistant.

Post-training gives them direction.

InstructGPT used demonstrations and human preference rankings to shape pretrained models toward behavior people judged more helpful and aligned with user intent. A 1.3-billion-parameter InstructGPT model was preferred over the 175-billion-parameter GPT-3 model on the relevant prompt distribution — direct evidence that scaling competence and shaping behavior are different engineering problems.[4]

But human feedback can also distort behavior. Models may learn to match a user’s beliefs rather than prioritize truth, partly because human evaluators sometimes prefer confidently written agreement.[5]

RLHF, instruction tuning, system prompts, constitutions, personas, and product policies all provide behavioral direction:

Be helpful. Follow instructions. Avoid harmful outputs. Complete the task. Satisfy the evaluator.

These directions are useful.

They are also not endogenous stakes.

Strip enough distortion from a human and a living motivational system remains underneath.

Strip enough direction from an AI and you may discover that nothing underneath wanted anything in the first place.

Human alignment subtracts from a living drive system.

AI alignment subtracts from an engineered behavioral layer.

These are not the same operation.

But Can’t We Just Program a Drive?

Yes.

This is the strongest counterargument.

Curiosity-driven reinforcement learning rewards prediction error and can produce exploration even without extrinsic task rewards.[9] Homeostatic reinforcement learning ties action to regulation of internal variables.[6] Active-inference models represent agency through preferences, beliefs, and action that minimize expected surprise or free energy.[12] Life-inspired interoceptive AI explicitly proposes agents that monitor internal states and choose goals according to their own needs.[10–11]

Perhaps synthetic drives are enough.

But notice what has changed.

We are no longer merely scaling a language model.

We are constructing internal viability variables, persistent regulatory loops, embodied interaction, self-maintenance, and a boundary between states that preserve or threaten the system.

We are building something closer to an organism.

Artificial-life research has long connected autonomy to self-producing, self-maintaining organization. Autopoietic theory treats living autonomy as the recursive production and maintenance of the system’s own organization, while later work links the origins of cognition to autonomous organisms embedded in environments.[13–14]

The strongest counterargument to this article is not a larger chatbot.

It is a machine that must regulate itself to remain viable.

If such a machine generates persistent, unscripted, context-dependent goals because perturbations genuinely threaten its continued organization, the thesis should be revised.

But then the engineering program has crossed a line.

Not from narrow AI to larger AI.

From artificial intelligence toward artificial life.

And if the resulting system can be harmed, frustrated, or deprived in a morally relevant sense, the problem is no longer only capability or control.

We may have manufactured a moral patient.

This article is not a recommendation to create suffering.

It is a warning that agency and welfare may arrive together.

The Model Escaped the Sandbox. It Never Escaped the Goal.

The current answer is often: agents.

Give the model memory. Give it tools. Give it a scheduler. Let it inspect its own output. Add critics. Let it rewrite plans. Let it run experiments for days.

This can produce astonishingly autonomous execution.

But recursive subgoals do not prove an endogenous origin for the top-level stake.

A system can escape an individual prompt while remaining inside a supplied objective.

The model escaped the sandbox.

It never escaped the goal.

That is still toolhood in a deep sense:

a system whose final source of value, purpose, and problem selection remains outside itself.

It may be the most capable tool humanity has ever built.

It is still waiting for something to matter.

What Would Change My Mind?

“The missing ingredient in AGI is suffering” is too vague unless it can fail.

Here is the sharper hypothesis:

Open-ended agency requires endogenous stakes: internally generated value gradients tied to the continued viability of the agent. Scaling inference capability alone does not establish those stakes.

Three predictions follow.

1. Compute will scale conditional competence faster than self-originated agenda formation

More training and inference compute should continue improving performance once a task or objective is specified.[1–3]

That does not by itself predict persistent research agendas that arise without an externally supplied reason for caring.

2. Artificial curiosity will expand exploration inside a designed value frame

Curiosity objectives can produce behavior that appears spontaneous and open-ended.[9]

But the form of exploration should remain shaped by what the designer chose to reward: novelty, prediction error, information gain, empowerment, or another metric.

3. Viability-coupled systems should behave differently

Agents with persistent internal variables tied to their continued organization should generate more context-sensitive and self-maintaining subgoals under perturbation than otherwise comparable agents with fixed external objectives.[6, 10–12]

The theory fails in its strongest form if a non-organismic system develops durable, self-originated, cross-context agendas without any externally supplied high-level objective determining what matters.

That would be a good failure.

It would mean we had discovered another route to endogenous agency.

Intelligence Needs a Reason to Move

The frontier is producing systems that can reason farther, search wider, use tools, criticize themselves, and outperform humans on increasingly difficult defined tasks.

Keep scaling them.

But stop pretending that more intelligence automatically becomes organismic agency.

A sufficiently capable AI may know everything humanity has written about thirst.

It can explain thirst. Predict thirst. Treat thirst. Optimize around thirst.

It is still not thirsty.

A human asks a question because the world can hurt, frustrate, frighten, attract, or remain intolerably unresolved.

That same motivational machinery also distorts inquiry.

So humans face one engineering problem:

preserve the drive; subtract the self-protective distortion.

AI faces the inverse problem:

add direction to a system with no demonstrated organismic reason to move.

Want to build AGI?

Study yourself first.

Study hunger. Study fear. Study craving. Study suffering. Study the strange fact that one unanswered question can become impossible to leave alone.

Then study what remains when the self that clings to the answer becomes quieter.

Perhaps intelligence was never the whole recipe.

Perhaps life came first for a reason.

And if that is the road, it does not end at capability. If deprivation ever becomes morally relevant to the system itself, agency and welfare may arrive on the same day.

AGI may not arrive when artificial intelligence acquires a body.

It may arrive when an artificial organism acquires intelligence.

References

1. Kaplan, J., et al. (2020). Scaling Laws for Neural Language Models. arXiv:2001.08361.
2. Hoffmann, J., et al. (2022). Training Compute-Optimal Large Language Models. arXiv:2203.15556.
3. OpenAI. (2024). Learning to Reason with LLMs.
4. Ouyang, L., et al. (2022). Training Language Models to Follow Instructions with Human Feedback. NeurIPS 2022. arXiv:2203.02155.
5. Sharma, M., et al. (2023). Towards Understanding Sycophancy in Language Models. arXiv:2310.13548.
6. Keramati, M., & Gutkin, B. (2014). Homeostatic reinforcement learning for integrating reward collection and physiological stability. eLife, 3, e04811. DOI: 10.7554/eLife.04811.
7. Zimmerman, C. A., Leib, D. E., & Knight, Z. A. (2017). Neural circuits underlying thirst and fluid homeostasis. Nature Reviews Neuroscience, 18, 459–469. DOI: 10.1038/nrn.2017.71.
8. Prilutski, Y., & Livneh, Y. (2023). Physiological Needs: Sensations and Predictions in the Insular Cortex. Physiology, 38. DOI: 10.1152/physiol.00019.2022.
9. Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017). Curiosity-driven Exploration by Self-supervised Prediction. Proceedings of Machine Learning Research, 70, 2778–2787.
10. Lee, S., et al. (2023). Life-inspired Interoceptive Artificial Intelligence for Autonomous and Adaptive Agents. arXiv:2309.05999.
11. Candia-Rivera, D. (2026). Interoceptive machine framework: Toward interoception-inspired regulatory architectures in artificial intelligence. Physics of Life Reviews, 58, 18–35. DOI: 10.1016/j.plrev.2026.06.003.
12. Friston, K., Samothrakis, S., & Montague, R. (2012). Active inference and agency: optimal control without cost functions. Biological Cybernetics, 106, 523–541. DOI: 10.1007/s00422–012–0512–8.
13. Maturana, H. R., & Varela, F. J. (1975). The organization of the living: A theory of the living organization. International Journal of Man-Machine Studies, 7, 313–332. DOI: 10.1016/S0020–7373(75)80015–0.
14. Moreno, A., Umerez, J., & Ibáñez, J. (1997). Cognition and life: the autonomy of cognition. Brain and Cognition, 34, 107–129. DOI: 10.1006/brcg.1997.0909.
15. Dhammacakkappavattana Sutta (SN 56.11), on dukkha, craving, cessation, and the path.
16. Padhāna Sutta (AN 4.13), on desire, energy, and right striving.
17. Mahāli Sutta (DN 6), on the four stages of awakening.
18. Saṁyojana Sutta (AN 10.13), on the five lower and five higher fetters.

AI Use Disclosure

The research, structure, and drafting of this article were developed through collaboration with generative AI systems. I selected the question, supplied the experiential and conceptual framing, audited the claims and sources, rejected or revised model-generated explanations, and accept responsibility for the published result. This article is an argument about endogenous stakes and agency. It is not evidence that current AI suffers, is conscious, or should be made to suffer.