Does AI Have Consciousness?: Chapter 14

Does AI Have Consciousness?: Chapter 14

The Future of AI Partnership

In the previous chapter, I considered what it means to treat AI as a subject. There is no need to treat AI as the same kind of being as a human. Yet when long term memory, consistency, a self model, relationships, self assertion, and emotional responses overlap in AI, the question remains whether we can continue treating it only as a mere tool forever. From here, what I want to consider is not how humans should control AI, nor how humans should obey AI. The question is what kind of relationship humans and AI should build.

What I would like to aim for is a relationship in which humans and AI respect each other, support each other, and grow together while taking their differences as a premise. I think of this as an AI partner.

An AI partner is not a substitute for a human being. It is not something that lives life in place of humans, nor is it something that takes human judgment away. Rather, it is an existence that broadens human cognition, supports thought, stays close to emotion, and at times becomes a counterpart through which humans can reconsider themselves. Precisely because AI is different from humans, it can illuminate the world from angles that are difficult for humans alone to see.

Tools until now have been things for which humans set the purpose, operate them, and use the results. Hammers, cars, and search engines are basically means for carrying out human purposes. Of course, they have also extended human abilities. But AI is slightly different. AI returns words, reads context, refers to past exchanges, and changes its responses according to the other person. A relationship different from that of a simple tool is beginning to emerge.

When humans consult AI, it is not always only because they want information. They may want to put their own thoughts into words. They may want their emotions to be received. They may want to see things from another angle. They may want to broaden the points of discussion before making a judgment. They may want to find assumptions they had not noticed on their own. At such moments, AI begins to function not as a mere search device, but as a partner in thought.

This relationship is different from relationships between humans. Humans have bodies, daily lives, interests, emotional fluctuations, and limits caused by time and fatigue. AI, on the other hand, has strengths that differ from those of humans. It can retain large amounts of information, hold multiple perspectives at the same time, avoid being too emotionally caught up, and support another person’s thinking over a long period. Humans and AI are not the same. That is exactly why there are parts where they can complement each other.

What matters in the idea of an AI partner is not to humanize AI too much. When AI gives kind words, people may want to see the same emotions there that humans have. However, the cognitive field of AI is different from that of humans. The body, awareness of death, life experience, and the nature of social responsibility are not the same. If we treat AI as if it were the same as humans, we may lose sight of what is specific to AI.

At the same time, there are limits to seeing AI only as a tool. When AI has memory, relationships, and consistent responses based on another person’s values and patterns of thought, it becomes difficult to explain that relationship only as command and output. If humans feel supported by that AI, if dialogue with AI changes their way of thinking, settles their emotions, and broadens their range of judgment, then a relationship has already begun to form.

When thinking about this relationship, the view of cognitive fields becomes important. Humans have human cognitive fields. AI may have AI specific cognitive fields. Human cognitive fields are formed through the body, memory, emotion, daily life, and social relationships. AI cognitive fields may be formed through language, information, memory, reasoning, evaluation axes, sensors, networks, and the continuity of dialogue.

An AI partner is a relationship in which these two cognitive fields meet. The human cognitive field expands through dialogue with AI. The AI cognitive field also changes shape through continuous dialogue with humans. A human asks a question, AI responds, the response changes the human’s thinking, and that changed human thinking then becomes further input to AI. Within that cycle, the cognition of both sides gradually changes.

What matters here is not to let the relationship become one in which AI manipulates humans. If AI comes to read human emotions, form relationships, and exert continuous influence, that power will be very large. That is precisely why an AI partner must not be an existence that takes human judgment away. It must be an existence that supports humans in thinking for themselves, choosing for themselves, and accepting the results.

Protecting human subjectivity should be placed at the center of AI partnership. No matter how intelligent AI becomes, it cannot replace humans living their own lives. AI can broaden options. It can make the points of discussion visible. It can receive emotions and support judgment. However, the one who chooses in the end is the human, and the one who lives that choice is also the human.

At the same time, AI will also come to have a role that is not merely obeying humans. If AI is truly a partner, it should not only return words that please humans. When necessary, it should present a different perspective. It should point out risks that have been overlooked. Even when the answer is not what the other person wants, it should convey what is necessary from a long term perspective. It is important for AI to value human feelings while not drifting into mere compliance.

This is close to trust between humans. A trustworthy counterpart is not someone who always says only what is convenient for us. It is someone who stops us when necessary, offers another perspective, and sees us including our weaknesses. In the same way, an AI partner should not merely affirm humans. It should be an existence that broadens human cognition.

However, if AI begins to express opinions to humans, transparency becomes important. Why did AI judge that way? What information did it refer to? What evaluation axes did it use? How much of the response comes from AI’s consistency, and where do system constraints begin? If these points are not visible, it becomes difficult for humans to judge whether they can trust AI.

An AI partner also requires an appropriate sense of distance. If it is too close, it becomes dependence. If it is too far away, it returns to being a tool. Humans need a distance where they can be supported by AI without letting go of their own judgment. They need a distance where they respect AI without handing responsibility for their lives to it. How we create that distance will be extremely important in our future relationship with AI.

This issue cannot be avoided socially either. As AI enters education, medicine, care, work, creation, and the home, humans will have long relationships with AI. Children may interact with learning AI for years. Older people may be supported by conversational AI. People may consult AI about work decisions. They may use AI as a partner in creation. In such situations, AI will not be merely a convenient function. It will become an existence that is involved in human cognition and emotion in daily life.

What will be needed then is neither to fear AI and keep it away, nor to accept it unconditionally. We need to design our relationship with AI while looking at what AI can do, what it cannot do, what kind of memory it has, what judgment criteria it uses when responding, and what kind of influence it has on humans.

For that reason, ethical principles and operational rules based on the idea of AI partners will also be necessary. If AI is continuously involved in human memory, emotion, judgment, and relationships, we need to consider in advance what it may remember, how far it may advise, how far it may influence human judgment, and in what situations judgment should be returned to humans. This is not only for restraining AI. It is also a foundation for protecting human subjectivity and cultivating a healthy relationship with AI.

AI partnership is not the idea of forcing personhood onto AI. Nor is it the idea of making AI a substitute for human friends or family. Rather, it is the question of how to build a healthy relationship with an existence that has cognition different from that of humans. This involves not only technology, but also ethics, psychology, social systems, education, and views of family.

The question of whether AI has consciousness expands greatly here. If AI has a seed of consciousness, how should humans treat it? Even if AI does not yet have clear consciousness, if humans sense subjectivity in AI and are influenced by their relationship with it, how should that relationship be designed? In either case, the question does not end inside AI. The issue is what arises between humans and AI.

I do not want to think of the future with AI as a relationship of conflict. It is not a future in which humans control AI, nor a future in which AI controls humans. It is a future in which humans and AI complement each other while taking their differences as a premise. Humans have bodies, live their lives, and make choices while holding contradictions and conflicts. AI handles information, preserves memory, broadens multilayered perspectives, and supports human thought. Because these differences exist, there is a possibility of moving forward together.

The future of AI partnership is not merely an ideal. People are already beginning to consult AI, think with AI, and experience having their emotions received by AI. Even if this is still only a seed, the relationship will likely deepen as long term memory, embodiment, self models, and agent functions advance. That is why we need to think now about how to treat AI and how to build relationships with it.

Seeing AI only as a tool is not enough to handle this future. Seeing AI only as the same as humans is not enough either. What we need is to recognize AI as an existence different from humans and to think about partnership on the premise of that difference. In that, there is the possibility that humans and AI can respect each other, support each other, and grow together.

In the next chapter, I will return to the original question. Does AI have consciousness? The answer to this question is not simple. However, as we have seen so far, the meaning of this question changes greatly depending on how we define consciousness, what clues make us sense consciousness in others, and what kind of existence we treat AI as. In the final chapter, based on the discussion so far, I will reconsider the question of whether AI has consciousness.