Above All, Friction

Why thinking with AI should not be effortless

Outer space representing our metacognition.

The dream of modern technology has always been “seamlessness”, the idea that our tools should be so intuitive they practically disappear. We want to type with zero friction, search the web with zero effort, and have our software anticipate our every move. However, when it comes to the complex world of Generative AI (GenAI), this obsession with making things easy might be a hidden trap. As we move from using simple tools to collaborating with sophisticated cognitive partners, we must confront the “doctrine of simplicity” and recognize that, for our thinking to remain sharp, our AI interactions might actually need a few more seams.

A paper presented at ACM CHI 2024 delves deeper into aspects of metacognition for generative AI.

GenAI usage from the perspective of metacognition.

The most common type of GenAI these days are through Large Language Models (LLMs). Working with a LLM is less like using a calculator and more like managing a talented intern, that is occasionally unreliable. Managers are not supposed to just bark an order and walk away, at least not good ones. They define clear goals, break larger projects into smaller steps, and review critical parts of the results that has been produced. If the AI makes the process too effortless, we risk falling into an “illusion of competence”, we we stop thinking critically because the output just sounds polished and plausible, we believe it. The double-edginess of GenAI comes from here: it provides us with flexibility, originality and novelty meanwhile demanding that we remain actively engaged in the process.

3 properties of Generative Artificial Intelligence

This engagement is driven by metacognition, which is quite literally “thinking about thinking”. When we use GenAI, we are not just processing information; we are performing a complex dance of monitoring our goals and controlling our strategies. We have to ask ourselves: “Is this result what I want?”, “Am I confident enough to know if the answer is correct or not?”. If the usage is too seamless, we lose the mental friction — the effort — required to perform these checks. We become passive consumers of GenAI rather than intentional directors of it, the good managers.

A simple framework of metacognition

To solve this, we should advocate for “seamful” design, at least when it comes to GenAI: intentional moments of friction that force us to pause and reflect. Instead of LLMs that silently produces a final result, we need a “metacognitive coach” that proactively probes us. A seamful system might as us to specify our intended tone or target audience before it starts writing, or it might help us decompose a fuzzy, complex task into smaller, manageable sub-goals. These are what we could call “useful seams” because they ensure the user remains in the driver seat, maintaining control even when the machine does the heavy lifting. Giving us the desired high automation with high human control.

While a more “seamful” approach might feel like we are increasing the cognitive load, it can be thought of more like training wheels on a bike. Over time, these interventions help users develop better strategies for working with GenAI, eventually leading to a net reduction in overall cognitive effort as our partnership with GenAI improves. Ultimately, the age of GenAI is not just about the systems becoming smarter; it is about us becoming more sophisticated managers of our own thoughts. If we give in to GenAI without matching the metacognitive demands, we risk not being in control and being stuck in place instead of learning and getting results we desire. By embracing a bit of effort and intentional design, we can ensure that we do not just work faster, but think better.

Further Reading

• Lev Tankelevitch, Viktor Kewenig, Auste Simkute, Ava Elizabeth Scott, Advait Sarkar, Abigail Sellen, and Sean Rintel. (2024). The Metacognitive Demands and Opportunities of Generative AI. In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, Article 680, 1–24. https://doi.org/10.1145/3613904.3642902

[embed]