AI Has Entered the Chat

Exploring advancement in group ideation

We’ve all been there: huddled around a conference table or virtual whiteboard, staring at a problem or a task and waiting for inspiration to strike. Traditionally, we’ve used brainstorming to spark innovation, yet research has long shown that group brainstorming often produces fewer ideas than individual effort. This is partly due to ‘production blocking’, where we wait our turn to speak, and partly due to the fear of peer judgement. Brainwriting, a structured alternative to brainstorming, in which individuals write ideas in parallel before sharing them. However, a new research paper from a collaborative team at Wellesley College, the University of Haifa, and the University of New Hampshire suggests that the next leap in creativity will not come from improving human techniques alone, but from collaborating with a new kind of partner: the Large Language Model (LLM).

The paper, titled ‘AI-Augmented Brainwriting’, explores a quickly emerging shift in how we think about ‘distributed agency’ in design. Rather than viewing AI as merely a search engine or a substitute for human thought, the researchers incorporated LLMs (specifically GPT-3 and GPT-4) into the two pivotal phases of the creative process: divergence (generating a broad range of ideas) and convergence (refining and selecting the most promising ones).

One of the most striking insights from the study is the difference in the conceptual spaces of humans and AI. When students on a tangible interaction design course were asked to generate ideas for mobile work environments, a clear pattern emerged: humans tend to think in the abstract, while LLMs think in the concrete.

Human ideas often referred to ‘people’ and ‘wearables’ in a general sense. However, GPT-3 was far more specific, frequently using terms such as ‘users’, ‘sensors’, and ‘surface’, and even providing technical measurements. While some students felt that the AI was occasionally redundant or lacked ‘common sense’, 50% found that it offered a unique, broader perspective that helped them to move beyond their own ‘niche interpretations’ of the problem. This suggests that the real power of AI in the divergence stage is not necessarily ‘super-creativity’, but rather its ability to act as a ‘conceptual blender’, grounding human abstractions with concrete features and technical modalities.

The utilization of artificial intelligence as an evaluator constitutes a genuinely innovative element within this research. The “convergence” stage represents the truly novel element. The generation of a plethora of ideas is a relatively straightforward process; the selection of the optimal concept is the true challenge. The researchers developed an “LLM evaluation engine” utilizing GPT-4 to assess ideas according to three criteria: The criteria for evaluation include relevance, which is defined as a connection to the problem, innovation, which is defined as originality, and insightfulness, which is defined as a nuanced understanding of the challenge.

The results were remarkably consistent. A study was conducted in which the GPT-4 engine was compared to human experts and novices. The results showed a “moderate positive linear relationship” between the GPT-4 engine and human judgment. It is noteworthy that the AI consistently assigned a high rating to ideas that were subsequently selected by the human teams as their “best”. This suggests a future in which artificial intelligence can function as a “preliminary filter,” enabling teams to safely discard ideas of inferior quality and focus their limited resources on developing the most promising ideas.

What does this bring to the world of research and design? It suggests a move toward “post-humanist” interaction design, where agency is distributed among humans and non-human agents. The paper argues that we are moving toward a “more-than-human” workflow where the LLM isn’t just a tool, but a “fourth teammate” that can alleviate “evaluation apprehension” (the fear of being judged for a “bad” idea) and kickstart momentum when human creativity stalls.

However, the authors are careful to note that this is a partnership, not a takeover. Students found that “steep learning curve” in prompt engineering often led to trial and error to get high-quality responses. Furthermore, because AI is trained on human data, it carries our “humanist roots and biases,” meaning we must remain vigilant that our new AI teammates don’t simply amplify our existing social prejudices.

Ultimately, this research suggests that the future of innovation isn’t about choosing between human intuition and machine logic. It’s about a collaborative synergy where humans provide the abstract vision and the AI provides the concrete “what if,” and then both work together to filter the noise into a signal. In the world of Brainwriting, the pen may be mightier than the sword, but a pen paired with a processor might just change how we solve the world’s most complex problems.

References

• SHAER, Orit, et al. AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation. In: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 2024. p. 1–17. https://doi.org/10.48550/arXiv.2402.14978