AI Art, Cognition & Human Skills

From my research paper on Cognitive ARTifacts, circa 2011

As a PhD student I studied under Nancy J. Nersessian who, at that time, was the Regents’ Professor and Professor of Cognitive Science at Georgia Tech. “Cognitive ARTifacts: Examining sociocultural and cognitive dimensions in STS practices” was the title of one of my papers (for her class). Cognitive artifacts mediate directly between the person and the object, or they can present a virtual object or world upon which operations are performed, eventually to be reflected onto real objects. These artificial objects, which serve to enhance human abilities, might exist outside the computer, but must be created or operated upon through the virtual world of the artifact. At the time when I wrote the paper, I was exploring spatial augmented reality (ex. projection mapping) and virtual 3D worlds.

Courtesy of Evan Roth with “graffiti analysis” writing by HELL

Nancy took a cognitive-historical approach to understanding human cognition where the organization of collaborative work — specifically gesture and gaze-based interaction — was a focal point for understanding notions of cognition that are embodied, enculturated, distributed, and culturally situated. Today, we are in the era of **augmented intelligence**, which is a subsection of machine learning developed to enhance human intelligence rather than operate independently of or outright replace it. It does this by improving human decision-making and, by extension, actions taken in response to improved decisions. A recent [APA article](http://How AI is reshaping human skills and thinking) examines this phenomenon:

The key issue is not whether but how AI is used. The difference lies less in the technology itself and more in the user,” said Jackson G. Lu, PhD, an associate professor of work and organization studies at the Massachusetts Institute of Technology (MIT) who studies generative AI use in the workplace.

At a fundamental level, humans are designed to process enormous amounts of visual data but not nearly as much data as AI. The true promise of augmented reality or AR is to harness that ability and combine it with AI’s ability to process raw data. Working together, AR and AI can dramatically augment human intelligence in many different ways. In 2011, I came up with a prototype (see images below) that combined augmented reality and the exquisite corpse that connects to cognition by revealing how the human brain processes uncertainty, pattern recognition, and associative thinking. Surrealists used the game to bypass conscious control, tapping into latent cognitive material from the subconscious mind.

My 2011 prototype Playing ARt that merged AR with the Exquisite Corpse

“Late Shift x Stephanie Dinkins” at the Guggenheim Museum (2024)

Stephanie Dinkins. “Data Trust” at the Institute of Contemporary Art San José in 2025

The cognitive mechanisms in exquisite corpse pre-date the predictive processing of machine learning but we can learn from them nonetheless. Artists like Stephanie Dinkins are using AI to force an adaptive shift in real-time (human) mental modeling, through collaborative storytelling, memories and the natural environment.

In Data Trust, those community stories are stored in bacteria that grow alongside okra and California black oak trees — a poetic, literal merging of culture, memory, and environment. The text encoded into DNA can already be decoded back, imperfectly, into fragments of math, memory, and philosophy, or what Dinkins calls “a weird poem.” — AI Innovation Institute website

In 2024, Dinkins was publicly presenting demonstrations that were built on displaying interactions between humans and AI. This work evolved into “Data Trust,” a 2025 project that uses augmented intelligence to collect stories into an open dataset, one that might eventually influence how AI models learn. The project began with a question: How can people who are often underrepresented in data actively teach machines who they are? Dinkins a developed a custom mobile app called *The Stories We Tell Our Machines. *Every story shared, whether through text, voice, or conversation, fed an AI system that generated visual interpretations on large OLED screens.

Stephanie Dinkins. “If We Don’t Who Will” at the Broklyn Academy of Music in 2025

AI and machine learning face fundamental issues around what constitutes intelligence and how human intelligence actually works. AI solutions to date have shown discouraging levels of bias, and the creation of training data sets for AI and deep learning often relies on unethical and exploitative work practices. This is why projects such as “Data Trust” are critical. Augmented reality also faces ethical issues, ranging from data privacy to complex issues of digital rights in mixed reality environments. It is also important to consider equity and access: ***Is the technology available and accessible to all who need to use it?*** Publishing open datasets is a big step but people also need knowledge on how to make use of these datasets, as well as how to apply relevant knowledge and skills in the real world.

Medical research is using generative AI to revisit “Miller’s Pyramid of Clinical Competence,” which is a four-tier hierarchical framework created by Dr. George Miller in 1990 to assess clinical skills and knowledge. The model moves from foundational theory to real-world practice through the levels of Knows, Knows How, and Shows How.

• Knows focuses on pure recall of facts, information, and basic knowledge
• Knows How focuses on understanding how to apply and interpret knowledge to solve problems
• Shows How focuses on demonstration and performance in a controlled, artificial setting
• Does focuses on action and autonomous performance in actual, everyday practice

At every stage, a person must gain relevant knowledge and skills, and display the proper attitude for learning and doing. I think that there are sociocultural and cognitive dimensions to this process that need to be considered. My idea is to repurpose Miller’s framework (and GenAI) for other areas outside of medicine, such as art and design.

JXTA youth design cyphers and the Opportunity Crossing mural

In 2025, I worked with youth apprentices at Juxtaposition Arts or JXTA to come up with ideas for a community mural. I went into my TVC (techno-vernacular creativity) bag and facilitated a series of design cyphers that foster collaboration through brainstorming, concept mapping, and peer critique. The social dynamic of the design cypher is a repeating loop or algorithmic sequence where ideas (inputs) get shared, evaluated, and built upon in real time. Next, I pulled together the elements and created a few compositions for the JXTA team to consider.

Mural design artifacts from the JXTA design cypher

[embed]

The JXTA/Opportunity Crossing mural gave me a chance to engage youth and community members in the AI art making process and the community embraced the project. Generative AI was used to enhance the process. The apprentices learned how to use AI image generators to brainstorm and organize their thoughts.. We also had community input that helped drive the design process. In the future, I envision a longer, more involved process that scaffolds augmented intelligence (learning): the teacher shows or models how to do a task; the group practices the task together with the teacher; and the learners do the tasks by themselves.