DIGITAL ETHICS & CYBERSECURITY

The Dystopian Reality of “Viral Prompt” Trends

Exploring the link between viral AI trends and the rise of synthetic identity theft

source: YouCam AI Pro

“I only upload my photo to the AI apps and post it on socials just for fun like everyone else, because it’s viral right now..chill out!” Okay, but is it really safe?

By: Gabriela Z

Abstract

In recent years, we have seen a surge in consumer-facing Artificial Intelligence (AI) applications that require users to upload personal imagery, which often features their own faces or, worse, their children’s faces, to generate stylized digital content. While marketed as harmless entertainment, did you know that these platforms facilitate the mass voluntary disclosure of sensitive biometric data?

This research critiques the “gamification” of data harvesting, arguing that the immediate social gratification of AI-generated content creates a “Privacy Paradox” that blinds users to long-term security risks. It argues that the ‘gamification’ of AI tools creates a deceptive environment where users (which often driven by social trends) unknowingly contribute to permanent surveillance databases, compromising not only their own digital security but also the future privacy of minors through proxy consent.

By examining the lifecycle of uploaded imagery, this paper explores how such data is integrated into permanent training databases, potentially fueling future deepfake technology, synthetic identity theft, and unauthorized surveillance. Ultimately, this research calls for a re-evaluation of digital literacy frameworks to address the hidden costs of the AI-driven “fun” economy.

Keywords: Artificial Intelligence, Biometric Data, Privacy Paradox, Surveillance Capitalism, Deepfakes, Digital Ethics.

Introduction

The digital landscape is currently defined by the “viral cycle”, a phenomenon where millions of users flock to Artificial Intelligence (AI) applications to transform their likenesses into artistic avatars, historical figures, or hyper-realistic fantasies. From “AI Yearbooks”, the viral “ask ChatGPT to define myself” challenge, to aging filters, these tools leverage sophisticated generative adversarial networks (GANs) to provide instant aesthetic gratification. For the average user, the transaction is viewed as a harmless exchange of an image for a moment of social engagement.

However, beneath this veneer of “fun” lies a significant ethical and security crisis. This research argues that these applications exploit the privacy paradox, where users’ desire for social validation and digital belonging overrides their rational concerns regarding data security. By gamifying the submission of biometric data, AI developers have successfully normalized the mass harvesting of facial geometry, one of the most sensitive and permanent forms of personal identification.

This paper will examine how voluntary disclosure fuels the “biometric databases” utilized by bad actors for social engineering, deepfake-based scams, and synthetic identity theft. By dismantling the illusion of the “harmless filter,” this paper seeks to advocate for a more rigorous standard of digital literacy and corporate accountability.

Methodology

This mini-research utilizes a qualitative case study design combined with Critical Discourse Analysis (CDA) to evaluate the operational friction between user intent and data harvesting. The study selects two historically viral AI platforms (ChatGPT and Gemini) as primary case studies.

Discussion and Findings

In 2022, the concept of chatbot began to be recognized by the general public as a smart tool to assist us in working and learning. The system functions like a “human” but is smarter than a typical search engine. While it normally takes us at least 15 minutes to find accurate information using Google, Reddit, e-books, and various websites, with the ChatGPT chatbot, we only need to wait approximately 1 minute to receive an instant answer. Due to this sophistication, in less than a month (specifically in December 2022), ChatGPT reached about 1 million users, marking the beginning of its explosion in popularity.

Its popularity continued massively throughout early 2023, becoming the fastest-growing application at the time. ChatGPT became popular due to its advanced ability to respond to various commands, ranging from answering questions and writing programming code to generating creative content quickly and in a manner resembling human conversation. Also in 2023, Gemini AI was released with the same concept: a smart chatbot just like ChatGPT.

An increasing number of people use these chatbots not only to obtain information instantly, but many also use them to consult and “chat” using everyday language, just as they would with a friend. It is also very common for people to use them for physical and mental health consultations. OpenAI released data showing that a small fraction of ChatGPT users utilize the chatbot to discuss mental health issues. Approximately 0.15% of its more than 800 million weekly users show signs of having suicidal plans, which translates to over one million people per week. OpenAI also noted instances of users exhibiting high emotional attachment, as well as symptoms of psychosis and mania. Although these cases are relatively rare, the company acknowledges their significant impact and is committed to strengthening security systems and enhancing ChatGPT’s capability to respond to users with sensitive conditions.

Viral Trends

Beyond serving as a place to vent, chatbots like ChatGPT and Gemini have also become mediums for people to create art. The use of chatbots for art encompasses creating visual works (AI Art), writing creative narratives, and even generating professional documents for artists. The chatbot acts as a virtual assistant that helps artists overcome creative blocks, compile portfolios, and generate visuals instantly using text prompts. Chatbots are equipped with the ability to transform text descriptions (prompts) into artwork. You can request the AI to produce a specific painting style (for example: realism, surrealism, or even Ghibli animation), generate design assets, or create content thumbnails.

This very phenomenon eventually gave rise to numerous ChatGPT “prompt” trends that went viral on the social media platform TikTok. One trend that gained massive popularity in 2024 was the “Ghibli” trend, where transforming photos into images in the style of Studio Ghibli became highly sought after by social media users worldwide. This Japanese animation studio was founded by Hayao Miyazaki and is renowned for legendary films such as Spirited Away and My Neighbor Totoro. The creation process is quite simple: users only need to upload their original photo with a clear view of their face, and then use the prompt, “Please change my photo into a Ghibli cartoon style.” In less than 5 minutes, the image is finished and ready to be “distributed” on social media.

source: https://voidslash.com/

Although the process is easy and appealing, this trend has actually sparked debates regarding potential copyright infringement. Some parties argue that the AI merely mimics an art style, which is not protected by copyright, while others highlight the legal foundations of this practice.

Evan Brown, a partner at the law firm Neal & McDevitt, assesses that the AI generates work based on an art style rather than a specific expression, which is what copyright typically protects. “Copyright law generally protects specific expression, not an art style itself,” he stated. OpenAI itself has not provided a response regarding the data used to train its AI models or the legality of this latest feature.

Then, in early 2026, the ChatGPT “Caricature of Me and My job” trend circulated and went highly viral on social media. To join the viral trend, you only need to upload a clear photo of yourself to ChatGPT and use this exact prompt: “Create a caricature of me and my job based on everything you know about me”. The AI will synthesize its memory of your interactions to generate a custom cartoon that reflects your profession and personality.

source: https://www.thenewdaily.com.au/

Another trend that recently went viral on social media involves creating Polaroid photos featuring celebrities, crushes, or deceased friends/relatives/family members together in a single photograph. This trend became highly viral because many users were extremely satisfied with the results, which appeared incredibly real and did not look edited at all. They simply upload their photo to Gemini AI and use the prompt: “Take a photo taken with a Polaroid camera. The photo should have a slight blur and a consistent light source, like a flash from a dark room, scattered throughout the photo. Don’t change the faces. Change the background behind the two people to a white curtain. With both people hugging each other.”

source: gemini.ai

With so many proliferating trends on social media, from a privacy perspective, is all of this actually safe?

The Illusion of Ephemeral Engagement

The consumer-facing interface of viral AI applications is the primary infrastructure for data ingestion, relying heavily on behavioral psychology and deceptive design to facilitate rapid data disclosure. The user interface (UI) is structured to mimic a frictionless, low-stakes game rather than presenting data submission as a formal, high-stakes privacy transaction

When a user uploads a face to participate in a viral trend (such as the Studio Ghibli or Polaroid filter), the UI typically displays a rapid loading bar followed immediately by the final, stylized artwork. Some apps even include text notes on the screen stating, “Photos are processed safely”. This clean, immediate visual output tricks the user into perceiving the transaction as a closed loop:

Image In --> Filter Applied --> Image Out.

The UI completely isolates the user from the backend reality. There are no visual indicators, warning screens, or progress trackers that inform the user that their facial geometry has just been mapped, extracted, and synchronized into an external, permanent corporate database for machine learning model training.

The Downstream Consequences: Cyber Exploitation

The mass, voluntary disclosure of facial geometry through viral AI applications has fundamentally altered the economics of cybercrime. Historically, gathering high-quality, verified personal assets for social engineering required targeted phishing or extensive open-source intelligence (OSINT) harvesting. Today, consumer-driven AI trends act as a crowdsourced data utility for malicious actors.

When users upload high-resolution, multi-angle close-ups of their faces to generate “Caricatures” or “Ghibli” avatars, they are handing over the exact structural data points (interpupillary distance, jawline curvature, orbital metrics) used by security algorithms. In this modern era, modern financial and corporate systems heavily rely on “Know Your Customer” (KYC) protocols and biometric face verification for account creation and access. This means that every time you feed your face to a viral AI app for a quick laugh, you are voluntarily giving away the very biometric key that locks your bank account.

source: https://www.researchgate.net/

Cybercriminals utilize specialized deepfake software injected directly into virtual camera streams to bypass automated liveness checks. Armed with a victim’s extracted facial map and leaked personal identifiable information (PII) from traditional data breaches, syndicates can execute seamless account takeovers or open fraudulent lines of credit, effectively rendering traditional biometric security obsolete. High-fidelity facial coordinates are the precise engine required to train Generative Adversarial Networks (GANs).

Scammers also harvest these readily available social media “trends” to generate non-consensual deepfakes. In early 2024, a multinational firm in Hong Kong lost $25 million USD after an employee was deceived by a multi-person videoconference call consisting entirely of deepfaked likenesses of his coworkers and the Chief Financial Officer. The raw assets needed to construct these convincing, moving digital puppets are exactly the types of clear, multi-angle facial uploads required by viral “Caricature” or “Ghibli” trends.

As highlighted by the viral trend of recreating Polaroid photos with deceased relatives, the emotional manipulation of generative AI is highly effective. Cybercrime syndicates systematically exploit this by pairing facial imagery with voice cloning to execute devastating social engineering attacks. In a typical “Virtual Kidnapping” or emergency scam, a bad actor calls an elderly relative using a cloned voice of their child or grandchild, claiming they have been arrested, kidnapped, or hospitalized.

To make the extortion convincing, criminals require contextual verification. By monitoring public social media feeds where users post AI-generated family reunions or “Caricatures of my job,” attackers gain immediate intelligence regarding family structures, professional settings, and emotional pain points. If a scammer can send a hyper-realistic, AI-modified image or video via WhatsApp during the fraudulent call to “prove” the emergency, the psychological compliance of the victim reaches near 100%, resulting in immediate wire transfers or cryptocurrency ransoms.

This is really dangerous, but why do people still want to do it?

Viral Mimicry and the Social Cost of Non-Participation

While dark patterns handle the internal manipulation within the app, the external catalyst that drives users to the platform is the weaponization of the “Fear of Missing Out” (FOMO). In the context of digital sociology, viral trends such as the 2024 “Ghibli filter,” the 2026 “Caricature of Me,” or the “Deceased Loved Ones Polaroid” trend do not exist in a vacuum. They function as highly visible markers of cultural currency on platforms like TikTok and Instagram.

This social dynamic overrides privacy concerns through two primary mechanisms:

1. The Threshold of Social Proof: When a user opens their social media feed and sees peers, influencers, and public figures participating in a specific AI trend (e.g., “flexing” their Ghibli avatars), a powerful psychological shift occurs. According to the theory of Social Proof by psychologist Dr. Robert Cialdini, individuals look to the behavior of others to determine correct actions. When participation becomes ubiquitous, the baseline of normal behavior shifts. Non-participation is no longer seen as a neutral choice; it is perceived as social exclusion or being “out of touch.”
2. The Immediate vs. Delayed Risk Asymmetry: FOMO creates an intense, immediate psychological penalty (the anxiety of missing a viral cultural moment right now). Conversely, the threat of biometric data harvesting is highly abstract, invisible, and delayed (the potential of a scammer using your face years from now). Human psychology is fundamentally flawed at calculating risk when faced with immediate social rewards versus distant, theoretical consequences. Therefore, users choose the immediate gratification of social alignment and deliberately ignore the vague warning signs of data insecurity.

Ultimately, the desire to belong to the digital collective transforms user behavior from cautious to compliant. The user rationalizes the risk with thoughts like “everyone else is doing it, so it must be fine,” or “chill out, it’s just viral right now.” AI developers capitalize on this social vulnerability, relying on peer pressure to act as a free, self-replicating marketing campaign that continuously feeds high-quality facial data into their corporate ecosystems.

The Architecture of Behavioral Modification

To fully comprehend the systemic mechanics behind voluntary biometric disclosure, it is essential to analyze these viral phenomena through the lens of Shoshana Zuboff’s theory of Surveillance Capitalism. Zuboff defines this economic mutation as the unilateral claiming of private human experience as free raw material for translation into behavioral data. Traditionally, mass surveillance systems, such as those described in classic dystopian literature or implemented by authoritarian states, rely on coercion, state control, and the instillation of fear to enforce compliance and extract information.

The brilliance (and inherent danger) of contemporary surveillance capitalism, however, lies in its complete inversion of this power dynamic. It does not force compliance; it seduces it. As observed in the proliferation of viral AI photo trends, consumer-facing tech entities do not need to deploy authoritarian tactics to seize citizens’ biometric markers. Instead, they weaponize psychological vulnerabilities, specifically peer conformity, digital mimicry, and the Fear of Missing Out (FOMO), to transform a highly invasive data-harvesting process into a coveted cultural ritual.

Through this framework, the user is manipulated into a state of active supplication. Driven by the acute anxiety of digital erasure and the desperate need to maintain cultural currency within their social feeds, users effectively “beg” to surrender their most permanent and sensitive biometric data. The transaction is cloaked in the rhetoric of self-expression and community participation. When a user uploads a high-resolution image of their face or their child’s face to generate a Ghibli-style avatar or a nostalgic Polaroid, they do not perceive themselves as exploited digital laborers. They view themselves as active participants in a shared cultural moment.

Consequently, surveillance capitalism successfully shifts the burden of data collection from the corporation to the individual. The company no longer has to surreptitiously mine for data; the user willingly organizes, optimizes, and delivers high-fidelity facial templates directly into corporate databases, entirely free of charge. By exploiting the human desire for social belonging, AI developers have achieved the ultimate capitalist efficiency: converting a profound threat to long-term biosecurity into an ephemeral, celebrated social game.

Conclusion

This mini-research has deconstructed the deceptive dynamics of the contemporary “fun” economy in consumer-facing generative AI applications. By analyzing viral social media trends such as the Studio Ghibli filters, personalized caricature generators, and deepfaked familial reunions, this study exposed the profound friction between casual user intent and systemic data harvesting.

The findings demonstrate that users are not merely engaging in harmless entertainment. Instead, they are being systematically seduced by weaponized FOMO and frictionless user interfaces into becoming unpaid digital laborers. By voluntarily surrendering high-fidelity facial geometry, users are continuously fueling centralized corporate databases and, by extension, providing the raw materials required by global cybercrime syndicates.

To mitigate the rapid, unregulated extraction of consumer biometric data, a multi-stakeholder intervention is required. Relying solely on individual user vigilance is insufficient when facing highly engineered dark patterns. The following structural interventions are recommended:

1. Before an AI application can access a device’s camera roll or process a facial upload, the UI must display a standardized, high-contrast warning screen. This screen should explicitly state, in non-legalistic language, whether the facial data will be retained for machine learning model training or shared with third parties.
2. Consent to a platform’s general Terms of Service must legally be decoupled from consent to biometric processing. Users must be allowed to use an app’s basic functionalities without being forced to forfeit their permanent biometric rights.
3. Prohibition of Minor Data Ingestion. If a minor’s face is detected, the application should be legally prohibited from syncing that structural data to external cloud databases.

Educational institutions and public cyber-awareness campaigns must modernize their curricula to address the realities of generative AI. Digital literacy can no longer just focus on avoiding phishing links; it must teach citizens to calculate the lifetime value of facial geometry. Public awareness must emphasize that participating in a transient, five-minute social media trend carries a permanent digital cost.

References: Hong Kong Deepfake Financial Fraud Case (February 2024). Widely reported by global cybersecurity agencies and news outlets.

Zuboff, S. (2019). The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. PublicAffairs.

Acquisti, A., Brandimarte, L., & Loewenstein, G. (2015). Privacy and human behavior in the age of information. Science, 347(6221), 509–514.

Support writers. All our non-profit’s offerings here.

Click here to subscribe to the IT Chronicles newsletter.

Click for Wordsmith, Mystery Writing, Write Like Stephen King, more

By the EIC Susan Brearley with Ideogram