How AI Actually Works Under the Hood: From Your Prompt to the Final Response

Most people use AI every day.

They ask ChatGPT to explain a concept, generate code, write emails, solve bugs, or even help with cybersecurity tasks.

But have you ever wondered what actually happens after you type a prompt and hit Enter?

As someone learning cybersecurity and AI, I wanted to understand what happens behind the scenes instead of treating AI like a magic box.

In this article, we’ll follow the complete journey of a prompt through modern AI systems and understand the network services, APIs, models, and infrastructure that make everything work.

The Big Picture

When you type a prompt, the request goes through several layers before a response is generated.

{
  "prompt":"What is SQL Injection?"
}

Simple from the outside.

Very complex on the inside.

Step 1: You Enter a Prompt

Let’s say we ask:

“What is SQL Injection?”

The frontend (browser or application) sends this request to an API server.

The request may look something like:

POST /chat
Content-Type: application/json
{
  "prompt":"What is SQL Injection?"
}

The API server receives the prompt and prepares it for the AI model.

Step 2: Tokenization

AI models don’t understand words directly.

They understand numbers.

Before processing the prompt, the text is broken into smaller pieces called tokens.

Example:

Hello World

May become:

[15496, 2159]

A longer sentence becomes hundreds of tokens.

Think of tokenization as translating human language into a language the model can understand.

Step 3: Embeddings

Now the tokens are converted into vectors.

A vector is simply a list of numbers.

For example:

"cat"

may become:

[0.12, -0.55, 0.82, ...]

Words with similar meanings end up close to each other in vector space.

For example:

Cat
Dog
Tiger
Lion

are usually closer together than:

Cat
Database
Firewall

This allows AI models to understand relationships between concepts.

Step 4: The Transformer

This is where the real magic happens.

Modern AI systems use a neural network architecture called the Transformer.

The Transformer introduced a concept called Attention.

Attention allows the model to determine which words are important when generating the next token.

Consider:

The cat sat on the mat.

When processing the word “sat”, the model pays more attention to “cat” than unrelated words.

This ability to focus on relevant information is what made modern AI possible.

Step 5: Predicting the Next Token

Contrary to popular belief, AI does not “think” like humans.

It predicts the most probable next token.

Example:

The sky is

Possible predictions:

blue  -> 80%
gray  -> 10%
red   -> 5%
other -> 5%

The model chooses a token.

Then predicts the next token.

Then another.

And another.

This process repeats extremely fast until a complete response is generated.

Step 6: GPUs Do the Heavy Lifting

Training and running large AI models requires enormous computational power.

This is where GPUs come in.

GPUs are designed to perform thousands of calculations simultaneously.

Without GPUs:

• Training would take years
• Responses would be extremely slow
• Modern LLMs would not be practical

What Happens on the Network?

This part is especially interesting from a cybersecurity perspective.

Many people focus on the AI model itself but forget the surrounding infrastructure.

AI systems rely heavily on APIs and network services.

A typical setup might look like this:

User
 |
443
 |
Frontend
 |
8000
 |
Backend API
 |
11434
 |
LLM Server
 |
GPU

Each layer communicates using network protocols.

Common Ports Used in AI Environments

When building AI labs or AI-powered applications, you’ll often encounter these ports:

ServiceDefault PortHTTP80HTTPS443Flask5000FastAPI8000Jupyter Notebook8888Ollama11434Open WebUI3000PostgreSQL5432Redis6379Elasticsearch9200Prometheus9090

Understanding these services is important for both developers and security professionals.

AI From a Pentester’s Perspective

As security professionals, we should not only understand how AI works but also how it can be attacked.

Common attack surfaces include:

1.Exposed AI APIs

Attackers may interact directly with model endpoints.

2.Prompt Injection

Manipulating model behavior using crafted prompts.

3.Insecure Tool Access

AI agents with excessive permissions.

4.Exposed Vector Databases

Sensitive embeddings and proprietary data.

5.Publicly Accessible Jupyter Servers

A surprisingly common misconfiguration.

6.Exposed Ollama Instances

Many local AI deployments accidentally expose their model APIs.

Why Understanding the Architecture Matters

Many people are learning how to use AI.

Far fewer people understand how AI actually works.

As cybersecurity professionals, understanding the underlying architecture helps us:

• Build better AI applications
• Secure AI environments
• Identify attack surfaces
• Perform AI security assessments
• Understand future technologies

The next generation of security engineers won’t just secure web applications and networks.

They will secure AI systems too.

And that journey starts with understanding what happens after you hit Enter.

Final Thoughts

AI isn’t magic.

Behind every response is a chain of APIs, tokenization systems, transformer networks, GPUs, databases, and infrastructure working together in milliseconds.

The more I learn about AI, the more I realize that understanding the fundamentals is far more valuable than simply using the tools.

Whether you’re a developer, cybersecurity student, pentester, or AI enthusiast, learning how these systems work under the hood will give you a significant advantage in the years ahead.

Because you can’t secure what you don’t understand.

Tags: Cybersecurity Networking Penetration Testing Infosec

— Written by

Aruvasaga Chithan A

Ethical Hacker & Cyber Security Researcher.

Thanks for reading — your support keeps me writing. See you in the next article…

*Linkedin.*