The Digital Version of Brainstorming: How Big Language Models Work?

Artificial intelligence has become an indispensable part of our daily lives, especially chatbots and text-generating systems. But how is it that these “intelligent” systems can communicate so naturally with us, write poetry, and even answer our complex questions? Let’s take a closer look at the brainstorming-like working principles of Large Language Models (LLM).

How do LLMs Learn?

The main task of an LLM is to “model the probability distributions of words or tokens in a sequence”, just as a poet dances with words. We can liken this to a child reading a storybook: He tries to predict what the next word might be by looking at the previous words. This predictive ability is so well developed that the resulting texts can be surprisingly fluent and meaningful. However, this “autoregressive” nature, in which each word is actually predicted based on the words that preceded it, can make it a bit time-consuming to create long texts.

Generated from Whisk

So how does this prediction process work? This is where tokenization comes into play. Texts are broken down into “tokens”, small snippets that LLMs can understand. Tokens are more general than words and help to deal even with things like typos. Think of it this way, like the vaults of a word bank, each token has its own unique identity and the model learns how the same token is used in different contexts (for example, the word “bank” can mean both a financial institution and a riverbank). These tokens are passed through a neural network (most often a Transformer) into a meaningful representation and the model predicts the probability distribution of the next word.

What about the Transformer Architecture?

The architecture at the heart of LLMs is a specialized neural network structure called a Transformer. We can liken this architecture to the embankments that regulate the flow of water in the bed of a river. Just as the water is divided into different tributaries, focusing on different areas in each tributary, the Transformer weights the importance of different words in the text through a “Self-Attention” mechanism. This allows the model to focus on the relevant words in the whole sentence, not just the words immediately next to it, when predicting a word. Just like the gradient that determines the direction of a river’s flow, the “Gradient Descent” algorithm optimizes the learning process by reducing the errors (loss) in the model’s predictions. This is how almost all modern AI models learn.

Personalizing LLMs: Tailoring Fine-Tuning

An LLM completes his or her “basic training” by doing a general language modeling on huge internet texts. But this general knowledge model is not enough to converse with us or to specialize in a particular field. This is where “post-training” and “fine-tuning” come in. We can liken this to a general tailor’s machine being tuned to sew a specific suit.

• Supervised Fine Tuning (SFT):

In this phase, the model is trained on smaller, high-quality datasets to learn specific output formats, such as question-answer formats, much like an apprentice being tutored by a master. Rather than adding new knowledge to the model, SFT shapes its existing knowledge so that it acquires the ability to answer in a certain “helpful assistant style”. But beware: SFT with too little data can cause the model to hallucinate the correct information.

• Preference Alignment:

This is like the final touch after the SFT. It is used to make the output of the model more in line with human preferences. Just like a tailor revises a dress according to his client’s final wishes, at this stage the model learns to produce better responses with feedback given by the human or other LLMs.

Generated from Whisk

When fine-tuning, you can retrain all the parameters of the model (Full Fine-Tuning), which requires a lot of resources. Or you can use Parameter Efficient Techniques, which is a smarter way. For example, with LoRA (Low-Rank Adaptation), instead of retraining the entire model, you train far fewer parameters by simply adding small “adapter matrices”. This is like optimizing only a few important parts of a large factory instead of replacing all the machines!

Speed and Elegance: The Efficiency of LLM Inference

The knowledge generation process of LLMs, i.e. “inference”, can be quite costly. Therefore, many ways have been developed to speed up this process.

• Quantization: Just like reducing the file size of a photo by reducing its quality, this technique reduces memory and computational costs by converting the model’s weights into lower bit-widths.
• Sparsification: This reduces the size of the model by removing unnecessary “zero” values from the model. It’s like removing all the unnecessary bolts from a car engine, but in a way that the engine still runs smoothly!
• Speculative Decoding: This super cool technique predicts several future words using a smaller and faster “draft model”. The main model checks these predictions in a single step. If the prediction is correct, it moves forward quickly; if it is wrong, it slows down and corrects it. You can think of it like a chess player predicting moves and moving quickly.
• Retrieve-Augmented Generation (RAG): This is one of the most popular ways of adding new information to LLMs. It allows the model to access external, up-to-date and accurate information. For example, a chatbot might fetch information directly from a news site when you ask for the latest news. This also reduces the risk of the model “hallucinating”.

Firewall: Risks and Defense Strategies in LLMs

Like any powerful technology, LLMs have vulnerabilities. No matter how robust a castle is, it can still have weak points.

• Prompt Injection: A malicious user can insert commands into the input that override the model’s normal instructions. This can cause the model to exhibit unwanted or harmful behavior.

Source: nvidia

• Jailbreaking: These techniques allow the model to bypass built-in security measures to produce inappropriate or unethical content that it would normally reject.

Various strategies are being developed to defend against such attacks. For example, using multilingual training data or “Red Teaming”, which tests vulnerabilities with attacker simulations. To strengthen the castle’s defenses, both fortifying its walls and conducting continuous drills!

Big Language Models have many complex components, ranging from pre-training to fine-tuning, inference efficiency to security measures. Metrics such as “perplexity” and “scaling laws” are used to evaluate the performance of these models. These laws allow us to understand how model performance changes as model size, amount of data and computational power increase.

LLMs are also an important milestone in human history. Language has always been at the center, ever since humans began to write down their thoughts. Now artificial intelligence is bringing this power to the digital world.

So it’s no longer just “how does it work?” but “how will the relationship between humans and AI evolve?” And the answer to this question will become clearer every day as LLMs continue to evolve.