In-Context Representations of Quantitative Scales in LLMs

Have you ever asked an LLM to grade an article you wrote? Or rate your dinner on a scale from 1 to 10?

Asking an LLM to provide a numeric rating requires a type of integration of information that goes beyond constructing a grammatically valid sentence. First, the LLM must have the necessary knowledge to provide a rating. Additionally, a number by itself is not a valid rating, it has to be described relative to the set of available options. For example, a grade of 10 has a different meaning if it’s on a scale from 1–10 compared to a scale from 1–100. An evaluation space needs to be properly mapped out in order to be able to perform a grading task. In this short post, we explore the efficacy of quantitative rating scales in LLMs.

Key points:

• A next-word prediction task requires the model to — at the very least — adhere to grammar rules and the natural language statistical patterns in sentences.
• A value judgement needs to be made about the concept in the sentence to map it into the evaluation space.
• The model must make a value judgment about the concept in the sentence to successfully map it into the evaluation space (e.g., 1–5, 0–100%, F to A+).

Representation of latent evaluation space

Language-based value judgements

The simplest of value judgements are those where the meaning of the word encodes the gradation (e.g., Bad, Good, Best). Models could learn valuations like “bad”, “good”, “best” based on contexts. Sentences with negative language lead to the use of lower-valuation words. These attributions are useful in a qualitative way but they are not inherently quantifiable. Additionally, it is not straightforward to quantify the degree of improvement from ‘Good’ to ‘Great’ or from ‘Great’ to ‘Excellent’.

Similarity Analysis Using Embeddings

We generate approximately 1,500 example sentences, each incorporating a different language-based value judgements. An example would be“the quality of the [written essay] is [bad]”, where 30 different topics, 5 different phrasings and 10 quantitative judgements are generated. (As per comparison, we also generated 12 language-based value judgements, as seen in Figure 1.) These sentences are then converted to embeddings using various embedding models (OpenAI, Gemini, Cohere, and open source models from huggingface) and results in a single vector for each sentence. Pairwise vector differences can be computed using cosine similarity, creating a dissimilarity matrix, which we can visualise using Multi-Dimensional Scaling*.

We map out the similarities of language-based value judgements in Figure 1 and learn that there is a general trend from bad to good, with a couple of non-linear relationships. For example, we observe that ‘terrible’ and ‘dreadful’ have similar meanings, with ‘dreadful’ being slightly more negative. ‘Acceptable’ and ‘decent’ exhibit a similar degree of dissimilarity from both ‘good’ and ‘mediocre’.

Figure 1: Language-based value judgements. The distance between dots represents the similarity between two terms. Generated using the OpenAI text-embedding-3-large embedding model.

We can apply the same principle to numeric scores, see Figure 2. For this purpose, we re-write the example sentences to include a score with a maximum of 10 (e.g., “x out of 10”, “x/10”, or “x on a scale from 1 to 10”). The resulting embeddings are distributed in a more linear sequence than those of the qualitative judgments but also contain minor distortions. Firstly, some numbers appear unusually similar to others. The number ‘1’ is more similar to ‘5’ and ‘10’ than expected, which causes ‘5’ to be slightly displaced from its expected sequence. This could be due to various semantic factors. Secondly, we observe that the distance between numbers is not always constant; the difference between 6–7 and 7–8 appears smaller than that of 9–10.

Figure 2: Dissimilarity between quantitative ratings. The distance between two dots represents the dissimilarity between the two numbers. Generated using the OpenAI text-embedding-3-large embedding model.

Comparing performance between models.

If the model were to represent all 10 numbers on a scale from 1–10 in a perfectly linear way, the dissimilarity matrix would exhibit a perfectly linear pattern. We can evaluate the dissimilarity matrices of various embedding models by measuring their deviation from this ideal model. As shown in Figure 3, we see that performance varies but the top performing models are very close to strictly linear representations of the in-context representations of the numbers 1–10.

Although a perfectly linear representation serves as a valid benchmark, it may not be a completely realistic objective. Psychological literature suggests that humans perceive numbers at the extreme ends of scales as having larger perceptual differences.

Figure 3: Alignment of embedding models to a perfectly linear representational space of the numbers 1 to 10. The (upper triangle of the) dissimilarity matrices were correlated against an ideal dissimilarity matrix which assumes a linear progression from scores ranging from 1 to 10. The OpenAI models show highest alignment to the linear representation.

Conclusion

LLMs reliably represent various rating scales in their embedding spaces.

While top models exhibit global accuracy, their numeric representations can be biased due to alternative meanings of words or numbers when these are used in different contexts. This may also reflect human biases present in the training data. By accounting for biases in quantitative scales, these models can be effectively applied to quantitative evaluations. How do you think LLMs can learn these representations?

*For more information on these scaling methods see Hajonides et al. 2021 Neuroimage.