K-Nearest-Rank Scores: A Tool for Local Structure Analysis in Embeddings

When using a low-dimensional projection, we should always be aware that it is merely a projection of a higher and more complex dimensional space. To truly trust these projections, we need to assess how well they preserve the original sample distribution.

Photo by Saint Rambo on Unsplash

Dimensionality reduction techniques such as PCA, t-SNE, and UMAP are widely used for both feature reduction and visualizing data in lower-dimensional spaces. Being able to visually explore how samples are distributed can reveal key insights that might otherwise remain hidden. However, before relying on these projections, it’s important to ask a few critical questions: 1. How well does the reduced-dimensional space represent the original high-dimensional structure? 2. How consistent are the embeddings generated by PCA, t-SNE, and UMAP with each other? In this blog, I’ll demonstrate how to quantify the similarity between different projections using the K-Nearest-Rank Similarity score (KNR-score).

If you like the content, please give it an applause, and follow me to read my blogs! Also, try the hands-on examples. This will help you to learn quicker, understand better, and remember longer.

A brief introduction

The primary goal of feature extraction methods such as PCA, t-SNE, and UMAP is to reduce the dimensionality of datasets in a way that the low-dimensional representation remains a faithful approximation of the original high-dimensional space.

Broadly speaking, there are two types of dimensionality reduction techniques: linear and non-linear transformations. Linear methods, such as Principal Component Analysis (PCA) construct a new set of dimensions or latent variables by forming linear combinations of the original features. Non-linear methods, on the other hand, such as t-distributed Stochastic Neighbor Embedding (t-SNE), aim to preserve local relationships between samples, even if this comes at the expense of accurately representing global (dis)similarities. As a result, t-SNE is particularly effective at preserving local structures, since it is less influenced by large dissimilarities elsewhere in the dataset.

A major advantage of non-linear mappings is their suitability for low-dimensional representations and data visualization. Linear methods like PCA, by contrast, are typically used to reduce model complexity, enhance computational efficiency, identify important features, and mitigate the curse of dimensionality. However, PCA can also be quite informative for visualization, especially when combined with interpretative tools such as PCA loadings. For more information, I recommend reading the following blog [1] for more details about interpreting PCA loadings.

[embed]What are PCA Loadings, and How To Effectively Use Biplots? A practical guide for getting the most out of Principal Component Analysis.medium.com

K-Nearest-Rank Similarity Score (KNR-score) Methodology.

When mapping high-dimensional data to lower dimensions, distances between similar samples must be well preserved instead of the overall distribution of samples. But how can we quantify the differences between the projections? Or in other words, we want to project samples relative to each other in the low-dimensional representation as seen in the original (high) dimensional space. Note that such an approach is also great for visual inspections because it can help to discover relationships between samples but it thus needs to be accurate. To compare the embedding of samples in two different projections, such as high dimensionality vs. low dimensionality or t-SNE 2D-map vs. PCA map, we can quantify the local similarities across two maps based on a scale-dependent similarity measure. This quantification approach was originally described in this paper [2] and has the following steps to quantify the sample-wise similarity between two maps:

1. Compute the Euclidean distances between the samples within each map.
2. Rank sample-wise using the Euclidean distance. Samples with the smallest distances are on top.
3. Compare the ranks of map X to the ranks of map Y for kX and kY nearest neighbors.
4. Quantify the overlap between the ranks.

The final output is a matrix that describes how similar the maps are for the k-nearest neighbors. The score is a measure that describes the sample-wise similarity between the maps. A perfect overlap between two maps would result in a score of 1, meaning that all k-nearest neighbors of one map are similar to the other map. We can then color the matrix reddish for high values and blue-ish for low values. Such visualization can help to quickly interpret the comparison of the maps.

Figure 1. Schematic overview to systematically compare local and global differences between two sample projections. Image from the author.

Let’s talk a bit more about the scoring method. The score is based on the kx and ky nearest neighbors of each sample. In the first step are the ranks are computed for sample i in map X and are compared to the j neighbors; rxij. The first nearest neighbor of sample i will have rank 1, the second nearest neighbor rank 2, etc. Analogously, ryij is the rank of sample j with respect to sample i in map Y. Now we can compute a score on the interval [0, 1] (see equation 1), where the variable n is the total number of samples, and the indicator function is given by equation 2.

Equation 1.

Equation 2.

The score Sx,y(kx, ky) will have a value of 1 if, for each sample, all kx nearest neighbors in map X are also the ky nearest neighbors in map Y or vice versa.

Comparison of Feature Extraction Methods.

To better understand the underlying statistics, let’s directly proceed with a hands-on example where I will use the mnist dataset and create embeddings using PCA, t-SNE, and UMAP. The question that we will answer is: how “similar” are different maps in terms of the sample distribution? First, we need to install a KNRscore library that will do the heavy lifting to quantify the similarities between the maps. The output is a matrix of scores that we can use to create insightful plots.

pip install KNRscore

# Load general libraries
import numpy as np
from sklearn import manifold, decomposition
from umap import UMAP
# Import library
import KNRscore as knrs

# Load mnist example data
X, y = knrs.import_example(data='digits')

# PCA: 50 PCs
map_pca = decomposition.TruncatedSVD(n_components=50).fit_transform(X)
# tSNE: 2D
map_tsne = manifold.TSNE(n_components=2, init='pca').fit_transform(X)
# UMAP: 2D
map_umap = UMAP(densmap=True).fit_transform(X)
# Random
map_rand=np.c_[np.random.permutation(map_tsne[:,0]), np.random.permutation(map_tsne[:,1])]

# Scatter
fig, ax = knrs.scatter(map_pca[:,0], map_pca[:,1], labels=y, s=75, title='PCA')
fig, ax = knrs.scatter(map_tsne[:,0], map_tsne[:,1], labels=y, s=75, title='t-SNE')
fig, ax = knrs.scatter(map_umap[:,0], map_umap[:,1], labels=y, s=75, title='UMAP')
fig, ax = knrs.scatter(map_rand[:,0], map_rand[:,1], labels=y, s=75, title='Random')

First, we reduce the dimensionality of the Digit dataset using PCA, t-SNE, and UMAP. The 2D plot is shown in Figures 2 (A-D). T-SNE and UMAP show a better separation of the classes compared to PCA. However, we can only see the first 2 dimensions, but PCA has 50 dimensions under the hood. For visualization purposes, we are limited to only plotting the first 2 (or 3) dimensions. We can thus only visually observe that various classes seem to be well separated; it is, however, impossible to compare the exact sample distribution with another map just by eye.

Figure 2. Scatterplot of the 4 projections using the MNIST dataset. (A) PCA. (B) t-SNE (C) UMAP, (D) Random. Image from the author

Create an Interactive Scatterplot For Better Understanding.

With the D3Blocks library [3], we can create an interactive scatterplot to visually see how samples are distributed between the different projections. More details about D3blocks can be found in this blog. See the code block below to create such scatterplots. In the next section, we will quantify the similarity between the data points. For more details about D3blocks, see this blog for in depth details:

[embed]D3Blocks: The Python Library to Create Interactive, Standalone, and Beautiful D3.js Charts Bring your charts alive using D3Blocks: The Python Library to Create Interactive, Standalone, and Beautiful D3.js…medium.com

pip install d3blocks

# Create interactive scatter plot
from d3blocks import D3Blocks

# Initialize
d3 = D3Blocks()

# Make scatter
d3.scatter(map_pca[:,0],
           map_pca[:,1],
           x1=map_tsne[:,0],
           y1=map_tsne[:,1],
           x2=map_umap[:,0],
           y2=map_umap[:,1],
           label_radio=['PCA', 't-SNE', 'UMAP'],
           scale=True,
           tooltip=list(map(lambda x: 'Sample_id: '+x, np.arange(0, map_pca.shape[0]).astype(str))),
           color=y.astype(int).astype(str),
           filepath='scatter_embeddings.html')

Figure 3. Interactive scatterplot created with D3Blocks.

Quantify The Similarity Between Projections Using The KNR-score.

At this point, we have our different projects, and now we can examine the differences in the sample distribution quantitatively. The quantification step is rather straightforward with the KNRscore library. After importing the KNRscore library, we provide the coordinates as input to the *compare* function, and after the computation, it will return a score for each K-nearest neighbor.

High-Dimensional PCA versus Low-Dimensional t-SNE Space.

In our first comparison, we examine the similarity between the 50-dimensional PCA versus the 2-dimensional t-SNE space. See the code block below.

import KNRscore as knrs

# Quantify PCA vs. tSNE
scores = knrs.compare(map_pca, map_tsne, n_steps=5)

# Plot
fig, ax = knrs.plot(scores, xlabel='PCA-50D', ylabel='tSNE-2D')

Figure 4. KNR-scores between t-SNE and PCA-50D. The axis depicts the number of nearest neighbors. The 50d-PCA is shown on the x-axis and t-SNE on the y-axis. The colors depict the normalized similarity scores between the k-nearest neighbors. Image from the author.

The output of KNR-score is thus a matrix where each point is the normalized similarity between the k-nearest neighbors of the two input projections. When looking at the heatmap, we see reddish color, meaning high scores, and thus high similarity on both local and global scales (Figure 7). This indicates that the neighbors of the samples in the 50D PCA map are similarly distributed as the sample distribution in the 2D t-SNE map. Or in other words, t-SNE did a very good job in reducing the dimensionality and preserving the local similarities between the samples. Let's now proceed and examine how well the samples in the 2 dimensions of PCA are preserved compared of the sample distribution in t-SNE.

Low-Dimensional PCA versus Low-Dimensional t-SNE Space.

Here we are going to compare the 2-dimensional PCA versus the 2-dimensional t-SNE following the same procedure. In Figure 5, we see the KNR-scores. We see much lower scores on average than with our previous comparison when using 50D-PCA. In the bottom left corner are the local scales, i.e., the closest number of nearest neighbors, which are the lowest. This means that samples have different K-neighbors between the projections. The scores become slightly larger (green-ish) on the global scale, meaning that, on average, more similar K-neighbors are seen. We can now also make similar comparisons with UMAP as shown in the next part.

import KNRscore as knrs

# Quantify PCA-2D vs. tSNE
scores = knrs.compare(map_pca[:, 0:2], map_tsne, n_steps=5)

# Plot
fig, ax = knrs.plot(scores, xlabel='PCA-2D', ylabel='tSNE-2D')

Figure 5. KNR-scores between t-SNE and PCA-2D. The axis depicts the number of nearest neighbors. The 2D PCA is shown on the x-axis and t-SNE on the y-axis. The colors depict the similarity scores between the k-nearest neighbors. Image from the author.

Comparison of t-SNE Versus UMAP Projections.

Both t-SNE and UMAP showed clear separation of the classes in the scatterplot (Figure 2B and C). However, it is yet unknown whether they also have a similar sample distribution. A quantification with KNRscores showed overall high scores (Figure 6), which means that the sample distribution on both local and global scales is quite similar between the projections. However, when we carefully look at the local scales (bottom left corner) we see a score of 0.5–0.6 (green-ish) for the 6 nearest neighbors. This means that the same subcluster may have been formed but different samples are located next to each other.

import KNRscore as knrs

# Quantify UMAP vs. tSNE
scores = knrs.compare(map_umap, map_tsne, n_steps=5)

# Plot
fig, ax = knrs.plot(scores, xlabel='UMAP-2D', ylabel='tSNE-2D')

Figure 6. KNR-scores between t-SNE and UMAP. The axis depicts the number of nearest neighbors. UMAP is shown on the x-axis and t-SNE on the y-axis. The colors depict the similarity scores between the k-nearest neighbors. Image from the author.

Comparison of t-SNE Versus Random Coordinates.

The final comparison we are going to make is with random coordinates. We would expect that the KNRscores are low scores which means that no similarities are found on both local and global scales. The results in Figure 7 confirm what we expect.

import KNRscore as knrs

# Quantify Random Coordinates vs. tSNE
scores = knrs.compare(map_rand, map_tsne, n_steps=5)

# Plot
fig, ax = knrs.plot(scores, xlabel='Random (2D)', ylabel='tSNE (2D)')

Figure 7. KNR-scores between t-SNE and Random coordinates. The axis depicts the number of nearest neighbors. Random data is shown on the x-axis and t-SNE on the y-axis. The colors depict the similarity scores between the k-nearest neighbors. Low similarities are seen on local or global scales. Image from the author.

Wrapping up.

Dimensionality reduction techniques are great for feature reduction and visualization of samples in low dimensions. Although we expect that the low-dimensional space should represent the (original) high dimensionality, it is not always the case. The KNRscore library quantifies how the sample distribution of one projection is compared to another. Such insights help to examine and understand the stability of the mapping and whether you can use the results for decision-making. When we use the results of a low-dimensional space for decision-making, we should always be aware that a projection is merely a projection of a higher complex dimensional space.

Be safe. Stay frosty.

Cheers, E.

If you like the content, please give it an applause, and follow me to read my blogs! Also, try the hands-on examples. This will help you to learn quicker, understand better, and remember longer.

Software

• KNRscore library
• D3Blocks library

Let’s connect!

• Let’s connect on LinkedIn
• Follow me on Github
• Follow me on Medium

References

1. E. Taskesen, *What are PCA loadings and how to effectively use Biplots?*, Data Science Collective (DSC), July 2025
2. E. Taskesen et al, *Pan-cancer subtyping in a 2D-map shows substructures that are driven by specific combinations of molecular characteristics*, Scientific Reports Nature, 2016
3. E. Taskesen, *D3Blocks: The Python Library to Create Interactive and Standalone D3js Charts,* Medium, Data Science Collection, June 2025