How I Turned KPI Names Into Semantic Vectors

Most embedding systems today start with transformers, billions of parameters, and a GPU budget. Mine started with KPI names like Cpu_Failure_Ratio and disk_latency_ms. I realized something surprisingly powerful: operational metrics already contain semantic structure hidden directly in their naming conventions. So instead of using an LLM, I built a tiny embedding engine from scratch using token statistics, hashing, and weighted vectors. The result was a lightweight semantic system that could group related KPIs together with no neural network at all.

Why I Needed an Embedder Without an LLM

Most modern semantic search systems immediately reach for transformers, sentence embeddings, or large language models. But my problem was much narrower: I only needed to understand KPI names. These metric names were already generated using relatively strict naming conventions, meaning the semantics were partially encoded directly in the text itself.

For example, KPIs related to CPU metrics almost always contained words like cpu, while reliability metrics frequently included terms such as failure, error, or ratio. Latency-related KPIs consistently reused words like latency, response, or duration. Even without natural language understanding, the structure was already there.

Using a full LLM for this felt excessive. I did not need world knowledge, grammar understanding, or conversational reasoning. I only needed a way to capture similarity between technical metric names efficiently.

I also wanted something:

• lightweight
• deterministic
• fast to train
• easy to debug
• cheap to run
• and independent from external models or APIs

So instead of using deep learning, I approached the problem more like information retrieval and statistical NLP. The idea was simple: if KPI names are constructed from meaningful recurring tokens, then those tokens themselves can become the foundation of a semantic embedding system.

KPI Names Already Contain Semantic Signals

KPI names are usually not random strings, they follow predictable naming conventions that already encode meaning. Similar metrics naturally reuse the same words, which creates implicit semantic relationships.

For example:

• cpu_failure_ratio
• cpu_usage_percent
• disk_failure_ratio
• memory_consumption

Even without an LLM, it is obvious that:

• the first two KPIs are related through cpu
• the first and third are related through failure_ratio
• memory_consumption belongs to a different semantic group

Words like cpu, failure, latency, usage, and ratio effectively become semantic anchors. By detecting and weighting these shared tokens, similar KPIs can be grouped together automatically.

Cleaning and Tokenizing KPI Names

Before anything useful can be extracted, KPI names need to be normalized into a consistent textual form. Raw metric names are often noisy, filled with separators, formatting artifacts, and structural characters that don’t carry semantic meaning.

So the first step is cleaning: removing underscores, punctuation, braces, and other filler symbols, while also stripping out common stopwords like articles or connectors.

Example:

• raw: system_cpu_failure_ratio:{region}
• clean: system cpu failure ratio region
• tokens: [system, cpu, failure, ratio, region]

After this step, each KPI becomes a simple sequence of meaningful tokens. This makes it possible to treat every metric as a small “bag of concepts” rather than a rigid string format.

Building a Vocabulary With Hash Mapping

Once the KPIs are tokenized, the next step is to turn those tokens into something machine-friendly. I built a stable vocabulary where every unique token seen in the training data gets a deterministic ID (or hash). This effectively becomes a lightweight tokenizer vocabulary.

The idea is simple: instead of dealing with raw strings like cpu or failure, everything is mapped to a consistent numeric representation that never changes between runs.

This matters for a few reasons:

• Reproducibility: the same token always maps to the same ID
• Compactness: numbers are cheaper to store and process than strings
• Efficiency: lookup and vector construction become fast and predictable

Weighting Tokens by Importance

Unlike typical NLP setups, I didn’t want rare words to dominate the representation. KPI names are already compressed and structured, there are no “fluffy” words like in natural language. Every token is usually meaningful in context, so over-penalizing frequent terms would actually destroy structure.

Instead, I leaned into the opposite intuition: common tokens are often the strongest semantic anchors.

For example, words like cpu, memory, disk, failure, latency appear across many KPIs and define the core dimensions of the system. These should shape the vector more strongly, not less.

So looking at these KPIs:

Pod1_login_failure_ratio Pod2_login_failure_ratio Pod1_message_failure_ratio Pod3_memory_usage Pod4_memory_usage Pod4_cpu_usage

you can already see a multi-level structure forming.

At the highest level, the space naturally splits into two big clusters: failure_ratio vs usage. These two tokens define fundamentally different kinds of system signals, one is about reliability, the other about resource consumption.

Inside the failure_ratio cluster, there is a second level of structure:

• login_failure_ratio
• message_failure_ratio

These form subclusters based on what is failing, while still staying within the same broader failure domain.

Even more interesting is how scope behaves: pod1_login_failure_ratio is closer to pod1_message_failure_ratio than to pod2_login_failure_ratio, because the shared pod1 + failure context reinforces locality even more strongly.

This is exactly why a TF-IDF-style approach would fail here. If rare tokens were up-weighted, then pod1, pod2, pod3 would dominate similarity, breaking the semantic grouping. You would end up clustering by pod identity instead of behavior.

Compressing KPI Names Into Dense Vectors

After tokenizing and weighting, each KPI is still a variable-length list of tokens. To make this usable for search and clustering, I compress everything into a fixed-size representation: an 8-dimensional embedding vector.

Each token is first mapped to a small vector, then combined using a weighted average based on its importance. This produces a single dense vector per KPI, regardless of name length.

The key idea is that semantically similar KPIs end up producing similar vector compositions. Because they share many of the same tokens, their final embeddings are pulled toward the same region in vector space.

Even with just 8 dimensions, the space becomes surprisingly expressive. It is dense enough to smooth out noise, but small enough to stay fast for similarity search, clustering, and nearest-neighbor lookup.

Results, Limitations, and What Comes Next

The 8D embeddings produced surprisingly clean clustering even without any deep learning. Related KPIs consistently grouped together:

KPI Semantic Clusters in 3D Vector Space (created by the writer)

The 20 KPI names were tokenized by splitting on underscores, then each token was weighted by its corpus frequency to build sparse 8 dimensional vectors. Then compressed those vectors down to three principal components, capturing about 52% of the total variance across the two most separating dimensions.

The plot reveals four cleanly separated color-coded clusters: CPU metrics in blue, memory in green, disk and storage in yellow, and failure/reliability signals in red, with cpu_failure_ratio notably drifting between the blue and red groups, since it genuinely shares tokens with both, which is exactly the kind of nuanced overlap a good embedding system should surface.

The system is fast, fully deterministic, GPU-free, and works well for similarity search, clustering, and nearest-neighbor lookup over large KPI catalogs.

However, it only works because KPI naming is structured. It cannot capture deeper semantics like transformers, and it breaks when naming conventions drift or overlap becomes inconsistent. Future improvements could include subword or character n-grams for robustness, learned token embeddings, and dimensionality reduction (PCA/UMAP) for visualization, or even hybrid symbolic + neural approaches.

https://www.buymeacoffee.com/kisandrasns