Zero-Shot Call Classification: Matching Customer Intent at Scale with Semantic Embeddings

How We Match Customer Calls to Predefined Categories With Semantic Embeddings

Customer calls contain valuable signals about what is breaking, what customers need, and where operations are failing. The hard part is that those signals arrive as messy, unstructured conversations.

This article describes a production approach for extracting meaning from customer calls and matching each interaction to a predefined set of contact reason categories. Instead of relying on keyword rules or costly manual tagging, we use semantic embeddings to represent both calls and categories in the same vector space, then rank the best matches at scale.

At a high level, the pipeline looks like this — here is a concrete example of a customer disputing an invoice charge, traced through every stage:

The first stage is handled upstream. The rest of the article focuses on the classification system itself: how we encode category meaning, how we encode customer call summaries, and how we retrieve the best category matches efficiently.

Why This Problem Matters

Every day, thousands of customers call their service provider to report problems, ask for information or request account changes. Inside those conversations is operational data the business wants to understand: billing disputes, technical faults, contract changes, payment issues, retention risk, and much more.

In many contact centers, this information is still captured through manual tagging or simple rule-based systems built on keywords and regular expressions. Those approaches are usually easy to start with, but they are weak at capturing the actual meaning of the call.

That creates a downstream problem. If category labels are misclassified, analytics become unreliable, dashboards lose credibility, and stakeholders struggle to extract value from the data.

What we want instead is simple in principle: extract the relevant information from each call and match it to a stable, predefined set of customer contact reasons.

Start With A Better Input

Before classification even begins, an upstream topic modeling pipeline processes raw conversation transcripts and produces a summary for each interaction. Think of this as a condensed, structured interpretation of what the call was about — a few sentences capturing the key exchanges between the customer and the agent.

This summary is the input to our classification system.

Working on summaries instead of full transcripts gives us two immediate advantages. First, it removes a large amount of conversational noise: greetings, filler words, repetitions, and side discussions. Second, it reduces the computational cost of embedding. A concise summary is much cheaper to encode than a full call transcript, especially when inference runs daily over large volumes.

Why Embeddings Work Better Than Keywords

The naive approach to contact reason classification is a keyword list. If the customer mentions “invoice”, tag it as billing. If they say “not working”, tag it as technical fault. On paper, this looks simple and efficient. In reality, it breaks almost immediately.

Customers do not speak in clean, standardized labels. They describe problems indirectly, mix multiple issues in the same conversation, use informal language, and often assume context that is obvious to a human but invisible to a rule engine. A customer may never say “billing” and still be clearly talking about a payment issue. Another may say “it doesn’t work” when the real problem is activation, configuration, or service coverage.

This is the main limitation of keyword-based systems: they match words, not meaning. As soon as phrasing changes, the rule becomes fragile. The result is low recall, brittle logic, and a rule base that keeps growing without becoming truly reliable.

The core idea behind our approach is to classify by meaning, not by surface words. We map both the category definitions and the call summaries into the same high-dimensional vector space using an embedding model. In that space, texts with similar meaning end up close to one another. Classification becomes a similarity search problem.

The key advantage is that you do not need labeled training examples to match a summary to a category. What you need is a good representation of what each category means. If the definition of “billing dispute” is well captured as an embedding, then any call summary about disputed charges, invoice confusion, or unexpected fees should land nearby, even if the wording is different.

This makes the system effectively zero-shot. Adding or changing a category does not require retraining a classifier. It requires updating the category description and regenerating the category embeddings.

Category Definitions and Call Embeddings

Categories are created by business stakeholders. They define the taxonomy based on operational needs: what they want to monitor, which issues matter for reporting, and which call types require specific downstream actions.

Each contact reason category is then turned into a single, rich vector representation. In our setup, every category is described with a formal and specific definition of what it covers, for example: ”Requests related to the monthly invoice amount, disputed charges, or payment confirmation.”

Stakeholders decide what a category means; the model’s job is to represent that meaning as faithfully as possible and use it for matching.

At the simplest level, a category definition can be embedded with Sentence Transformers like this:

from sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence-transformers/paraphrase-multilingual-mpnet-base-v2")
category_definition = (
"Billing dispute. Requests related to unexpected charges, invoice confusion, or payment verification."
)
category_embedding = model.encode(
category_definition,
normalize_embeddings=True,
convert_to_numpy=True,
)

In practice, a bare definition is rarely enough to fully capture what a category means. A single sentence describes the concept, but it does not cover the range of ways customers actually express it. Some phrasings are obvious; others are indirect, colloquial, or domain-specific. A definition alone may not pull the embedding close enough to all the real-world formulations the system needs to match.

To address this, each category is described not only by a definition but also by a list of synonyms and a list of positive examples — short sentences that represent realistic call summaries belonging to that category.

A YAML entry for a single category looks like this:

id: Billing & Payments
definition:
  Requests related to unexpected charges on the monthly invoice,
  disputed amounts, or payment verification.
synonyms:
  - Wrong charge on bill
  - Invoice does not match
  - Double payment
positive_examples:
  - The customer says they were charged twice for the same service.
  - The caller disputes an unexpected fee on the latest invoice.
  - The user asks why the total is higher than the agreed price.

The definition gives the model a clean, abstract description. The synonyms add terminological breadth — alternative phrasings and jargon that a definition alone might miss. The positive examples anchor the embedding to concrete, realistic sentences that look like actual call summaries. Together, they cover both the conceptual meaning and the surface-level language the model will encounter at inference time.

Each of these three components is embedded separately. The definition is encoded as a single text block. Synonyms and positive examples are each encoded individually and then mean-pooled into a single vector per component. This avoids the noise of concatenating many short phrases into one long string, and lets the model attend properly to each item.

Here is a self-contained example that shows the full flow — encoding, mean pooling, and weighted combination:

import numpy as np
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence-transformers/paraphrase-multilingual-mpnet-base-v2")

# Define the category
definition = "Billing dispute. Requests related to unexpected charges, invoice confusion, or payment verification."
synonyms = [
"Wrong charge on bill",
"Invoice does not match",
"Double payment",
]
positive_examples = [
"The customer says they were charged twice for the same service.",
"The caller disputes an unexpected fee on the latest invoice.",
"The user asks why the total is higher than the agreed price.",
]

# Encode each component separately - -
v_def = model.encode(definition, normalize_embeddings=True)
v_syn = model.encode(synonyms, normalize_embeddings=True)
v_syn = np.mean(v_syn, axis=0) # mean-pool into one vector
v_ex = model.encode(positive_examples, normalize_embeddings=True)
v_ex = np.mean(v_ex, axis=0) # mean-pool into one vector

#Weighted combination - -
w_def, w_syn, w_ex = 0.6, 0.2, 0.2
vec = w_def * v_def + w_syn * v_syn + w_ex * v_ex
vec /= np.linalg.norm(vec) # re-normalize to unit length

The weights control how much each component influences the final embedding. In most configurations, the definition carries the highest weight — it is the authoritative signal, synonyms and examples act as corrections.

This design gives stakeholders a clear interface. They write definitions, add synonyms and examples in plain language, and the system translates that into a single vector that captures the full intended meaning. When a category is not matching well in production, the fix is usually not a model change — it is a better definition or a few more examples.

On the other side of the pipeline, the same embedding model is applied to customer call summaries. Each summary is encoded into the same vector space, so that calls and categories become directly comparable through similarity.

Matching Calls To Categories At Scale

Once both embedding tables are populated, the actual classification happens entirely inside BigQuery. A SQL query computes the dot product between each summary embedding and every category embedding, ranks the results, and picks the top 3 matches per call.

For example, if both embeddings are already L2-normalized, the top-3 categories can be retrieved with a query like this:

WITH similarity_scores AS (
SELECT
  calls.id,
  calls.customer_call_summary,
  categories.contact_reason,
  categories.contact_reason_embedding,
  1 - ML.DISTANCE(
  customer_call_summary_embedding,
  categories.contact_reason_embedding,
  'COSINE'
  ) AS cosine_similarity,
FROM `project.dataset.call_embeddings` AS calls
CROSS JOIN `project.dataset.category_embeddings` AS categories
),

ranked_matches AS (
SELECT
  *,
  ROW_NUMBER() OVER (
  PARTITION BY id
  ORDER BY cosine_similarity DESC
  ) AS rank_position
FROM similarity_scores
)

SELECT
id,
customer_call_summary,
contact_reason,
cosine_similarity,
rank_position
FROM ranked_matches
WHERE rank_position <= 3
ORDER BY id, rank_position;

This query produces a ranked output for each interaction, with the most semantically similar categories at the top.

There is a practical engineering reason to do this in SQL rather than in Python. At contact center scale, pulling all vectors into memory and computing pairwise similarities in an application layer is costly and hard to scale. BigQuery can do the ranking where the data already lives, store intermediate results, and keep the pipeline easier to monitor and reproduce.

The output is a ranked list. For each interaction, we retrieve the top-k contact reason categories along with their similarity scores. That gives downstream consumers more than a single predicted label. It also gives them a confidence signal and a way to inspect ambiguous cases.

Why This Works Well In Production

A few design choices make this approach robust in a real production setting:

No retraining required. Because the system is zero-shot, updating or changing contact reasons does not trigger a fine-tuning cycle. A domain expert can add a new category or rewrite a definition, and the system will pick it up on the next embedding run.

Decoupled update frequencies. Category embeddings are recomputed only when definitions change — which might be once a quarter. Summary embeddings are computed daily. This separation keeps costs low and pipelines independent.

Interpretability. The similarity score is a meaningful, human-readable signal. A score of 0.92 means the summary is nearly identical in meaning to the category definition. A score of 0.55 means the match is tentative — and an analyst can intervene.

Where This Approach Can Fail

No system is perfect, and this one is no exception.

• Results are strongly affected by the size and granularity of the taxonomy. As the number of contact reasons grows, the matching problem becomes harder. Some categories start to overlap semantically, and the distance between the best match and the second-best match becomes smaller. In practice, a taxonomy with a few broad categories is easier to separate than one with many fine-grained, highly similar categories. The model may still retrieve relevant candidates, but ranking the exact top-1 category becomes more difficult when category boundaries are too narrow or too numerous.
• Embedding quality depends on definition quality. If a category description is vague or overlaps heavily with another, even a perfect embedding model will not produce clean separations. Investing time in good definitions is as important as the ML infrastructure.
• The top-1 prediction isn’t always right. For ambiguous summaries (e.g., a customer who starts with a technical complaint and ends with a billing question), the top-3 output correctly captures the ambiguity — but downstream processes need to decide how to handle it.

Final Thought

Automatically classifying customer contact reasons is a problem that looks deceptively simple on paper and turns out to be surprisingly rich in engineering and linguistic decisions.

The approach described here works well because it treats categories as semantic objects, not just labels in a lookup table. By representing both category meaning and call meaning as vectors, we can match free-form conversations to a predefined taxonomy in a way that is scalable, flexible, and easy to evolve over time.

If you are building something similar, the biggest lever is not only model choice. It is the quality of the category definitions themselves. Strong definitions and clear boundaries are what make the embedding approach reliable in practice.