Exploring Google Cloud’s Agent Assist

Hello Medium and LinkedIn! It’s been a while — between Google I/O, the Google Cloud NORTHAM Partner Tech Roundtable, and business as usual work, the summer and fall seemed especially busy. In terms of BAU work, I’ve had a lot of clients that are curious about Google Cloud’s Agent Assist and how they can use its capabilities to improve their call center operations. I’ve been building a small demonstration of how Agent Assist works and today I figured I’d walk through that demo in the form of a blog!

Agent Assist is a relatively new product, first entering general availability on 28 February, 2022; I say “relatively” new, as three years seems like a decade ago in the fast-paced AI era we all find ourselves in. I am happy to add that Agent Assist has been receiving regular AI-enabling updates over those three years, and we’ll be going over some of the AI capabilities that clients have been asking about.

Evan, that’s nice and all — but what actually is Agent Assist?

Excellent question, my valued reader! Here’s my attempt at a simple answer:

Agent Assist connects with your existing client communication channels (customer chats, dialers / phones, etc.) and provides real-time value to your agents so that they may more quickly and accurately fulfill client requests.

But what does “value” mean, and how does Agent Assist provide it?

Also a very fair question! Agent Assist provides value (mostly in the form of decreased average handle times (AHT)) via several features that can be integrated directly into whatever tools your agents are working in; as of 29 October, 2025, Agent Assist provides the following configurable features:

• Conversation Summarization: The purpose for the call, the sentiment, and whether the call was resolved are all automatically provided to the agent, with the ability to review what was generated.
• (Proactive) Generative Knowledge Assist: Generative knowledge assist is essentially RAG geared towards call center agents — your agents ask questions in a chatbox, and Agent Assist provides relevant answers based on your documentation and articles.

Proactive Generative Knowledge Assist proactively finds relevant articles and documentation based on the real-time transcript / chat history of the conversation.

• Smart Reply: Provides the agent a selection of possible responses to choose from based on your organizations historical conversation data.
• Transcription: You might want to be sitting down for this one: the transcription feature transcribes the current conversation. As obvious as that is, there are a couple of cool aspects of this feature, which we’ll cover later.

There are a couple of additional features that were added this month, such as OpenAPI tools and AI coach; I will be looking to explore those in a future blog!

How Does it Work?

Agent Assist is part of Google Cloud’s broader Customer Engagement Suite solution (CES — previously called CCAI / Contact Center AI).

CES is a collection of AI-powered API driven products that integrate with and augment popular customer experience solutions (such as Genesys, LivePerson, Twilio, etc.), as well as custom ones. Agent Assist leverages several of its sibling product’s APIs to integrate with these various CX tools and provides frontend components that can be woven into said tools, allowing your agents to use Agent Assist features where they already work.

Enabling and Configuring Agent Assist

Agent Assist (and the rest of CES) has a completely separate console from the rest of the Google Cloud products.

The focal point of Agent Assist is the conversation profile. The conversation profile dictates what features are available to your agents, and how those features behave.

Example Agent Assist Conversation Profile

Features that require significant configuration have their own place in the Agent Assist console; for example, conversation summarization requires that you reference a “Generator”. Generators can be created and found in the “Summarization” menu of Agent Assist.

Example of a Summarization generator’s configuration.

Some features of Agent Assist rely on other parts of Customer Engagement Suite. For example, Generative Knowledge Assist relies on Conversational Agents.

Other parts of Customer Engagement Suite can also be used when conversations are integrated with Agent Assist, such as Conversational Insights, which can help you keep track of your contact center’s analytics. I will explore more of these products and relationships in future blogs; alas, we must return to our Agent Assist journey.

So I have my conversation profile fully configured — how come Agent Assist hasn’t magically materialized into my contact center?

Ah, what a wonderful alternate reality that would be! Sadly, we still need to actually integrate Agent Assist into said contact center — let’s explore that.

Simplified Architecture of Agent Assist

In this simplified architecture diagram, Agent Assist is shown as integrating into your existing CX software suite’s UI. Customizations to your CX software are performed to send audio, video, etc. to Agent Assist via the UI Connector Application. The UI Connector Application also receives various other events, such as a new conversation being selected or initiated, so that the UI modules are updated accordingly.

The UI Connector Application communicates to Customer Engagement Suite via the DialogFlow API, which is where the heavy lifting takes place.

But Evan, even the simplified architecture seems complicated — I don’t want to build all of that!

I hear you, and I’m pleased to say you don’t have to! Well, you mostly don’t have to — Google provides out of the box UI modules and even an entire UI Connector Application that you can deploy to your GCP environment to sidestep a lot of custom development — but not all of it.

Agent Assist Pre-Built Modules

Let’s start with the pre-built UI modules; Google provides several Javascript web component libraries that you can import into your existing contact center solution, provided it has an integrated browser experience and can indeed run Javascript (such as Genesys Desktop).

In the screenshot below, I’ve taken several of Google’s pre-built UI modules and integrated them with my 911 emergency dispatch demo. My demo’s UI is a completely custom web application written in React, but this could just as easily be Genesys or some other contact center dialer. Also, small shoutout to Gemini CLI for helping me build my UI and boilerplate React relatively quickly.

Example of pre-built modules integrated with existing solution.

The above screenshot includes three Agent Assist capabilities — Transcription, Conversation Summary, and Generative Knowledge Assist, though technically only two Agent Assist UI modules are shown on screen.

Both Conversation Summary and Generative Knowledge Assist are provided using the Container V1 module, which provides configurable support for:

• Knowledge Assist
• Generative Knowledge Assist
• Smart Reply, and
• Conversation Summary.

Container V1 is the easiest UI module to integrate with an application, as it’s fairly self sufficient and you do not need to initialize a separate UI module connector. The trade-off with Container V1 is that it doesn’t have all of the latest and greatest features available.

Container V2 has the latest features, but it does require initialization of a separate UI module connector — I will be modifying my demo to include it for a later blog!

The Container Agent Assist UI modules offer a great way to configure most of the available Agent Assist features; however, if your software’s UI requires that you spread out the features in different areas, or you want to use a feature not provided by one of the Container modules (i.e. Transcription), you must implement the standalone UI modules.

All of the Agent Assist UI modules receive events from your broader software suite’s environment, allowing them to act on the latest available information. In the below example, the agent has provided their initial greeting and has been given some context around the call — the Agent Assist UI Transcript module has subscribed to new-message-received events, which my demo dialer emits.

Agent Assist subscribes to real-time events in your CX software, allowing your agents to get immediate help.

To round out the demo, below is an example of Agent Assist providing a summary through its UI modules — I especially like that agents can override the generated summary right in the UI. You can also see that the Summary Generator that I setup for my Conversation Profile is taken into account here; I opted out of the “Reason for cancellation” predefined section, and so it is removed from the UI module, too.

An example of Agent Assist generating a summary.

So far we’ve covered configuring Agent Assist in the CES console and the UI modules; we still need to touch on backend connectivity.

Due to my demo being relatively simple, I chose to spin everything up locally — my architecture looks something like this:

1. My custom call center frontend imports the Container V1 and Transcript Agent Assist modules and initializes them with the appropriate values. Note that the UI modules do authenticate and connect with the Dialogflow API directly to initiate call summarizations, knowledgebase searches, etc, though a proxy can be specified if desired.
2. When new conversations are initiated, my frontend service calls my backend service.
3. My backend service also connects directly to the Dialogflow API to let Customer Engagement Suite know when new conversation data is available — you can think of it as my “UI Connector Service” in the simplified diagram from earlier.

This setup works well for my demo needs; however, a dummy React app and a local FastAPI service do not exactly constitute an enterprise solution.

So, what does a real world Agent Assist integration look like?

Remember how Google provides OOTB Agent Assist UI modules? They also provide an Agent Assist Integration Backend Module — here’s their architecture diagram:

Source: https://cloud.google.com/agent-assist/docs/backend-basics

The solution architecture shown above is much more representative of an enterprise scalable architecture, and is exactly what you end up with if you deploy Google’s Agent Assist Integrations repository. While the above solution works well generally, the repository also includes vendor-specific integrations for popular CX and CRM offerings such as LivePerson and Salesforce.

What’s Next and Final Thoughts

We’ve covered a lot in this blog! We’ve gone from “what is Agent Assist” to “here are the specific UI Javascript components that you can deploy”, and yet there’s still so much more to cover. Nevertheless, I hope that this has been helpful and I appreciate those of you who have made it this far!

I’ve made a couple of callouts throughout the blog where more time is required, namely around Container V2, the backend modules, and the other Agent Assist features that this combination can unlock.

In terms of final thoughts, Agent Assist is an incredible collection of features that can add a lot of value to contact center operations, especially in shops that already have a Google Cloud presence.

So, Is Agent Assist Perfect?

Like everything else in life though, there is bad with the good — in this case, the “bad” is Agent Assist’s documentation. It is my single biggest gripe with Agent Assist — both the UI module events documentation and the Dialogflow API documentation are incredibly incomplete and rife with errors.

Let’s pick on a relatively minor example from the Dialogflow API Conversation class. Conversations have an ID attribute called “name” that you must populate when submitting a CreateConversationRequest. And guess what? That ID requires much more than just “projects/” (it should be projects/<project>/locations/<location>/conversations/<conversation_id>).

My personal favourite is from the UI module events documentation; at one point, I needed to emit a new-message-received event so that the Transcript module was made aware that it needed to update the visible conversation.

Naturally, I took a peek at the NewMessageReceivedPayload to see what my message structure needed to look like. What I found didn’t exactly fill me with joy (I had to copy and paste a link to some other random Dialogflow API type), but that’s fine — I can work with that.

Unfortunately for me, the object I found in the reference documentation was so detached from what the UI modules actually expected that I had to setup breakpoints in Google’s minified Javascript code in order to figure out what the actual structure needed to be. In case you’re curious, this is the schema that the UI module was actually expecting:

{
    name: response.data.message_id,
    participantRole: (nextMessage.participant === "agent" ? "HUMAN_AGENT" : "END_USER"),
    content: messageToSend,
    createTime: response.data.timestamp
}

Final Final Thoughts

I hate to leave things on a bit of a sour note — so I will say this: the documentation is of poor quality because this is still a relatively new product for Google Cloud’s portfolio. To improve adoption, however, the documentation needs to be in a better place.

I will be making suggestions and reporting on some of these documentation shortcomings to Google during this blog series, and I would encourage others to do the same!

I look forward to exploring more of Agent Assist and CES’s capabilities in my subsequent blogs — if there are specific features you are curious about, please leave a comment! Until then, have a safe and happy Halloween weekend!