Building an AI Voice Qualifier for VICIdial Without Replacing the Call Center

Outbound call centers already have the hardest parts of sales operations in place: campaigns, dialing rules, lead routing, agent availability, and closer workflows.

The problem is not usually that the dialer is missing. The problem is that human closers spend too much time on early qualification.

That is the gap this solution solves.

We built an outbound-only AI voice qualifier that sits between a VICIdial answered call and the human closer group. It does not replace VICIdial. It does not own CRM, lead storage, inbound calls, or long-term transcripts. It handles the first 40 to 50 seconds of a live call, qualifies the lead, and transfers only promising calls to a human closer.

The result is a focused AI layer for live outbound calls:

VICIdial dials the lead

-> customer answers

-> AI agent receives the call as a SIP remote agent

-> AI qualifies the customer in real time

-> qualified customer is transferred to a VICIdial closer group

-> human closer completes the sale

The Solution

The solution is intentionally narrow: use AI where it creates leverage, and keep the existing call center workflow intact.

VICIdial remains responsible for:

• outbound campaigns

• lead dialing

• call routing

• closer queues

• human sales workflow

The AI service is responsible for:

• answering only routed outbound calls

• creating one isolated runtime session per active call

• listening to the caller through streaming speech-to-text

• generating short qualification responses

• speaking back through low-latency text-to-speech

• detecting interruption and stopping stale audio immediately

• transferring qualified calls to the closer group

This matters because many AI calling systems try to become the dialer, CRM, transcript store, analytics platform, and agent workflow all at once. That creates risk, operational friction, and long implementation cycles.

This solution does the opposite. It adds a real-time AI qualification layer around the system the call center already uses.

Why This Shape Works

In a sales floor, the human closer is the expensive resource. The AI should not try to replace that person during the close. It should protect their time.

The first part of many outbound calls is repetitive:

• Does the person have a moment?

• Are they the right customer profile?

• Are they interested enough to continue?

• Should this call go to a closer?

That is a good fit for AI because the interaction is short, structured, and easy to measure.

The close itself is different. It often needs judgment, objection handling, trust, pricing discussion, and compliance awareness. That remains with the human closer.

So the system uses AI as a qualification gate, not as a full sales replacement.

Architecture

At runtime, the application acts like a SIP phone extension registered to the VICIdial environment. When VICIdial dials a lead and the lead answers, the call is routed to the AI agent extension. The app answers the SIP INVITE, negotiates RTP media, and starts the live qualification session.

High-level architecture:

VICIdial outbound campaign

| answered customer call

VICIdial / Asterisk telephony layer

| SIP INVITE + RTP audio

Call Agent runtime

• → Deepgram streaming STT

• → OpenAI Realtime reasoning

• → ElevenLabs streaming TTS

SIP REFER to VICIdial closer group

The core runtime is a Node.js, Express, and TypeScript service. It uses Redis for ephemeral active-call state, WebSocket-based provider streams, SIP/RTP for telephony, and strict validation around all external input.

The important design decision is that live calls are not modeled as normal request-response API flows. A phone call is a streaming session. Audio, transcripts, model output, TTS chunks, telephony events, and call status changes all happen over time.

That means the system needs a stateful runtime per call.

Each active call has:

• a call ID

• campaign context

• current lifecycle status

• isolated provider sessions

• an event timeline

• active TTS generation state

• transfer or hangup capability

The live call lifecycle looks like this:

bridging -> listening -> thinking -> speaking -> transferring -> transferred

-> ended

-> failed

Redis stores only operational runtime state. It is not used as a permanent transcript database or CRM. Terminal calls expire after a short TTL.

Real-Time Voice Pipeline

The voice pipeline is built around low-latency streaming:

1. VICIdial routes the answered call to the AI SIP extension.

2. The app answers the SIP INVITE and opens RTP media.

3. Caller audio streams to Deepgram as ulaw/8kHz speech.

4. Final transcripts are passed into OpenAI Realtime.

5. OpenAI returns response text and qualification decisions.

6. Response text is chunked into speakable phrases.

7. ElevenLabs streams TTS audio back as ulaw/8kHz.

8. The RTP bridge sends audio back into the live call.

9. If qualified, the app sends a SIP REFER to the VICIdial closer group.

The system avoids request-response audio handling for live calls because that pattern adds too much delay. Instead, every provider connection is treated as part of the live call session.

Barge-In Is Not Optional

For voice AI, interruption handling is one of the difference-makers.

If the AI keeps talking after the customer starts speaking, the experience feels broken. In this solution, caller speech start triggers immediate barge-in handling:

• outbound audio playback stops

• active OpenAI output is canceled

• active TTS generation is invalidated

• stale audio chunks are ignored

• the call returns to listening

The implementation uses generation IDs so old TTS chunks cannot play after an interruption. This is critical because streaming systems can still deliver late chunks after cancellation. Without generation tracking, the caller may hear outdated audio after they already interrupted.

The Solution We Provided

We provided a production-shaped backend for an outbound-only VICIdial AI qualifier.

The implementation includes:

• strict TypeScript and ESM project structure

• Express app split from process startup for testability

• Zod-based environment and request validation

• centralized config access

• centralized error handling

• Redis-backed live call store

• active call lifecycle model

• SIP registration and call handling

• RTP media bridge for live audio

• Deepgram streaming STT wrapper

• OpenAI Realtime session wrapper

• ElevenLabs streaming TTS wrapper

• campaign registry configuration

• closer-group transfer gateway

• readiness and health endpoints

• ops API for active calls, transfer, hangup, metrics, and events

• lightweight ops UI for runtime visibility

• tests around runtime behavior, routes, and configuration

The scope is deliberately constrained.

This v1 does not include:

• inbound calls

• CRM writeback

• RAG

• permanent transcript persistence

• generic call simulators

• direct VICIdial database writes

• non-VICIdial transfer targets

Those exclusions are not missing features. They are guardrails. They keep the first production version focused on the highest-value workflow: qualify answered outbound calls and route good prospects to humans.

Security and Operational Decisions

Because this system touches phone numbers, live calls, provider keys, and operational call events, security is part of the architecture rather than an afterthought.

Key decisions:

• external input is validated with Zod

• provider secrets stay in environment configuration

• ops routes are protected by an internal bearer token when configured

• raw audio is not logged

• phone numbers and call events are treated as sensitive

• stack traces are not exposed to clients

• VICIdial databases are not written to directly

• SIP and RTP ports should be firewalled to trusted telephony infrastructure

The goal is to integrate with the call center safely while minimizing data retention.

Performance and Latency

Voice systems are judged by timing. Even a technically correct response can feel wrong if it arrives too late.

The performance strategy is:

• keep audio streaming instead of batching

• use telephony-native ulaw/8kHz audio where possible

• keep hot paths non-blocking

• avoid unnecessary format conversions

• use Redis only for lightweight runtime state

• clean up sessions deterministically on transfer, hangup, failure, or shutdown

• track latency around STT, OpenAI, TTS, and transfer operations

The AI qualification window is short, so every part of the pipeline has to support fast turn-taking.

Why This Is a Practical AI Call Center Pattern

The most practical AI deployments do not start by replacing the whole business process. They attach to one measurable workflow and improve it.

Here, the measurable workflow is simple:

answered outbound call -> AI qualification -> closer transfer

That gives the team clear success metrics:

• fewer unqualified calls reaching closers

• faster qualification

• better closer utilization

• measurable transfer rate

• lower average human time per lead

• controlled AI provider cost per answered call

The architecture also keeps failure modes manageable. If AI qualification fails, the system can hang up cleanly or transfer based on configured rules. If a campaign is not configured, it can fall back to a default closer group or fail safely. If the caller interrupts, stale audio is canceled.

That is the kind of control a real call center needs before trusting AI in a live voice path.

Final Thought

AI voice systems should not be designed as demos. They should be designed as operational components.

This solution works because it respects the existing call center stack:

• VICIdial owns dialing and closer workflow

• the telephony layer owns live call transport

• Redis owns temporary runtime state

• AI owns short qualification conversations

• humans own the close

That separation makes the system easier to test, easier to operate, and easier to improve without expanding scope too early.

Hooks You Can Use

1. We did not build an AI call center. We built the missing AI qualification layer for the call center that already exists.

2. The best use of AI in outbound sales is not replacing closers. It is making sure closers only talk to better-qualified prospects.

3. Most AI voice demos break when the caller interrupts. In production, barge-in handling is not a feature. It is a requirement.

4. Instead of replacing VICIdial, we let VICIdial keep doing what it does best: dialing, routing, and managing closer workflows.

5. The architecture is simple by design: answered call in, AI qualification in the middle, human closer at the end.

6. AI voice becomes practical when it stops trying to own the entire sales process and starts owning one measurable workflow.

7. For outbound sales, the first 50 seconds can decide whether a closer should spend the next 10 minutes on the call.

8. The safest v1 is not the biggest system. It is the one with the clearest boundaries.

9. A real-time AI caller is not an HTTP endpoint. It is a live streaming session with cancellation, state, latency, and cleanup.

10. The goal was not to automate every conversation. The goal was to protect human closers from conversations that should never reach them.