tokio::select! Is a Loaded Gun — Understanding Cancel Safety Before It Silently Destroys Your Data

Part 1 of 2 — Async Rust Cancellation Series

The Bug That Leaves No Trace

Here is a bug that leaves no trace.

No panic. No compiler warning. No log line. Your async Rust service is running, requests are coming in, and somewhere inside a tokio::select! block, a database write is being silently dropped. The user gets a success response. The data was never saved.

This is not a theoretical edge case. It happens in production Rust services written by experienced engineers — because cancel safety is one of the few hazards in async Rust that the compiler cannot catch for you.

This post is Part 1 of a two-part series. We will use one running example — a user registration handler that writes to a database — and break it three different ways. Part 2 shows you how to fix every one of them.

By the end of this post you will understand:

• What cancel safety means and why tokio::select! triggers it
• How the reserve pattern protects cancel-unsafe code
• Why holding a Mutex across an await point multiplies your blast radius

Let’s start at the crime scene.

What Is Cancel Safety — And Why Most Developers Skip It

Every .await point in Rust is a potential cancellation point.

When you drop a future — by cancelling it, timing it out, or selecting away from it — execution stops exactly at whichever .await it was suspended on. No cleanup code runs. No destructor is called on the in-progress operation. The future simply ceases to exist at that point.

Cancel-safe futures can be dropped at any .await with no harm done. Reading from a channel, sleeping, waiting for a socket to become readable — if you cancel these mid-wait, nothing was partially done. The world is consistent.

Cancel-unsafe futures, if dropped mid-execution, leave the world in an inconsistent state. Writing to a database, sending a message, flushing a buffer — if you cancel these after they have started but before they have finished, you may have partially committed something. The caller received no error, but the operation did not complete.

The Tokio documentation marks async functions with their cancel safety. Most developers never read it.

// From the Tokio docs — these are cancel-SAFE:
// tokio::sync::mpsc::Receiver::recv()
// tokio::time::sleep()
// tokio::net::TcpStream::readable()

// These are cancel-UNSAFE:
// tokio::io::AsyncWriteExt::write_all()
// tokio::sync::Mutex::lock()   ← we will come back to this
// std::future::Future (any custom future with internal state)

The danger is not in using cancel-unsafe futures. It is in using them inside tokio::select! — because select! cancels every branch that did not win the race.

The Bug You Cannot See: A Real Example

Let’s build the crime scene step by step. We have a user registration service. A handler receives a registration request, writes the user to the database, and returns a success response. There is also a timeout — if the write takes too long, the handler should return an error.

Here is the naive implementation:

use tokio::time::{timeout, Duration};
use tokio::sync::mpsc;

// Simulates a database write - cancel-unsafe
async fn save_user_to_db(user: User) -> Result<(), DbError> {
    // Imagine this does:
    // 1. BEGIN TRANSACTION
    // 2. INSERT INTO users ...
    // 3. UPDATE counters ...
    // 4. COMMIT
    db_client.execute_transaction(user).await
}
async fn register_handler(
    user: User,
    mut shutdown: mpsc::Receiver<()>,
) -> Result<Response, Error> {
    tokio::select! {
        // Branch 1: Try to save the user with a timeout
        result = timeout(Duration::from_secs(5), save_user_to_db(user)) => {
            result??; // unwrap timeout, then db error
            Ok(Response::success())
        }
        // Branch 2: Shutdown signal received
        _ = shutdown.recv() => {
            Err(Error::ShuttingDown)
        }
    }
}

This code looks reasonable. It has a timeout and it handles shutdown. But it contains a critical flaw.

What happens when shutdown.recv() wins the race at exactly the wrong moment:

Timeline:
t=0ms   save_user_to_db begins
t=1ms   BEGIN TRANSACTION sent to DB ✓
t=2ms   INSERT INTO users sent to DB ✓
t=3ms   UPDATE counters — in flight...
t=3ms   shutdown signal arrives
t=3ms   select! cancels save_user_to_db ← dropped HERE
t=3ms   Transaction is now open on the DB with partial data
t=3ms   DB connection returns to pool — transaction auto-rolled back
t=3ms   User gets Err(ShuttingDown) — BUT the INSERT was already written
        to the DB log before the rollback. Depending on your DB and
        isolation level, this may or may not leave ghost data.

There was no error from the database. There was no panic. The future was simply dropped mid-transaction. The database handled the rollback, but your metrics recorded a failed registration, your retry logic will try again, and depending on your DB’s isolation level you may have committed half the work.

The compiler cannot warn you about this. Rust’s ownership system only ensures the future is dropped cleanly — it says nothing about whether dropping it mid-execution is semantically correct.

The reserve Pattern: Separating Safe From Unsafe

The reserve pattern solves this by splitting the work into two phases:

1. The cancel-safe phase — acquire a “reservation” (a slot, a permit, a pre-allocated resource). This phase can be safely cancelled at any point because nothing has been written yet.
2. The cancel-unsafe phase — use the reservation to perform the actual write. This phase runs outside select! so it cannot be cancelled.

Here is what this looks like for our registration handler:

use tokio::sync::OwnedSemaphorePermit;
use std::sync::Arc;
use tokio::sync::Semaphore;

// Phase 1: cancel-SAFE - just acquires a write permit
// If this is cancelled, nothing harmful has happened
async fn reserve_write_slot(
    semaphore: Arc<Semaphore>,
) -> Result<OwnedSemaphorePermit, Error> {
    semaphore
        .acquire_owned()
        .await
        .map_err(|_| Error::SemaphoreClosed)
}
// Phase 2: cancel-UNSAFE - the actual DB write
// This MUST complete - we never put it inside select!
async fn commit_user(permit: OwnedSemaphorePermit, user: User) -> Result<(), DbError> {
    let result = db_client.execute_transaction(user).await;
    drop(permit); // release slot after commit
    result
}
async fn register_handler(
    user: User,
    mut shutdown: mpsc::Receiver<()>,
    semaphore: Arc<Semaphore>,
) -> Result<Response, Error> {
    // ✅ Phase 1 inside select! - cancel-safe
    let permit = tokio::select! {
        permit = reserve_write_slot(semaphore) => permit?,
        _ = shutdown.recv() => return Err(Error::ShuttingDown),
    };
    // ✅ Phase 2 outside select! - runs to completion, cannot be cancelled
    commit_user(permit, user).await?;
    Ok(Response::success())
}

The key insight: the select! block now only contains futures that are safe to cancel. Acquiring a semaphore permit is safe to cancel — if we cancel, we just didn’t get the permit, and nothing was written. The actual DB write happens after the select! returns, where it cannot be interrupted.

This is the reserve pattern. It is not limited to semaphores — the same idea applies to:

• Pre-allocating a buffer slot before writing
• Acquiring a channel capacity before sending
• Opening a transaction handle before executing queries
• Reserving an ID before inserting a record

The Mutex Trap: How Locks Expand Your Blast Radius

Now let us look at a subtler version of the same problem. Suppose your registration handler needs to update an in-memory cache as well as the database. You reach for tokio::sync::Mutex:

use tokio::sync::Mutex;
use std::sync::Arc;
use std::collections::HashMap;

async fn register_handler_with_cache(
    user: User,
    cache: Arc<Mutex<HashMap<UserId, User>>>,
    mut shutdown: mpsc::Receiver<()>,
) -> Result<Response, Error> {
    tokio::select! {
        result = async {
            // ❌ PROBLEM: lock() is cancel-unsafe
            // If select! cancels this branch while we hold the lock,
            // the MutexGuard is dropped - but the DB write below
            // may have already partially executed
            let mut cache_guard = cache.lock().await;   // ← cancel-unsafe
            save_user_to_db(user.clone()).await?;        // ← cancel-unsafe
            cache_guard.insert(user.id, user);
            Ok::<_, Error>(Response::success())
        } => result,
        _ = shutdown.recv() => Err(Error::ShuttingDown),
    }
}

This has two cancel-unsafe operations inside select!, chained together. When shutdown wins the race:

• If cancelled after cache.lock() but before save_user_to_db returns: the DB write was never committed, the cache was never updated. This is actually fine — both are consistent at "not written".
• If cancelled after save_user_to_db returns but before cache_guard.insert(): the DB has the user, the cache does not. Inconsistent.
• If cancelled while cache.lock() is waiting: fine, nobody held the lock. But now consider: what if another task already holds the lock and is itself waiting on an await point? You have a deadlock-adjacent situation where tasks are blocking each other during shutdown.

tokio::sync::Mutex is specifically documented as cancel-unsafe because the lock can be held across a cancellation boundary, leaving state permanently locked or inconsistent.

The fix: replace shared mutable state with message passing.

use tokio::sync::mpsc;

// Dedicated cache actor - owns the HashMap, serialises all access
async fn cache_actor(mut rx: mpsc::Receiver<CacheMessage>) {
    let mut cache: HashMap<UserId, User> = HashMap::new();
    while let Some(msg) = rx.recv().await {
        match msg {
            CacheMessage::Insert { user, ack } => {
                cache.insert(user.id, user);
                let _ = ack.send(()); // signal completion
            }
            CacheMessage::Shutdown => break,
        }
    }
}
enum CacheMessage {
    Insert { user: User, ack: tokio::sync::oneshot::Sender<()> },
    Shutdown,
}
async fn register_handler_actor(
    user: User,
    cache_tx: mpsc::Sender<CacheMessage>,
    mut shutdown: mpsc::Receiver<()>,
) -> Result<Response, Error> {
    // Phase 1: reserve capacity in the channel - cancel-safe
    let permit = tokio::select! {
        permit = cache_tx.reserve() => permit.map_err(|_| Error::CacheClosed)?,
        _ = shutdown.recv() => return Err(Error::ShuttingDown),
    };
    // Phase 2: DB write and cache update - outside select!, runs to completion
    save_user_to_db(user.clone()).await?;
    let (ack_tx, ack_rx) = tokio::sync::oneshot::channel();
    permit.send(CacheMessage::Insert { user, ack: ack_tx });
    ack_rx.await.map_err(|_| Error::CacheClosed)?;
    Ok(Response::success())
}

mpsc::Sender::reserve() is cancel-safe — documented explicitly in Tokio. If the reservation is cancelled, no message was sent, and the channel state is unchanged. The actual send happens after select!, atomically and without interruption.

What You Now Know — and What Comes Next

You have seen cancel safety break in three distinct ways:

Break #1 — A cancel-unsafe future inside select! gets dropped mid-transaction. No error, no panic, partial state in your database.

Break #2 — The reserve pattern fixes it by keeping the cancel-safe acquisition inside select! and the unsafe write outside.

Break #3 — tokio::sync::Mutex across await points creates a second cancel-unsafe chain. Message-passing actors eliminate shared mutable state entirely.

The pattern emerging across all three fixes is the same: decide what is safe to cancel and what is not, then structure your code so the unsafe parts never live inside a cancellation boundary.

Part 2 takes this further. We will look at spawning background tasks for cancel-unsafe work, implementing graceful shutdown sequences so your service drains cleanly on Ctrl+C, building fan-in cancellation channels for cooperative shutdown, and a first look at what async traits finally give us.

Found a production case where cancel safety bit you? Drop it in the comments — the more specific the better.

Tags: #Rust #AsyncRust #Tokio #ConcurrentProgramming #BackendDev #RustLang #SystemsProgramming

Written with 🦀 Rust and a healthy respect for .await points.