Tokio, Tower, Hyper, and Rustls: Building High-Performance and Secure Servers in Rust — Part 3

In Part 2 of this series, we explored the configuration of Rustls for mutual TLS (mTLS) and proposed a practical code example. Before we dive into the complete server implementation in Part 4, we’ll now take a closer look at the middleware stack. As outlined earlier, the server composes three middleware layers and one final service:

Middleware Stack Overview

1. Rate Limiter (Layer + Service) Enforces basic, time-based request limiting. This version is intentionally simple and not production-ready — we’ll discuss its limitations and potential improvements later.
2. Timing Middleware (Layer + Service) Measures and logs the duration of each incoming request.
3. Logging Middleware (Layer + Service) Logs each incoming HTTP request to standard output for basic observability.
4. Echo Service (Service) A lightweight service that implements two example GET endpoints for demonstration.

Middleware Module Structure

The middleware components are organized as follows:

middleware/             # Tower layers and services
  ├── mod.rs
  ├── echo.rs           # Echo service with sample endpoints
  ├── logger.rs         # Logging middleware
  ├── rate_limiter.rs   # Rate limiting middleware
  └── timing.rs         # Timing middleware

The Echo Service

The EchoService (in file echo.rs) is a minimal yet functional implementation of a Tower service. It demonstrates how to handle different HTTP paths and return appropriate responses.

Functionality

• GET / → Returns a default echo message.
• GET /help → Returns a help message.
• Any other path → Returns a 404 Not Found.

Here’s the implementation:

use std::{
    convert::Infallible,
    task::{Context, Poll},
};

// Async utilities from the futures crate
use futures::future::{ready, Ready};

// Hyper HTTP types
use hyper::{body::Incoming, Request, Response, StatusCode};

// HTTP body utilities
use bytes::Bytes;
use http_body_util::{combinators::BoxBody, BodyExt, Full};

// Tower service trait
use tower::Service;

/// A simple Tower service that echoes responses based on the request path.
///
/// This service handles two GET endpoints:
/// - `/` returns a default echo message.
/// - `/help` returns a help message.
/// - Any other path results in a 404 Not Found response.
///
/// This is primarily used as a demonstration of integrating a Tower service
/// into a middleware stack.
#[derive(Clone, Debug)]
pub struct EchoService;

impl Service<Request<Incoming>> for EchoService {
    /// The type of response returned by the service.
    type Response = Response<BoxBody<Bytes, Infallible>>;
    /// The error type that may occur during handling.
    type Error = hyper::Error;
    /// The future that resolves to a response or error.
    type Future = Ready<Result<Self::Response, Self::Error>>;

    /// Checks whether the service is ready to process a request.
    ///
    /// In this simple service, it's always ready immediately, so this returns
    /// `Poll::Ready(Ok(()))` without delay.
    fn poll_ready(&mut self, _cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        Poll::Ready(Ok(()))
    }

    /// Handles the incoming HTTP request and returns an appropriate response.
    ///
    /// # Behavior
    /// - If the request path is `/`, it responds with "Echo!".
    /// - If the request path is `/help`, it responds with a help message.
    /// - Any other path returns a 404 Not Found response.
    ///
    /// All responses use a boxed HTTP body (`BoxBody`) for compatibility
    /// with Tower and Hyper service layers.
    fn call(&mut self, req: Request<Incoming>) -> Self::Future {
        let path = req.uri().path();
        let body = match path {
            // Match root path and return default echo response
            "/" => "=====> Echo!\n",
            // Match /help path and return help content
            "/help" => "=====> This is the help page.\n",
            // All other paths return 404 Not Found
            _ => {
                let mut not_found = 
                   Response::new(Full::new(Bytes::from("Not Found")).boxed());
                *not_found.status_mut() = StatusCode::NOT_FOUND;
                return ready(Ok(not_found));
            }
        };

        // Construct the response with the matched message
        let response = Response::new(Full::new(Bytes::from(body)).boxed());
        ready(Ok(response))
    }
}

Key Takeaways

• poll_ready() signals that the service is ready to handle a request. In more complex services (e.g., ones that query a database), this might return Poll::Pending if not immediately ready.
• call() contains the core logic to handle incoming requests and return a response.
• BoxBody is used to wrap the response body in a uniform, type-erased format that works across multiple layers and services.

What is BoxBody?

BoxBody is a utility that wraps a concrete HTTP body type (like Full<Bytes>) into a dynamic, boxed type. This allows multiple middleware layers and services — each potentially producing different body types — to work together under a common abstraction.

In other words: ***BoxBody provides interoperability* between layers by hiding the actual type of the body, much like Box<dyn Trait> does in trait objects. It’s especially useful when building composable systems with Tower and Hyper, where each layer might have its own implementation of body streaming or buffering.

The Logger Layer and Service

Logging is an essential feature in modern web servers. It helps track incoming requests, monitor responses, and debug unexpected behaviors. In this example, we implement a Tower middleware layer that logs both HTTP requests and responses using tracing.

The logger is implemented in two parts (file logger.rs):

1. **SimpleLoggerLayer** – a Layer that wraps a service with the logging logic.
2. **SimpleLogger** – a Service middleware that logs each request and response.

SimpleLoggerLayer

use std::{
    fmt::Debug,
    future::Future,
    pin::Pin,
    task::{Context, Poll},
};

// Hyper HTTP types
use hyper::{Request, Response};

// Tower traits
use tower::{Layer, Service};

/// A Tower `Layer` that wraps a service with logging functionality.
///
/// This layer creates a `SimpleLogger` which logs request and response
/// details to stdout or tracing logs (depending on configuration).
#[derive(Clone)]
pub struct SimpleLoggerLayer;

impl<S> Layer<S> for SimpleLoggerLayer {
    type Service = SimpleLogger<S>;

    /// Wrap the given service with a `SimpleLogger`.
    fn layer(&self, inner: S) -> Self::Service {
        SimpleLogger::new(inner)
    }
}

SimpleLogger Service

This is the actual logging middleware. It implements the Service trait and wraps an inner service of type S. When a request comes in, it logs the request metadata, forwards the request to the inner service, and then logs the response or error.

/// A middleware service that logs incoming requests and outgoing responses.
///
/// This logger middleware captures basic request information (method and path)
/// and logs the resulting response status or any encountered error.
#[derive(Clone)]
pub struct SimpleLogger<S> {
    inner: S,
}

impl<S> SimpleLogger<S> {
    /// Creates a new `SimpleLogger` wrapping the provided inner service.
    pub fn new(inner: S) -> Self {
        Self { inner }
    }
}

impl<S, ReqBody, ResBody> Service<Request<ReqBody>> for SimpleLogger<S>
where
    S: Service<Request<ReqBody>, Response = Response<ResBody>> + Clone + Send + 'static,
    S::Future: Send + 'static,
    ReqBody: Debug + Send + 'static,
    ResBody: Debug + Send + 'static,
    S::Error: Debug + Send + 'static,
{
    type Response = Response<ResBody>;
    type Error = S::Error;
    type Future = Pin<Box<dyn Future<Output = Result<Self::Response, Self::Error>> + Send>>;

    /// Checks if the underlying service is ready to receive a request.
    ///
    /// This simply delegates to the wrapped service's `poll_ready` method.
    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        self.inner.poll_ready(cx)
    }

    /// Handles an incoming request by logging its metadata and then passing it to the inner service.
    ///
    /// After awaiting the response, it logs the status code or any error that occurred.
    fn call(&mut self, req: Request<ReqBody>) -> Self::Future {
        tracing::info!("--> Request: {} {}", req.method(), req.uri());

        // Clone the inner service to use it in the async block
        let mut inner = self.inner.clone();
        let fut = inner.call(req);

        // Log the response or error once the future resolves
        Box::pin(async move {
            let response = fut.await;
            match &response {
                Ok(res) => {
                    tracing::info!("<-- Response: {}", res.status());
                }
                Err(err) => {
                    tracing::error!("!! Error: {:?}", err);
                }
            }
            response
        })
    }
}

The Timing Layer and Service

(file timing.rs)

use std::{
    future::Future,
    pin::Pin,
    task::{Context, Poll},
    time::Instant,
};

// Hyper HTTP types
use hyper::{Request, Response};

// Tower traits
use tower::{Layer, Service};

/// A Tower `Layer` that wraps a service with timing functionality.
///
/// This layer produces a `TimingMiddleware` that logs how long each request takes
/// to complete. It is useful for performance monitoring and debugging latency.
#[derive(Clone)]
pub struct TimingLayer;

impl<S> Layer<S> for TimingLayer {
    type Service = TimingMiddleware<S>;

    /// Wrap the given service in the `TimingMiddleware`.
    fn layer(&self, inner: S) -> Self::Service {
        TimingMiddleware::new(inner)
    }
}

/// A middleware that measures and logs the time taken to process each request.
///
/// It starts a timer before forwarding the request to the inner service and logs
/// the elapsed time after the response is returned.
#[derive(Clone)]
pub struct TimingMiddleware<S> {
    inner: S,
}

impl<S> TimingMiddleware<S> {
    /// Creates a new instance of the `TimingMiddleware`.
    pub fn new(inner: S) -> Self {
        Self { inner }
    }
}

impl<S, ReqBody, ResBody> Service<Request<ReqBody>> for TimingMiddleware<S>
where
    S: Service<Request<ReqBody>, Response = Response<ResBody>> + Clone + Send + 'static,
    S::Future: Send + 'static,
    ReqBody: Send + 'static,
    ResBody: Send + 'static,
    S::Error: std::fmt::Debug + Send + 'static,
{
    type Response = Response<ResBody>;
    type Error = S::Error;
    type Future = Pin<Box<dyn Future<Output = Result<Self::Response, Self::Error>> + Send>>;

    /// Checks if the underlying service is ready to process a request.
    ///
    /// Delegates to the wrapped service's `poll_ready` method.
    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        self.inner.poll_ready(cx)
    }

    /// Processes the request and logs the elapsed processing time.
    ///
    /// - Starts a high-resolution timer (`Instant::now()`) just before forwarding the request.
    /// - Logs the duration after the response is received.
    fn call(&mut self, req: Request<ReqBody>) -> Self::Future {

        let mut inner = self.inner.clone();
        let start = Instant::now(); // Start timer

        Box::pin(async move {
            let result = inner.call(req).await; // Await inner service response
            let _duration = start.elapsed();    // Measure elapsed time
            tracing::info!("== Took {:.2?}", _duration); // Log duration

            result
        })
    }
}

The TimingMiddleware is a lightweight but powerful addition to your service stack. It gives immediate visibility into how long each request takes, which is essential for:

• Performance analysis
• Bottleneck detection
• Real-time diagnostics during development

It’s especially useful when combined with other middleware like SimpleLogger, giving both context and timing for each request.

The RateLimiter Layer and Service

The RateLimiter middleware (file rate_limiter.rs) provides a simple form of request throttling to prevent overloading a service by limiting how frequently requests can be handled. It’s implemented as a Tower Layer and Service, making it easy to compose within a middleware stack.

use std::{
    convert::Infallible,
    fmt::Debug,
    future::Future,
    pin::Pin,
    sync::Arc,
    task::{Context, Poll},
    time::{Duration, Instant},
};

// Tokio async synchronization primitive
use tokio::sync::Mutex;

// Hyper HTTP types
use hyper::{Request, Response, StatusCode};

// HTTP body utils
use bytes::Bytes;
use http_body_util::{combinators::BoxBody, BodyExt, Full};

// Tower traits
use tower::{Layer, Service};

/// A Tower `Layer` that adds basic rate-limiting behavior to a service.
///
/// Each request is allowed only after a fixed delay since the previous one.
/// This is a very simple rate limiter — it applies globally (not per client/IP).
#[derive(Clone)]
pub struct RateLimiterLayer {
    limit_duration: Duration,
}

#[allow(dead_code)]
impl RateLimiterLayer {
    /// Creates a new `RateLimiterLayer` with the specified delay between allowed requests.
    ///
    /// # Arguments
    ///
    /// * `per` - Minimum duration between two allowed requests.
    pub fn new(per: Duration) -> Self {
        Self {
            limit_duration: per,
        }
    }
}

impl<S> Layer<S> for RateLimiterLayer {
    type Service = RateLimiter<S>;

    /// Wraps the inner service with the `RateLimiter` middleware.
    fn layer(&self, inner: S) -> Self::Service {
        RateLimiter::new(inner, self.limit_duration)
    }
}

/// A middleware that rate-limits incoming requests using a simple time window.
///
/// Only one request is allowed per `limit_duration`. If a request comes in
/// too early, it responds with `429 Too Many Requests`.
#[derive(Clone)]
pub struct RateLimiter<S> {
    inner: S,
    state: Arc<Mutex<RateLimitState>>,
    limit_duration: Duration,
}

/// Holds the state for rate limiting, specifically the next allowed request time.
#[derive(Debug)]
struct RateLimitState {
    next_allowed: Instant,
}

impl<S> RateLimiter<S> {
    /// Constructs a new `RateLimiter` middleware.
    ///
    /// # Arguments
    ///
    /// * `inner` - The inner service to wrap.
    /// * `per` - Duration to wait between allowed requests.
    pub fn new(inner: S, per: Duration) -> Self {
        Self {
            inner,
            state: Arc::new(Mutex::new(RateLimitState {
                next_allowed: Instant::now(),
            })),
            limit_duration: per,
        }
    }
}

impl<S, ReqBody> Service<Request<ReqBody>> for RateLimiter<S>
where
    S: Service<
            Request<ReqBody>,
            Response = Response<BoxBody<Bytes, Infallible>>,
        > + Clone
        + Send
        + 'static,
    S::Future: Send + 'static,
    ReqBody: Send + 'static,
{
    type Response = Response<BoxBody<Bytes, Infallible>>;
    type Error = S::Error;
    type Future = Pin<Box<dyn Future<Output = Result<Self::Response, Self::Error>> + Send>>;

    /// Checks if the inner service is ready to handle a request.
    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        self.inner.poll_ready(cx)
    }

    /// Processes a request while enforcing rate limits.
    ///
    /// - If the request arrives before the allowed time, a 429 response is returned.
    /// - Otherwise, the request is passed through to the inner service.
    fn call(&mut self, req: Request<ReqBody>) -> Self::Future {

        let mut inner = self.inner.clone();
        let state = self.state.clone();
        let delay = self.limit_duration;

        Box::pin(async move {
            let mut state = state.lock().await;
            let now = Instant::now();

            // Check if the request comes too soon
            if now < state.next_allowed {
                tracing::warn!("Too Many Requests");

                let body = Full::new(Bytes::from_static(b"Too Many Requests")).boxed();
                let mut response = Response::new(body);
                *response.status_mut() = StatusCode::TOO_MANY_REQUESTS;

                return Ok(response);
            }

            // Update next allowed request time
            state.next_allowed = now + delay;
            drop(state); // Explicitly release the lock

            // Forward the request to the inner service
            inner.call(req).await
        })
    }
}

Code Overview

This middleware allows only one request per configured time window. If a request arrives too early, it is immediately rejected with a 429 Too Many Requests response.

Key components:

• RateLimiterLayer: A Tower Layer that wraps the target service with rate-limiting logic.
• RateLimiter: A Tower Service that performs the actual throttling.
• RateLimitState: Holds the Instant timestamp for when the next request is allowed.
• Arc<Mutex<...>>: Used to safely share the state across multiple clones of the service in an async context.

How It Works

• A single timestamp (next_allowed) tracks when the next request is permitted.
• If the current time is before next_allowed, the middleware returns “429 Too Many Requests”
• Otherwise, it updates next_allowed and forwards the request to the inner service.

This rate limiter is global: it does not distinguish between different clients or IP addresses. Every request shares the same time window.

Limitations

This implementation is meant for demonstration purposes and is not production-ready. For example, when used from a web browser, the page load may trigger multiple requests in quick succession — such as:

• The main page load (GET /)
• The browser’s automatic request for /favicon.ico

Since these happen almost simultaneously, the second request will likely be rejected with a 429 Too Many Requests.

What You’d Want in Production

For real-world applications, a more sophisticated rate limiter is recommended. Common improvements include:

• Token bucket or leaky bucket algorithms
• Per-client or per-IP tracking
• Sliding window support or burst tolerance

These approaches offer greater flexibility and fairness, especially in multi-user environments.

Token Bucket Variant (Preview)

I’ve also implemented a version of the rate limiter using the token bucket algorithm, which may be introduced in a later part of this series.

Design Goals

• A shared token bucket for all requests (simplified design)
• The bucket refills tokens at a fixed rate: e.g., 1 token per ms, up to a maximum capacity
• A request is only allowed if a token is available In that case, the number of available tokens is decremented.

Benefits

• Allows bursts of traffic up to the bucket’s capacity
• Smoothly handles traffic over time
• Returns 429 Too Many Requests if no tokens are left

This approach is much more forgiving for common browser behaviors and more suitable for production scenarios.

Middleware Module Definition

The mod.rs file inside the middleware/ directory defines the structure of the middleware module and re-exports its components for easy use elsewhere in the application.

pub mod echo;
pub mod logger;
pub mod rate_limiter;
pub mod timing;

pub use echo::EchoService;
pub use logger::SimpleLoggerLayer;
pub use rate_limiter::RateLimiterLayer;
pub use timing::TimingLayer;

How to Use the Layers in the Server

In the server, we compose the middleware stack using Tower’s ServiceBuilder. Each middleware layer wraps the service below it, forming a processing chain from outermost to innermost.

Here’s how the layers are applied in the build_service function:

/// Builds the complete Tower service stack using layers.
/// Returns a boxed, cloneable service object.
fn build_service(
) -> BoxCloneService<Request<Incoming>, Response<BoxBody<Bytes, Infallible>>, hyper::Error> {
    ServiceBuilder::new()
        .layer(RateLimiterLayer::new(Duration::from_millis(1))) // Optional rate limiter
        .layer(TimingLayer) // Measures request processing time
        .layer(SimpleLoggerLayer) // Optional logging middleware
        .service(EchoService) // Core service logic
        .boxed_clone() // Boxed cloneable service for use with hyper
}

What is boxed_clone()?

Tower’s .boxed_clone() method converts the layered service into a boxed trait object:

Convert the service into a [Service] + Clone + Send trait object. This is similar to the [boxed] method, but it requires that Self implement Clone, and the returned boxed service implements Clone. See BoxCloneService for more details.

This has several advantages:

• It erases the concrete type of the stack, making it easier to return from functions.
• The service becomes cloneable and sendable across threads.
• It’s required by some frameworks like Hyper, which often expect boxed trait objects for compatibility.

boxed_clone() is similar to .boxed(), but requires that the service implements Clone, and returns a boxed version that can also be cloned.

Layer Execution Order

When the returned BoxCloneService is used by the server (e.g., in a request handler), the layers are executed in the order they were added — top to bottom.

That means:

1. The Rate Limiter is checked first.
2. Then the Timing Layer starts measuring the request duration.
3. The Logger records request metadata.
4. Finally, the request is passed to the EchoService, which generates the response.

The response then flows back up through the layers in reverse, allowing things like logging and timing to capture final output details.

Conclusion

In this third part of the series, we took a deep dive into the middleware architecture that powers our high-performance and secure Rust server.

We introduced three Tower-based middleware layers — Rate Limiting, Timing, and Logging — and demonstrated how they wrap around a simple EchoService to form a clean, composable, and observable request pipeline.

Each layer is designed to handle one specific concern:

• RateLimiter: Controls request frequency to protect resources, though the version shown here is simplified and intended for demonstration only.
• TimingMiddleware: Measures and logs how long each request takes to process, which is critical for performance monitoring and latency diagnostics.
• SimpleLogger: Captures essential request and response information to support debugging and basic observability.

We also explained how these layers are stacked using Tower’s ServiceBuilder, and how the BoxCloneService abstraction allows us to pass the entire stack seamlessly into the Hyper-based server. Importantly, we discussed the limitations of the naive rate-limiting approach and previewed a more robust Token Bucket-based variant that can better handle burst traffic and real-world client behavior (e.g., browsers making multiple requests in parallel).

What’s Next

In Part 4, we’ll integrate these layers into a fully working HTTP/HTTPS server using Tokio + Tower + Hyper + Rustls, then use a simple HTTP client to benchmark its performance under load.