Tokio, Tower, Hyper and Rustls: Building High-Performance and Secure Servers in Rust — Part 7

In this part of the series, we introduce two critical components that complete our secure and modular Rust server architecture:

1. A Path-Based Routing Service Acts as a lightweight reverse proxy that dispatches HTTP requests to backend services based on configurable path prefixes. This service handles TLS, authentication, and middleware, offering a secure entry point into a microservice system.
2. A Benchmarking Client A high-performance client that supports mutual TLS (mTLS) and JWT authentication. It’s designed for stress testing and analyzing performance across different service configurations, complementing tools like wrk.

Routing Service: Secure Reverse Proxy with Path Prefix Routing

The Routing Service examines each request’s path and forwards it to the appropriate backend, based on configurable prefix-based rules. This makes it ideal for microservice-based architectures where individual services are hidden behind a unified interface.

Backends typically run on private networks or are shielded by firewalls. While internal communication can use plain HTTP, all services can benefit from middleware such as logging, metrics, and authentication layers.

Example Configuration

Below is an example configuration where the routing service is deployed alongside three backend services:

# number of threads for Tokio to use for the server, 
# if not set, it defaults to the number of CPU cores times 2
tokio_threads=48
...
[[Server]]
ip= "192.168.178.31"
port=1337
# enable this server, defaults to false
enabled=true
# any name to identify server for logging, no further meaning
name = "routing service"
...
# important: we use the routing-service
service = "Router"
enabled = [...]
protocol = "HTTPS"
authentication = "JWT"
# enable the layers on top of 
[Server.Layers]
enabled = ["SimpleLogger","Timing"]
...
# the reverse routes for the Router Service
[Server.ReverseRoutes]
"/help" = "https://192.168.178.31:1338"
"/static" = "http://192.168.178.31:1339"
"/api" = "http://192.168.178.31:1330" 

The reverse routes define which backend URI each path prefix should be forwarded to. Requests are matched by prefix, and the router dynamically builds the full URI before forwarding the request.

Target Services

Each of the routed endpoints corresponds to its own backend server. For example:

...

# /help handler service
[[Server]]
name = "base_help_service"
protocol = "HTTPS"
ip="192.168.178.31"
port = 1338
service = "Echo"
enabled=true
[Server.Layers]
enabled = ["SimpleLogger","Timing"]
...

# /static handler service
[[Server]]
name = "base_static_service"
protocol = "HTTPS"
ip="192.168.178.31"
port = 1339
service = "Echo"
enabled=true
...
# define here further servers
...

Architecture Overview

In this setup, we define three cooperating services:

1. Routing service on port 1337

• Accepts external requests
• Secured with HTTPS and JWT authentication
• Routes requests based on path prefixes

2. Help service on port 1338

• Handles all /help requests
• Secured with HTTPS

3. Static service on port 1339

• Handles all /static requests
• Secured with HTTPS

Each backend is isolated, independently configurable, and secured behind the router.

RouterService Code

The RouterService is a custom implementation of Tower’s Service trait. It wraps a Hyper client (with HTTPS via Rustls) and forwards requests to backends according to path-based routing rules.

Key Responsibilities

• Load routing-rules from Config.toml
• Match incoming paths to backend URIs
• Forward requests and return backend responses

Construction

pub fn new(
    routes: Option<HashMap<String, String>>,
    server_name: impl Into<String>
) -> Self

• Builds a Hyper client using hyper-rustls, supporting both HTTP and HTTPS.
• Parses and sorts routes by prefix length (longest first)
• Ensures specific matches take precedence over general ones

Request Handling (call() method)

fn call(&mut self, request: Request<Incoming>) -> Self::Future

1. Extract Request Metadata:

• The URI path is extracted from the request.
• Routing rules are checked to find the best matching prefix.

2. Routing Decision:

• If no prefix matches, a 400 Bad Request is returned.
• If a match is found, the URI is rebuilt using the backend base + original path/query.

3. Request Forwarding:

• The request is reconstructed with the updated URI.
• A new HTTP/1.1 request is sent using the internal Hyper client.
• The response from the backend is returned to the original client.

4. Error Handling:

• If URI construction fails or body parsing fails, appropriate error responses are generated (400 Bad Request).

The full code

use std::{
    collections::HashMap,
    future::Future,
    pin::Pin,
    sync::Arc,
    task::{Context, Poll},
};

use bytes::Bytes;
use http_body_util::{BodyExt, Full};
use hyper::{Request, Response, StatusCode, body::Incoming, http::uri::Uri};
use hyper_util::client::legacy::Client;
use hyper_util::rt::TokioExecutor;
use tower::Service;
use tracing::trace;

use server::ServiceRespBody;
use server::SrvError; // BoxBody<Bytes, SrvError>

/// The `RouterService` is a Tower-compatible HTTP service that performs
/// prefix-based routing of incoming requests to different backend URIs.
/// It wraps a Hyper client, applies routing rules, and forwards requests
/// to the matching backend, rewriting the request URI as needed.
#[derive(Clone, Debug)]
pub struct RouterService {
    /// Hyper client instance with HTTPS support.
    client: Client<
        hyper_rustls::HttpsConnector<hyper_util::client::legacy::connect::HttpConnector>,
        ServiceRespBody,
    >,
    /// Vector of routing rules: (path prefix, backend URI), sorted by prefix length (longest first).
    rules: Arc<Vec<(String, Uri)>>, // (prefix, backend_uri)
    /// The server name (for logging and diagnostics).
    server_name: String,
}

impl RouterService {
    /// Constructs a new `RouterService`.
    ///
    /// # Arguments
    ///
    /// * `routes` - Optional routing table mapping path prefixes to backend URI strings.
    /// * `server_name` - Name of the server for logging.
    ///
    /// # Returns
    ///
    /// Returns a fully initialized RouterService.
    pub fn new(routes: Option<HashMap<String, String>>, server_name: impl Into<String>) -> Self {
        let server_name = server_name.into();

        // Build HTTPS connector with native root certificates.
        let https = hyper_rustls::HttpsConnectorBuilder::new()
            .with_native_roots()
            .expect("no native root CA certificates found")
            .https_or_http()
            .enable_http1()
            .build();

        // Construct a Hyper client with the HTTPS connector.
        let client: Client<
            hyper_rustls::HttpsConnector<hyper_util::client::legacy::connect::HttpConnector>,
            ServiceRespBody,
        > = Client::builder(TokioExecutor::new()).build(https);

        // Process and sort routing rules by prefix length (descending).
        let mut rules_vec: Vec<_> = match routes {
            Some(map) => map
                .into_iter()
                .filter_map(|(prefix, uri_str)| match uri_str.parse::<Uri>() {
                    Ok(uri) => Some((prefix, uri)),
                    Err(e) => {
                        tracing::warn!(
                            "{server_name}: Invalid URI in routing rules ({}): {}",
                            prefix,
                            e
                        );
                        None
                    }
                })
                .collect(),
            None => Vec::new(),
        };
        // Ensure longest prefixes are checked first for correct routing.
        rules_vec.sort_by(|(a, _), (b, _)| b.len().cmp(&a.len()));

        RouterService {
            client,
            rules: Arc::new(rules_vec),
            server_name,
        }
    }
}

impl Service<Request<Incoming>> for RouterService {
    type Response = Response<ServiceRespBody>;
    type Error = SrvError;
    type Future = Pin<Box<dyn Future<Output = Result<Self::Response, Self::Error>> + Send>>;

    /// Checks if the service is ready to accept a request.
    /// Always returns ready (this service is always ready).
    fn poll_ready(&mut self, _cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        trace!("poll_ready im router");
        Poll::Ready(Ok(()))
    }

    /// Handles an incoming HTTP request, applies routing rules, and forwards the
    /// request to the corresponding backend using the Hyper client.
    ///
    /// # Arguments
    ///
    /// * `request` - The incoming HTTP request.
    ///
    /// # Returns
    ///
    /// Returns a future that resolves to the proxied response from the backend,
    /// or an error response if no routing rule matches.
    fn call(&mut self, request: Request<Incoming>) -> Self::Future {
        trace!("call im router");
        let rules = Arc::clone(&self.rules);
        let client = self.client.clone();
        let server_name = self.server_name.clone();

        Box::pin(async move {
            let (request_parts, request_body) = request.into_parts();
            let request_path = request_parts.uri.path();

            // Prioritized rule match (sorted longest first for prefix matching).
            // Ensures for example /api matches /api, /api/v1, but not /apis
            let target_uri = rules
                .iter()
                .find(|(prefix, _)| {
                    request_path == prefix
                        || (request_path.starts_with(prefix)
                            && request_path.chars().nth(prefix.len()) == Some('/'))
                })
                .map(|(_, uri)| uri.clone());

            let target_uri = match target_uri {
                Some(uri) => uri,
                None => {
                    // No matching backend found; respond with 400 Bad Request.
                    tracing::warn!("{server_name}: NO MATCHING BACKEND FOR PATH: {request_path}");

                    #[cfg(feature = "boxed_body")]
                    let body: ServiceRespBody = Full::new(Bytes::from("No matching backend"))
                        .map_err(SrvError::from)
                        .boxed();
                    #[cfg(not(feature = "boxed_body"))]
                    let body: ServiceRespBody = Full::new(Bytes::from("No matching backend"));
                    let mut response = Response::new(body);
                    *response.status_mut() = StatusCode::BAD_REQUEST;
                    return Ok(response);
                }
            };

            // Construct a new URI by combining the target backend base with the request's path/query.
            let response_uri = {
                let mut target_uri_parts = target_uri.into_parts();
                target_uri_parts.path_and_query = request_parts.uri.path_and_query().cloned();
                Uri::from_parts(target_uri_parts)
            };

            let mut response_parts = request_parts;
            response_parts.uri = match response_uri {
                Ok(uri) => uri,
                Err(_) => {
                    // URI parts construction failed; respond with 400 Bad Request.
                    tracing::warn!("{server_name}: INVALID URI PARTS: {}", response_parts.uri);
                    #[cfg(feature = "boxed_body")]
                    let body: ServiceRespBody = Full::new(Bytes::from("Invalid Request Uri"))
                        .map_err(SrvError::from)
                        .boxed();
                    #[cfg(not(feature = "boxed_body"))]
                    let body: ServiceRespBody = Full::new(Bytes::from("Invalid Request Uri"));
                    let mut response = Response::new(body);
                    *response.status_mut() = StatusCode::BAD_REQUEST;
                    return Ok(response);
                }
            };

            // Always use HTTP/1.1 for backend requests.
            response_parts.version = hyper::Version::HTTP_11;
            // Optional: update the Host header to match the backend (commented out).
            // response_parts.headers.insert(
            //     hyper::header::HOST,
            //     hyper::header::HeaderValue::from_str(&response_parts.uri.authority().unwrap().to_string())
            //         .unwrap(),
            // );

            trace!("{server_name}: Forwarding request to: {:?}", response_parts);

            // Prepare the request body, handling boxed/non-boxed bodies via feature flag.
            #[cfg(feature = "boxed_body")]
            let body: ServiceRespBody = request_body.map_err(SrvError::from).boxed();
            #[cfg(not(feature = "boxed_body"))]
            let body = Full::new(request_body.collect().await?.to_bytes());

            // Build the forwarded request to send to the backend.
            let forwarded_request: Request<ServiceRespBody> =
                Request::from_parts(response_parts, body);

            // Forward the request to the matched backend.
            let response = client.request(forwarded_request).await?;
            trace!("{server_name}: router response: {:?}", response);

            // Adapt the backend response body as needed by feature.
            #[cfg(feature = "boxed_body")]
            {
                let response: Response<ServiceRespBody> =
                    response.map(|b| b.map_err(SrvError::from).boxed());
                return Ok(response);
            }
            #[cfg(not(feature = "boxed_body"))]
            {
                let (parts, body) = response.into_parts();
                let bytes = body.collect().await?.to_bytes();
                let response: Response<ServiceRespBody> =
                    Response::from_parts(parts, Full::new(bytes));
                return Ok(response);
            }
        })
    }
}

The implementation supports both boxed and unboxed HTTP bodies, configurable via the boxed_body Cargo feature. This makes the router adaptable to various runtime and memory efficiency requirements.

Summary of RouterService Benefits

• Lightweight and composable routing layer
• Dynamically forwards requests based on path prefixes
• Integrates with full service stack: TLS, authentication, logging, etc.
• Keeps internal services secure and separated behind reverse proxy logic

Feature Flag Support: The implementation supports both boxed and unboxed HTTP bodies using a Cargo feature (boxed_body), making the service more flexible for different deployments.

A HTTP/HTTPS/mTLS/JWT Client for Performance Measurements

To evaluate the performance of different server configurations (e.g., plain HTTP, HTTPS with TLS, mTLS, and JWT-secured endpoints), a custom benchmarking client was developed. While tools like wrk are excellent for basic load testing, this client adds the following capabilities:

• Supports HTTP, HTTPS, mTLS, and JWT authentication
• Sends a configurable number of requests (--num_req) in parallel batches (--num_parallel)
• Reports performance metrics, including total duration and average requests per second

This makes it ideal for profiling layered services built using Hyper, Tower, and Rustls.

How It Works

Command-Line Interface (CLI)

The client is configured via CLI using the clap crate. You can specify:

• Number of requests (-i)
• Concurrency level (--num_parallel)
• Security mode (-s: http, https, jwt, mtls)
• CA certificate path (--ca)
• Client certificate/key for mTLS (--cert, --key)
• JWT file path (--jwt)
• Server URI and path

Example Usage

# JWT-secured HTTPS requests
RUST_LOG=client=trace cargo run --bin=client -- --ca="./ca.pem" -p / -j ./token.jwt -s jwt -i 800

# mTLS-secured HTTPS requests
RUST_LOG=client=trace cargo run --bin=client -- --ca="./ca.pem" -p / \
  -c ./client.cert.pem -k ./client.key.pem -s mtls -i 65000

The full Code

use std::time::Instant;

use anyhow::Context;
use anyhow::Error;
use bytes::Bytes;
use clap::{Parser, ValueEnum};

use futures::StreamExt;
use futures::stream::FuturesUnordered;

use http_body_util::{BodyExt, Full};
use hyper::{Request, http::uri::Uri};
use hyper_util::{client::legacy::Client, rt::TokioExecutor};
use rustls::{ClientConfig, RootCertStore};
use tracing::{error, trace};

use server::utils;

/// Entrypoint: parse CLI, validate, build client, and launch parallel requests.
#[tokio::main]
async fn main() -> Result<(), Error> {
    //console_subscriber::init();
    tracing_subscriber::fmt::init();

    let cli = Cli::parse();
    // check cli params
    validate_cli(&cli);

    // read JWT token if jwt is specified
    let jwt_token = read_jwt(&cli);

    // build a https or http client based on the protocol
    // and build the URI from the CLI input
    let client = build_client(&cli);
    let uri = build_uri(&cli);

    // Launch N requests in parallel and print throughput
    let concurrency = cli.num_parallel as usize; // Tune this with parameter --num_parallel; Maybe 100, maybe 500 depending on your machine/network
    let total_requests = cli.num_req as usize; // Send this many total requests

    let mut futs = FuturesUnordered::new();
    let start = Instant::now();
    let mut completed = 0;
    let mut launched = 0;

    // Kick off up to `concurrency` (cli.num_parallel) requests to start
    let first_batch = std::cmp::min(concurrency, total_requests);
    for _ in 0..first_batch {
        futs.push(do_request(&client, &cli, jwt_token.as_deref(), uri.clone())); // fut has now a len() of first_batch
        launched += 1;
    }

    // Now poll futures, and for each completed one, launch a new until we reach total_requests
    while completed < total_requests {
        if let Some(res) = futs.next().await {
            // futs have the size concurrency
            completed += 1;
            match res {
                Ok(_) => {}
                Err(e) => error!("Request failed: {e}"),
            }

            if launched < total_requests {
                futs.push(do_request(&client, &cli, jwt_token.as_deref(), uri.clone()));
                launched += 1;
            }
        }
    }

    let duration = start.elapsed();
    let mean = total_requests as f64 / duration.as_secs_f64();
    trace!(
        "\nDuration: {}ms with {} total requests at concurrency {}\nMean requests per second: {mean:.0}  --> per request: {:.1}us",
        duration.as_millis(),
        total_requests,
        concurrency,
        1000000. / mean
    );

    Ok(())
}

/// Command-line arguments for configuring the client.
#[derive(Parser, Debug)]
#[clap(author, version, about)]
struct Cli {
    #[clap(short = 'i', long, default_value_t = 10)]
    num_req: u16,
    #[clap(long, default_value_t = 128)]
    num_parallel: u16,
    #[clap(short = 's', long, value_enum, default_value_t = Protocol::Https)]
    security: Protocol,
    #[clap(short = 'r', long, value_name = "Root ca")]
    ca: Option<String>,
    #[clap(long, short = 'j', value_name = "jwt file")]
    jwt: Option<String>,
    #[clap(short = 'c', long, value_name = "client cert")]
    cert: Option<String>,
    #[clap(short = 'k', long, value_name = "client-key")]
    key: Option<String>,
    #[clap(short = 'm', long, default_value = "GET", value_name = "request method")]
    method: String,
    #[clap(short = 'u', long, default_value = "192.168.178.31:1337")]
    uri: String,
    #[clap(long, short = 'p', default_value = "/", value_name = "request path")]
    path: String,
}

/// Supported protocols.
#[derive(Debug, Clone, ValueEnum, PartialEq)]
enum Protocol {
    Http,
    Https,
    Jwt,
    Mtls,
}

/// Enum to hold either an HTTP or HTTPS client.
enum MultiProtocolClient {
    Http(Client<hyper_util::client::legacy::connect::HttpConnector, Full<Bytes>>),
    Https(
        Client<
            hyper_rustls::HttpsConnector<hyper_util::client::legacy::connect::HttpConnector>,
            Full<Bytes>,
        >,
    ),
}

impl MultiProtocolClient {
    /// Dispatch request based on client type.
    pub async fn request(
        &self,
        req: Request<Full<Bytes>>,
    ) -> Result<hyper::Response<hyper::body::Incoming>, hyper_util::client::legacy::Error> {
        match self {
            MultiProtocolClient::Http(client) => client.request(req).await,
            MultiProtocolClient::Https(client) => client.request(req).await,
        }
    }
}

/// Validate CLI arguments according to the protocol requirements.
fn validate_cli(cli: &Cli) {
    match cli.security {
        Protocol::Https => {
            if cli.ca.is_none() {
                // error!("--ca <Root ca> must be set for HTTPS");
                // std::process::exit(1);
            }
        }
        Protocol::Jwt => {
            if cli.ca.is_none() {
                // error!("--ca <Root ca> must be set for JWT");
                // std::process::exit(1);
            }
            if cli.jwt.is_none() {
                error!("--jwt <jwt file> must be set for JWT");
                std::process::exit(1);
            }
        }
        Protocol::Mtls => {
            if cli.ca.is_none() {
                // error!("--ca <Root ca> must be set for mTLS");
                // std::process::exit(1);
            }
            if cli.cert.is_none() || cli.key.is_none() {
                error!("--cert <client cert> and --key <client-key> must be set for mTLS");
                std::process::exit(1);
            }
        }
        Protocol::Http => {}
    }
}

/// Build a root certificate store from system and custom roots.
fn build_root_store(ca_path: &Option<String>) -> RootCertStore {
    let mut root_store = RootCertStore::empty();
    root_store.extend(webpki_roots::TLS_SERVER_ROOTS.iter().cloned());
    match ca_path {
        Some(path) => {
            let root_cert = utils::load_certs(path, "mtlsclient");
            root_store.add_parsable_certificates(root_cert);
        }
        None => {}
    }
    root_store
}

/// Create a rustls ClientConfig, with or without mTLS.
fn build_tls_config(
    root_store: RootCertStore,
    cert: Option<&str>,
    key: Option<&str>,
) -> ClientConfig {
    match (cert, key) {
        (Some(cert_path), Some(key_path)) => {
            let certs = utils::load_certs(cert_path, "mtlsclient");
            let key = utils::load_single_key(key_path, "mtlsclient");
            ClientConfig::builder()
                .with_root_certificates(root_store)
                .with_client_auth_cert(certs, key)
                .expect("Failed to build client config")
        }
        _ => ClientConfig::builder()
            .with_root_certificates(root_store)
            .with_no_client_auth(),
    }
}

/// Build the appropriate HTTP(S) client based on the protocol.
fn build_client(cli: &Cli) -> MultiProtocolClient {
    match cli.security {
        Protocol::Http => {
            let client = Client::builder(TokioExecutor::new()).build_http::<Full<Bytes>>();
            MultiProtocolClient::Http(client)
        }
        Protocol::Https | Protocol::Jwt => {
            let root_store = build_root_store(&cli.ca);
            let config = build_tls_config(root_store, None, None);
            let https = hyper_rustls::HttpsConnectorBuilder::new()
                .with_tls_config(config)
                .https_only()
                .enable_http1()
                .build();
            let client = Client::builder(TokioExecutor::new()).build(https);
            MultiProtocolClient::Https(client)
        }
        Protocol::Mtls => {
            let root_store = build_root_store(&cli.ca);
            let config = build_tls_config(root_store, cli.cert.as_deref(), cli.key.as_deref());
            let https = hyper_rustls::HttpsConnectorBuilder::new()
                .with_tls_config(config)
                .https_only()
                .enable_http1()
                .build();
            let client = Client::builder(TokioExecutor::new()).build(https);
            MultiProtocolClient::Https(client)
        }
    }
}

/// Reads a JWT token from the provided file, if applicable.
fn read_jwt(cli: &Cli) -> Option<String> {
    if let Protocol::Jwt = cli.security {
        let path = cli.jwt.as_ref().unwrap();
        Some(
            std::fs::read_to_string(path)
                .expect("Failed to read JWT")
                .trim()
                .to_string(),
        )
    } else {
        None
    }
}

/// Build the full request URI based on the protocol and user input.
fn build_uri(cli: &Cli) -> Uri {
    let scheme = match cli.security {
        Protocol::Https | Protocol::Mtls | Protocol::Jwt => "https",
        Protocol::Http => "http",
    };
    let uri = format!("{}://{}{}", scheme, cli.uri, cli.path);
    let uri = uri.parse().expect("Invalid URI");
    trace!("Built URI: {}", uri);
    uri
}

/// Send a single request, set Authorization if JWT, 
/// and return the response body as String.
async fn do_request(
    client: &MultiProtocolClient,
    cli: &Cli,
    jwt_token: Option<&str>,
    uri: Uri,
) -> Result<String, Error> {
    let mut builder = Request::builder()
        .method(cli.method.to_uppercase().as_str())
        .uri(&uri);

    if cli.security == Protocol::Jwt {
        if let Some(token) = jwt_token {
            builder = builder.header("Authorization", format!("Bearer {token}"));
        }
    }

    let request = builder
        .body(Full::new(Bytes::from_static(b"Hello, World!")))
        .expect("Failed to build request");

    // trace!("Request: {:#?}", request);

    let response = client.request(request).await?;
    let status = response.status();

    // If you want non-2xx to be errors:
    if !status.is_success() {
        // Collect the response body for debugging (optional)
        let (_parts, body) = response.into_parts();
        let body_bytes = body
            .collect()
            .await
            .context("Failed to read error response body")?
            .to_bytes();
        let body_str = String::from_utf8_lossy(&body_bytes);
        return Err(anyhow::anyhow!(
            "Request to {uri} failed with status {status}: {body_str}"
        ));
    }

    // Collect body and handle errors
    let (_parts, body) = response.into_parts();
    let body_bytes = body
        .collect()
        .await
        .context("Failed to read response body")?
        .to_bytes();

    // Parse body as UTF-8, error if invalid
    let body_str =
        String::from_utf8(body_bytes.to_vec()).context("Response body was not valid UTF-8")?;

    Ok(body_str)
}

This client is a practical tool for benchmarking and verifying the performance of HTTP servers secured with TLS, JWT, and mTLS. It’s particularly useful when working with Tower-based service stacks where different layers may affect performance.

Key Features:

• Full support for HTTPS and mTLS (via rustls)
• JWT support for token-based authentication
• Parallel, asynchronous execution with detailed timing
• Easily configurable via command line

It complements tools like wrk by offering custom protocol handling, header injection, and fine-grained error diagnostics, which are essential for testing layered services.

Conclusion

To evaluate the performance of our server, we ran a stress test using [wrk](https://github.com/wg/wrk):

wrk -t16 -c256 -d30s https://192.168.178.26:1337

The server was configured with 64 Tokio worker threads and used the “Echo” service with HTTPS:

Running 30s test @ https://192.168.178.26:1337
  16 threads and 256 connections
  Thread Stats   Avg      Stdev     Max       +/- Stdev
    Latency      310.00us 482.27us  61.44ms   98.82%
    Req/Sec      49.43k   2.94k     56.29k    93.64%
  23,679,094 requests in 30.09s, 1.76GB read

Requests/sec: 786,923.45
Transfer/sec: 60.04MB

Interpretation

• Throughput: The server handled nearly 787,000 requests per second, demonstrating excellent scalability and efficient async processing.
• Latency: With an average latency of around 310 microseconds, response times remain very low — even under high concurrency (256 open connections).
• Stability: Over 98% of the requests stayed within one standard deviation of the mean latency, indicating consistent and predictable performance.

Final Thoughts

This result underscores the power of combining Tokio, Tower, Hyper, and Rustls to build high-performance, secure, and modular services in Rust. Our layered architecture — with path-based routing, TLS, and middleware support — scales well under load while keeping internal services cleanly separated and secure.

In the next part of the series, we’ll explore the benchmarking client in more depth and how it can be used for performance comparison across different configurations (e.g., varying authentication methods, TLS settings, or middleware stacks).

Note: I found a bug in the routing service’s certificate handling. Currently, only HTTP can be sent without errors. However, the bug has been found. -> The Bug has been fixed in the meantime