A flower does not think of competing; it just blooms.

Tiny Experiments — 5: Testing an API server running strongSwan application using pytest framework

REST API server using pytest framework

For validating the functionality of the [api_server.py](https://medium.com/@prasanth.swaminathan/0a63ae837068) REST APIs running on the Raspberry Pi, the pytest framework provides a simple, scalable, and automation-friendly testing approach. Pytest is a widely used Python testing framework that enables developers to write clear, concise test cases while supporting advanced capabilities such as fixtures, parameterization, and test automation.

In the context of this application, pytest is used to validate that the API endpoints responsible for namespace setup, IPsec configuration, SA status retrieval, traffic generation, and statistics collection behave correctly when invoked from a client system on the LAN.

The framework allows the REST APIs to be tested as black-box interfaces, where tests send HTTP requests (using libraries such as requests) and verify that the API responses and resulting system states match the expected outcomes.

Key value add when using pytest:

1. Simplicity and Readability: Pytest allows tests to be written using plain Python functions
2. pytest requires very little setup. Test discovery is automatic, and assertions are written using standard Python assert statements.
3. Automated Test discovery: Pytest automatically discovers tests based on naming conventions:

• Test files start with test_
• Test functions start with test_

4. Fixtures for Environment Setup: Pytest fixtures allow reusable setup steps to be defined once and used across multiple tests. This is particularly useful for preparing the lab environment, such as initializing namespaces or loading IPsec configurations.

5. Repeatable and Idempotent Testing: As the API server was designed with idempotent operations (for example the namespace setup API), pytest can repeatedly run tests without requiring manual cleanup.

A peek into the pytests’ fixtures and configuration, the foundational components of the framework, designed to provide a reliable, modular, and consistent context for running tests and its relevance to our setup:

Fixtures in pytest are reusable functions that prepare test environments or shared resources required by multiple tests. They allow setup logic to be defined once and reused across many test cases.

1. Pytest Fixtures

Fixtures in pytest are reusable functions that prepare test environments or shared resources required by multiple tests. They allow setup logic to be defined once and reused across many test cases.

In the context of the Raspberry Pi API server, fixtures can be used to:

• Initialize network namespaces (hostA, hostB)
• Run the IPsec setup API
• Prepare traffic generation environments

@pytest.fixture(scope="session")
def setup_ipsec():
    # Idempotent setup endpoint
    resp = post("/api/ipsec/setup", {})
    yield resp

2. Pytest Configuration

Pytest configuration allows common settings and parameters to be centralized so that test behavior can be controlled consistently across the entire test suite.

For testing the Raspberry Pi APIs, configuration can be used to store parameters such as:

API_BASE_URL = "http://192.168.1.100:8080"
NS_A = "hostA"
NS_B = "hostB"

Pytest does not strictly require a specific directory structure, but it follows standard naming conventions and automatically discovers tests based on these conventions and the directory hierarchy. Following is the directory structure outline, followed to test the REST APIs developed for the Raspberry Pi + strongSwan setup.

└── tests/
    ├── __init__.py
    ├── conftest.py                     # Test fixtures
    ├── config.py                       # The API_BASE url etc. to be used across tests
    ├── test_lib/
    │   ├── api.py                      # Defines the get(), post() methods and their corresponding processing
    │     
    ├── test_00_server_status.py        # Tests organized to handle the individual steps to setup/build the application
    ├── test_01_setup.py
    ├── test_02_init_host.py
    ├── test_02_load_conns.py
    ├── test_03_ipsec_child_add.py
    ├── test_04_ipsec_traffic.py
    ├── test_05_ipsec_sa_stats.py
    └── test_06_ipsec_cleanup.py

In pytest, assertions are made using the standard Python **assert** keyword, which the framework enhances with detailed introspection and reporting.

#  Assertion validating an API response (resp) returned a success status
assert resp["status"] in ["created", "already_exists"]

#  Assertion validating a key "setup_output" returned in the resp
assert "setup_output" in resp
assert len(resp["setup_output"]) > 0

# Handling an unexepcted response:
assert resp["status"] in ["created", "already_exists"], f"Expected ['created', 'already_exists'], but got {resp['status']}"

The get() and post() methods defined in the api.py that would be used all the test cases can be standardized to expect JSON responses. The APIs are to be developed following using the Flask's jsonify method, involves automatically converting Python data structures into a JSON-formatted response with the correct HTTP headers.

The test cases developed to test and verify the output of each of the APIs built and run using the api_server, can be individually tested for correctness, and later run together to setup them up as batch.

Logs from test cases run individually:

Test for a basic API server health check

Test for a setup the System — Namespaces, veth interfaces and basic ping

Test to Initialize the namespaces with strongSwan Instances and configuration

While Python’s standard assert statements are the core tool for validating test conditions in pytest, the framework significantly enhances them through a process called assertion rewriting: failing assertions in pytest provide highly detailed feedback, showing the values of subexpressions, attributes, comparisons, and operators involved in the failure.

An assertion error highlighting a mismatch of expected values, prompting a rewrite.

A look at the multiple, fine grained set of assertions that can be included in a test to make the test fool proof.

from test_lib.api import get, post  

def test_sa_stats():
    resp = get("/api/ipsec/stats", {"ns": "hostB"})

    #assert "stats" in resp
    #assert isinstance(resp["stats"], dict)

    ike = resp["event"]["net-test"]

    # Validate IKE SA State and IDs
    assert ike["state"] == "ESTABLISHED"
    assert ike["local-id"] == "hostB"
    assert ike["remote-id"] == "hostA"

    child = ike["child-sas"]["net-1"]

    # Validate Child SA State and Protocol
    assert child["state"] == "INSTALLED"
    assert child["name"] == "net"
    assert child["protocol"] == "ESP"

    # Validate Traffic Selectors
    assert child["local-ts"] == "10.10.1.0/28"
    assert child["remote-ts"] == "10.10.0.0/28"

    # Validate Packet Counters
    assert int(child["spi-in"], 16) > 0
    assert int(child["spi-out"], 16) > 0

A test case to check for the SA stats output invoking the swanctl — list-sas command

Pytest and its Integration with CI tools:

• Pytest can generate JUnit-style XML reports, which Jenkins can visualize and track automated regressions.
• A Jenkins Pipeline can be setup to checkout from Git, build the application, if required followed by running the pytest tests and publishing the results on the Jenkins’ build.
• CI Integration with GitHub Actions / GitLab CI

During testing, some practical challenges arise, such as ensuring APIs consistently return JSON responses, correctly handling query parameters for GET requests, and validating outputs that may include system command results.

The results are a repeatable and automated validation framework that verifies API behavior, IPsec tunnel establishment, interface configuration, and runtime statistics. Tests can confirm both API responses and underlying system state (e.g., Child SA status, SPI values, or veth interfaces), ensuring the control plane behaves as expected.

Finally, the pytest-based approach is extensible. Additional tests can be added for new endpoints, traffic validation, or error scenarios, and the test suite can be integrated with CI tools such as Jenkins or GitHub Actions.

The tests discussed above can be accessed here.

Recap of the Tiny experiments:

Tiny Experiments 1 — Exploring IPsec tunnels using veth pairs and strongSwan in a Standalone Linux Namespace architecture.

Tiny Experiments 2 — Exploring strongSwan’s commands to Manage and Monitor IPsec connections

Tiny Experiments 3 — Certificate based IKE authentication on veth pair lab setup

Tiny Experiments 4 — REST APIs to manage a strongSwan based veth pair lab setup.