True Test Coverage with Tracing

Improving test coverage by understanding the system’s behavior

There is no doubt about the importance of Test Coverage in software engineering, and today, with the increasing popularity of microservice architecture and distributed systems, it has become more critical. In this article, we want to see how to increase test coverage by using Tracing, which is one of the integral components of distributed systems. We will also examine best practices and tools in a sample project implemented using Java and Spring Boot.

Photo by Markus Spiske on Unsplash

· What is Test Coverage? ∘ Test Coverage vs Code Coverage · What is the gap to have a true test coverage? ∘ How can tracing data improve test coverage? ∘ Relation between end-to-end tests and Tracing data · Let’s get our hands dirty with real code ∘ How Digma can help us to have better Test Coverage ∘ Write integration test using MockWebServer ∘ Write end-to-end tests without mocking interactions · Final Thought

What is Test Coverage?

It is better to start by knowing what Test Coverage is exactly before discussing its conjunction with Tracing to improve Test Coverage. In short

The term Test Coverage refers to the extent to which a software product’s functionality has been tested.

It assesses how well the tests cover various aspects of the product functionality, such as different use cases, user scenarios, or edge cases. Test coverage can be measured at various levels, including:

• Functional Coverage
• Requirement Coverage
• Risk Coverage
• and more

Higher test coverage means more aspects of the software product have been tested, ensuring better quality.

Test Coverage vs Code Coverage

Two similar terms in software testing are Test Coverage and Code Coverage. However, they are not the same. Knowing the difference between the two is important and helpful for a better understanding of Test Coverage.

Code coverage is a measure that shows how many percentage percentages of lines, branches, functions, or conditions in the source code have been executed during testing.

Code Coverage focuses on the code rather than the functionality. Higher code coverage indicates that the tests have executed most of the code and will reduce the bugs or issues in the codebase.

Consequently, Test Coverage measures the degree to which the software’s functionality has been tested, while Code Coverage determines the proportion of the codebase the tests have executed.

What is the gap to have a true test coverage?

As mentioned in the previous section, implementing a distributed system is complex, and having acceptable test coverage for it is even more difficult.

A distributed system consists of multiple services that communicate with each other over the network.

This distributed nature introduces additional complexities in different aspects, such as communication, deployment, data consistency, and more.

To address these challenges, development teams need to adopt testing strategies such as:

• End-to-end testing
• Contract testing
• Leveraging tracing data
• Service virtualization

How can tracing data improve test coverage?

By leveraging tracing, we can improve test coverage by better understanding the system’s behavior and interactions between its components.

Without tracing, you may have tests in place to verify the functionality of each individual microservice. However, you may not have complete visibility into how these services interact with each other and whether the end-to-end flow.

Relation between end-to-end tests and Tracing data

To have even better coverage, end-to-end tests can help us a lot, instead of mocking interaction with other services, we try to run a real version of those services and interact with them, either by running them locally from code or in a docker container.

End-to-end testing enables us to ensure better test coverage, but it can be more challenging and resource-intensive to set up and maintain.

Since we interact with the real version of the system in end-to-end testing, we can analyze the system's runtime behavior using tracing data.

Let’s get our hands dirty with real code

caller-service interaction diagram

Let’s continue with an imaginary distributed application with three services (caller-service, decorate-service, and echo-service). These services are implemented using Java and Spring Boot (you can find the **source code here**).

[embed]GitHub - zarinfam/digma-tracing-test Contribute to zarinfam/digma-tracing-test development by creating an account on GitHub.github.com

As you can see in the diagram, the caller-service needs to call decorate-service and echo-service to build a given message from a REST API.

If we run all three projects and then call the /build API from the caller-service we will see the decorated result!

http :8080/caller/build/deli -> Echo:**deli**

How Digma can help us to have better Test Coverage

**Digma** is an IDE plugin that provides many handy observability features during development and allows us to trace our code quickly.

Digma can help developers identify areas in the code that have not been properly tested, allowing developers or testers to improve the effectiveness of the test suite.

They allow the developer to find the code test usage, which can be challenging in a distributed application due to several applications, various programming languages, and more. Digma pinpoints the tests that indirectly referenced the modified method, allowing the developer to select test execution and significantly reduce the testing time.

After **installing the Digma plugin** in your IDE, we need to re-run all three projects again. When we run the end-to-end test again. We will see the results in the Observability view in IntelliJ like this:

Digma’s Observability view in IntelliJ

To see tracing data, we can click on the Trace button on the right.

Digma’s Tracing view in IntelliJ

In addition to that, Digma gives us a lot of helpful insights about services such as:

• Scaling Issues
• Performance Impact
• Bottlenecks
• SQL Query Issues
• Chatty Microservices
• And more …

With Digma, we can also run all the tests that are going through the code you want to check.

Now that everything is running let’s start writing tests and using Digma to trace data to see how the code runs.

Write integration test using MockWebServer

One common practice for writing integration tests for this type of application is to mock service interactions. In our case, we will mock the echo-service and the decorator-service using the MockWebServer API.

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class CallerControllerIntegTest {
    private static int MOCK_SERVER_PORT;

    static {
        try (var serverSocket = new ServerSocket(0)) {
            MOCK_SERVER_PORT = serverSocket.getLocalPort();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    @DynamicPropertySource
    static void trackerProperties(DynamicPropertyRegistry registry) {
        String mockServerUrl = "http://localhost:" + MOCK_SERVER_PORT;
        registry.add("clients.decorate-service.url", () -> mockServerUrl);
        registry.add("clients.echo-service.url", () -> mockServerUrl);
    }

    private MockWebServer mockWebServer;

    @Autowired
    WebTestClient webTestClient;

    @BeforeEach
    void beforeEach() throws IOException {
        mockWebServer = new MockWebServer();
        mockWebServer.start(MOCK_SERVER_PORT);
    }

    @AfterEach
    void afterEach() throws IOException {
        mockWebServer.shutdown();
    }

    @Test
    void buildApi_validRequest_buildDecoratedMessage() {
        var decorateJsonResponse = new MockResponse()
                .setResponseCode(OK.value())
                .setBody("**test**")
                .addHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
        mockWebServer.enqueue(decorateJsonResponse);

        var echoJsonResponse = new MockResponse()
                .setResponseCode(OK.value())
                .setBody("""
                         {
                          "message": "Echo:**test**"
                         }
                        """)
                .addHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
        mockWebServer.enqueue(echoJsonResponse);

        webTestClient
                .get()
                .uri("/caller/build/test")
                .exchange()
                .expectStatus()
                .isOk()
                .expectBody(String.class)
                .isEqualTo("Echo:**test**");
    }
}

Testing services individually might not encompass all scenarios since it could overlook the dependencies and interactions between services.

If you have installed the Digma plugin in IntelliJ, when you run your integration or end-to-end tests in the IDE, Digma will automatically detect this and add observability to each test context, and you can see related tests to an asset in the Digma view in the test tab.

Digma’s Test view in IntelliJ

As you can see in the image above, Digma allows us to see the Test Coverage for each asset (in our case, assets are endpoints, but in general, they can be database queries or code locations). The main benefit is that we can see which tests are covering our endpoints and check the Test Coverage practically. We can also re-run tests as needed after making changes from the Digma view.

By clicking on the Open Trace button, you can see this trace view inside the IDE. As you can see, we can not see any real interaction between the caller-service and other services since we mocked the dependencies.

Digma’s tracing view for tests in IntelliJ

Write end-to-end tests without mocking interactions

Let’s go one step further and write end-to-end tests, for the sake of simplicity, We don't want to containerize services and use the Spring Boot ServiceConnection features to test them. If you are interested in more information about ServiceConnection, read the article below:

[embed]Using Spring Boot 3.1 integration with Testcontainers in Development time The magic of @ServiceConnection annotation!itnext.io

In this way, our end-to-end tests will be very simple:

class CallerServiceEndToEndTests {

    @Test
    void buildApi_validRequest_buildDecoratedMessage() {
        WebTestClient.bindToServer().baseUrl("http://localhost:8080").build()
                .get()
                .uri("/caller/build/deli")
                .exchange()
                .expectStatus()
                .isOk()
                .expectBody(String.class)
                .isEqualTo("Echo:**deli**");
    }
}

The only thing that you need to consider is we need to run all three services before running our end-to-end test.

If we follow the same process in the integration test section in the Digma view, we will see real interaction between the services in the Digma’s trace view.

Digma’s tracing view for tests in IntelliJ

It can be a great source of observability for us during development. We will be able to study the application in a controlled experiment, which is a huge opportunity to uncover issues and study how the system behaves to increase the Test Coverage.

Final Thought

Proper test Coverage is challenging in distributed systems, but the article shows that we can significantly improve it by writing end-to-end tests and leveraging tracing data provided by tools like Digma.

🙏 Thanks for reading. You can connect with me on:

**🖊️ Medium | [🐦 Twitter](https://twitter.com/zarinfam)**