Validating API requests on E2E tests using WDIO

Have you ever deployed your app to production and hours later realized that an API request was missing or incorrect? This might happen, for example, on analytics calls, that can be easily overlooked, or on really big projects, on which another team can mess up some parameters unintentionally. But don’t worry, end-to-end tests are a great solution to ensure that everything is working fine and you are firing your requests at the right time and using the correct data.

Not all solutions work for every test scenario due to the tool’s limitations or security measures. Because of that, we are going to explore some possibilities, so you can find the best method for your case.

For the examples, I’m going to use WebDriverIO (WDIO), which is a widely-used web automation framework that supports both Selenium and DevTools, and I’ll be evaluating the solutions using five different criteria (these are usually the ones that can cause you problems when trying to test the requests):

1. Can capture API requests;
2. Can inspect data from request/response;
3. Works for requests made in page load phase;
4. Is actively maintained;
5. Works when running tests on cloud platforms.

Intercepting the fetch method

If we are thinking of intercepting requests, let’s start with something simple. Why not replace the browser’s fetch function with our own? Then, we could add extra functionalities to validate the data.

Let’s try it. First, we need to keep a copy of the original fetch so we can call it at the end of our replacement function. Next, replace the fetch with a function that receives all the original arguments, validates them, and calls the original fetch — for the sake of brevity, we will skip validation and only add a console.log on the examples, but feel free to try your own implementations. It will look something like this:

it('should intercept API calls', async () => {
  await browser.url('my-website.com');

  await browser.execute(() => {
    const originalFetch = window.fetch;
    window.fetch = async (...args) => {
      console.log(args);

      return originalFetch.apply(this, args);
    };
  });

  const button = $('#my-button');

  await button.scrollIntoView();
  await button.click();
});

Seems like it would work, right? Well… not quite. WebDriver (and tools like WDIO) runs injected scripts in a sandboxed execution environment. This is similar to how Chrome extensions run in an “isolated world”: you can see the DOM, but you don’t actually share the same JavaScript context as the page itself. So when you overwrite the fetch function, you’re only changing it in the automation’s context — which will not be useful to intercept the requests.

And finally, here’s how this solution stacks up against our five criteria:

1. Can capture API requests ❌
2. Can inspect data from request/response ❌
3. Works for requests made in page load phase ❌
4. Is actively maintained ❌
5. Works when running tests on cloud platforms ❌

Intercept Service

One major advantage of WDIO is the number of available plugins, so you can add some extra functionalities to your automated tests. The Intercept Service is one example, which under the hood overwrites the fetch and XMLHTTPRequest functions, but it is able to bypass the isolated test context.

To enable it, you need to first download the package and add it to the services array:

npm install wdio-intercept-service -D

// wdio.conf.ts

export const config: WebdriverIO.Config = {
  // Other configs
  services: [
    // Other services
    'intercept'
  ],
};

If you are using TypeScript, you can also add the types on tsconfig:

// tsconfig.json

{
  "compilerOptions": {
    "types": [
      // Other types
      "wdio-intercept-service"
    ]
  }
}

Now, we have multiple ways of using this plugin to validate our requests. For example, you can use the expectRequest function to assert if your API was called at one specific endpoint and returned the expected status code:

it('should intercept API calls', async () => {
  await browser.url('my-website.com');
  browser.setupInterceptor();

  browser.expectRequest('POST','/my/api/route', 200);

  const button = $('#my-button');

  await button.scrollIntoView();
  await button.click();

  browser.disableInterceptor();
});

In this example, we were able to validate if the request was successful, but it might not be enough. To validate extra request details — like the headers or body — we can use the getRequests function to get the list of all API calls and then make our own checks.

it('should intercept API calls', async () => {
  await browser.url('my-website.com');
  browser.setupInterceptor();

  const button = $('#my-button');

  await button.scrollIntoView();
  await button.click();
  await browser.pause(1000);
  const requests = await browser.getRequests();

  console.log(requests);

  browser.disableInterceptor();
});

The snippet above shows that the intercept service works really well when we are testing calls made after a user interaction (like a button click). However, there are still some limitations. On the plugin’s official docs, it already mentions that API calls made during the page’s load phase will not be captured, simply because the service will be injected on the page after it’s fully loaded. Also, if the page is redirected right after the call, it might also not be able to catch them, so your tests will fail even though they were supposed to pass. It has also occurred to me — and some other GitHub users who raised issues on the plugin’s repository — that sometimes not all requests are captured, and you might end up with an empty array. The reason why this happens hasn’t been addressed yet, and neither fixed (in fact, the last update on the code was made in early 2024).

Despite the limitations and problems, this plugin might work for what you are looking for, but I don’t recommend using it, because, as we’ve seen, the project is no longer under support, and you might encounter more problems than solutions. In conclusion, we are starting to have some of our criteria met, but it’s not ideal yet:

1. Can capture API requests ✅
2. Can inspect data from request/response ✅
3. Works for requests made in page load phase ❌
4. Is actively maintained ❌
5. Works when running tests on cloud platforms ✅

WDIO BiDi protocol

Onto our next solution, let’s try to use something new. The WDIO BiDirectional (BiDi) protocol is an extension to the default protocol used to communicate with the browser. It adds some low-level functionalities that allow you to have more control over the test environment. Since it’s still under development, older browser versions might not support the protocol, so make sure to check before adding it.

First, you need to set webSocketUrl to true in your capabilities, like so:

// wdio.conf.ts

export const config: WebdriverIO.Config = {
  // Other configs
  capabilities: [
    // Other capabilities
    webSocketUrl: true
  ],
};

Now, we are ready to use all of BiDi’s functionalities. It doesn’t offer a direct way to assert if the requests were made like the intercept service, but it does allow you to capture requests in multiple ways. Let’s try with the networkAddIntercept. In this example, before even opening the page, we add the interceptor configured to stop the requests after they are completed. Then, we set a listener to execute every time a request is captured. After this initial setup, we finally open the page and perform some actions to complete the test.

it('should intercept API calls', async () => {
  await browser.networkAddIntercept({
    phases: ['responseCompleted'],
    urlPatterns: [{ type: 'string', pattern: 'my-api-url.com/path' }],
  });

  browser.on('network.responseCompleted', event => {
    console.log(event.request);
  });

  await browser.url('my-website.com');

  const button = $('#my-button');

  await button.scrollIntoView();
  await button.click();
  await browser.pause(1000);
});

And the result is… It works! We are able to capture all the requests, even those that are made during the page loading phase. It sounds too good to be true, right? That’s because we have other limitations. Due to security reasons, the BiDi protocol doesn’t expose the request and response bodies, it only returns their sizes as integers. So, if your intention is to validate the data sent or received by your API, unfortunately, this will not work. Even though you can use some other cool information that it returns, for example, the API call timings, which allows you to control the overall speed of your website.

We can also test some other BiDi functions, but the final output is always the same. If you want to try an example using the mock function to make the interceptor by hand, you can use this one (mind you, it has the same limitations listed above):

it('should intercept API calls', async () => {
  const mock = await browser.mock('**');

  mock.on('request', ({ request }) => {
    console.log(request);
  });

  await browser.url('my-sebsite.com');

  const button = $('#my-button');

  await button.scrollIntoView();
  await button.click();
  await browser.pause(1000);
});

And our final score for the WDIO BiDi protocol:

1. Can capture API requests ✅
2. Can inspect data from request/response ❌
3. Works for requests made in page load phase ✅
4. Is actively maintained ✅
5. Works when running tests on cloud platforms ✅

Puppeteer

We still have one more tool to test, so we can meet all of our criteria and be able to successfully validate our requests without any limitations. Right off the bat, Puppeteer only supports Chromium-based browsers, so if you need to test Mozilla or Safari you can already skip this section. But let’s give it a try anyway, what do we have to lose?

It’s important to mention that Puppeteer is the current replacement for the Devtools on WDIO, which is being slowly deprecated. Some functions are not yet implemented, but we can already take advantage of the event listeners. So let’s go on and install our dependencies:

npm install puppeteer-core

# Also install the types if you are using TypeScript
npm install @types/puppeteer -D

To start our tests, before opening the URL, we use the getPuppeteer function to get the Puppeteer instance and get our current working page. Then, we set a listener to the page requests, which will capture all API calls before they are sent, and we are free to proceed with the rest of the test.

it('should intercept API calls', async () => {
  const puppeteer = await browser.getPuppeteer();
  const pages = await puppeteer.pages();
  const page = pages[0];

  page.on('request', request => {
    console.log(request.response());
  });

  await browser.url('my-website.com');

  const button = $('#my-button');

  await button.scrollIntoView();
  await button.click();
  await browser.pause(1000);
});

Surprisingly, Puppeteer can get all the request and response data, it captures requests even on the page load phase, and you are even able to return mocked responses. It would be the perfect solution if it weren’t for two major limitations. One we already talked about, it’s only available on Chromium-based browsers, and the other one is that it doesn’t work on cloud web automation platforms (like LambdaTest, Sauce Labs, or BrowserStack Automate) since they don’t expose the Devtools to be used by WDIO. Working with Puppeteer is perfect if you are testing Chromium browsers locally, so if this meets your needs, I highly recommend it since it’s the most reliable tool.

And our final criteria match:

1. Can capture API requests ✅
2. Can inspect data from request/response ✅
3. Works for requests made in page load phase ✅
4. Is actively maintained ✅
5. Works when running tests on cloud platforms ❌

Conclusion

Validating API requests in end-to-end tests is not a one-size-fits-all problem. Each approach comes with its strengths and limitations, and the right choice depends heavily on your project’s requirements and environment.

If your goal is broad compatibility and cloud support, BiDi is currently your best option. If you need deep inspection of request and response data and you’re fine with local Chromium testing, Puppeteer is the way to go. I would not recommend the intercept service since it’s not maintained anymore, but if it’s the only option that fits your needs, go for it.

In the end, there’s no “right” solution, but by understanding the trade-offs of each tool, you can choose the setup that ensures your API requests are validated, your tests remain stable, and your deployments are more reliable.