REST API service throttling using HAProxy

In this article, we will provide a step-by-step guide for configuring traffic throttling for a REST API service. For the purpose of this article, we will consider a simple organization called myapp.dev that has representational website providing API documentation, and an API server providing REST interface for performing calculations.

HAProxy configuration

Our organization will have a website available at http://myapp.dev/ and a REST API running under http://myapp.dev/v1/hello-world?api_key=$uuid. In our case, will be using HAProxy 2.9-dev for throttling and load balancing the organization’s website and API servers. Configuration will have “defaults” settings; “frontend myfrontend” setting, allowing clients to retrieve website or call API; “frontend stats” settings, allowing authorized clients to retrieve HAProxy stats; “backend website_servers” settings handling website; and “backend api_servers” settings handling REST API services.

Defaults configuration

Default settings specify that HTTP protocol will be used as the default mode, and some timeout values that limit requests duration. Timeouts are set to 5 minutes, which is significantly more than necessary, and should be changed to more reasonable limits for readers’ intended use.

defaults mode http timeout client 300s timeout connect 30s timeout server 300s timeout http-request 300s

Back-end configuration for website

Back-end settings specify to use HTTP protocol for health checks (option, http-check), and it also enables HAProxy to forward a domain name from the initial request to a server handling a request (http-send-name-header). Configuration supplies maximum concurrent connections count, interval for health checks (default is 2 seconds), and number of successful responses to raise a faulty server (default-server). Now comes the essential part — stick-tables, which are used for throttling requests. In case the api_key is not passed with a request to the server, requests will be throttled according to IP addresses using the stick-table specified in the website_servers backend group.

backend website_servers
    option httpchk
    timeout check 30s
    http-send-name-header Host
    http-check send meth GET  uri /health
    default-server  maxconn 2000  inter 1s  rise 2
    server node1.myapp.dev node1.myapp.dev:8888 check
    server node2.myapp.dev node2.myapp.dev:8888 check
    stick-table type ip size 100k expire 1s store http_req_rate(1s)

Back-end configuration for API service

Back-end configuration for api specifies the same configuration as for the website, except for stick tables, which uses string type key instead of IP one. In case a request contains a specific path (“/v1/”), it will be considered an API request. If string api_key is supplied, HAProxy will use an api_key stick-table provided in the api_servers backend instead of one used for IPs. Both website and API service requests are stored in stick-tables for 1 second.

backend api_servers
    errorfile 404 /etc/haproxy/errorfiles/404.http
    errorfile 429 /etc/haproxy/errorfiles/429.http
    errorfile 502 /etc/haproxy/errorfiles/502.http
    errorfile 503 /etc/haproxy/errorfiles/503.http
    errorfile 504 /etc/haproxy/errorfiles/504.http
    option httpchk
    timeout check 30s
    http-send-name-header Host
    http-check send meth GET  uri /health
    default-server  maxconn 2000  inter 1s  rise 2
    server node1.myapp.dev node1.myapp.dev:8888 check
    server node2.myapp.dev node2.myapp.dev:8888 check
    stick-table type string size 100k expire 1s store http_req_rate(1s)

Front-end configuration for both website and API

Front-end myfrontend settings specify to bind to port 80 for http requests. REST API provided by myorg.dev might use both HTTP GET requests with the api_key parameter, and HTTP POST requests with a JSON payload having an api_key at the root. In order to process JSON payloads, HAProxy must buffer requests (http-buffer-request). An example of such HTTP POST request might look like the following:

{
  "function": "hello-world",
  "parameters": {"foo": "bar"},
  "api_key": "$uuid"
}

Front-end configuration has error file defined to specify what response will be given by HAProxy. In case a rate limit was reached, a 429 response will be sent to the client. Error file contain a list of headers and a body content that is sent to the client. For example, a 429.http file might look like the following:

HTTP/1.1 429 Too Many Requests
Cache-Control: no-cache
Connection: close
Content-Type: application/json

{"code":"Error","message":"Too Many Requests"}

Front-end configuration has the following boolean variables: “api_uri”, “has_param”, “has_json”, “exceeds_limit_param”, “exceeds_limit_json”, “exceeds_limit_ip“. Variable “api_uri” checks if there is a “/v1/” in a request URL, meaning a request was most likely sent for the REST API. Variable “has_param” checks if there is a GET “api_key” parameter in a request. Variable “has_json” checks if there is a JSON “api_key” property in a POST request. Variable “exceeds_limit_param” checks if there are more than 100 requests during the last second having a specific api_key. Variable “exceeds_limit_json” checks if there are more than the specified number of requests during the last second for an api_key taken from a JSON payload. Variable exceeds_limit_ip checks if there are more than the specified number of requests during the last second for the client IP.

After setting the variables, HAProxy tracks the requests in stick tables if the limit has not been reached for the corresponding method (GET api_key, POST json api_key, or client IP). If one or more limits were reached, for example, both an api_key and an IP limit for the API service, HAProxy will respond with a 429 error code and response. And finally, depending on the presence of “/v1/” in the path, either api_servers or website_servers backend group will be used to render a response.

frontend myfrontend
    bind *:80
    option http-buffer-request
    errorfile 429 /etc/haproxy/errorfiles/429.http
    acl api_uri path_beg /v1/
    acl has_param url_param(api_key) -m found
    acl has_json req.body,json_query('$.api_key') -m found
    acl exceeds_limit_param url_param(api_key),table_http_req_rate(api_servers) gt 100
    acl exceeds_limit_json req.body,json_query('$.api_key'),table_http_req_rate(api_servers) gt 100
    acl exceeds_limit_ip src,table_http_req_rate(website_servers) gt 100
    http-request track-sc0 url_param(api_key) table api_servers if !exceeds_limit_param has_param
    http-request track-sc0 req.body,json_query('$.api_key') table api_servers if !exceeds_limit_json has_json
    http-request track-sc0 src table website_servers if !exceeds_limit_ip !has_param !has_json
    http-request deny deny_status 429 if exceeds_limit_param or exceeds_limit_json or exceeds_limit_ip
    use_backend api_servers if api_uri
    default_backend website_servers

Front-end configuration for statistics

HAProxy statistics will be available at http://myapp.dev:8080/stats . It is recommended to limit the availability of this page using authorization or client IP address. In case of authorization, the reader might check the "stats auth” directive. In case of IP whitelisting, the reader might consider "tcp-request connection reject” configuration. Satistics front-end configuration is provided for completeness, but will not be discussed here in detail.

frontend stats
    bind *:8080
    stats enable
    stats uri /stats
    stats refresh 10s

Conclusions

There are some nuances in this approach, when the user passes api key to the website server; or the REST API service does not receive an api_key; or rate limit is reached for website request, and HAProxy responses with a 429 JSON given in 429.http file. But we will leave this as an exercise to the reader.

Literature

https://www.haproxy.com/documentation/haproxy-configuration-tutorials/proxying-essentials/custom-rules/stick-tables/

https://www.haproxy.com/documentation/haproxy-configuration-tutorials/proxying-essentials/custom-rules/acls/