🌟 Enterprise Feature 🌟 This feature is bundled with GraphQL-Enterprise.

Runtime Limiter

GraphQL::Enterprise::RuntimeLimiter applies an upper bound to processing time consumed by a single client. It uses Redis track time with a token bucket algorithm.

Why?

This limiter prevents a single client from consuming too much processing time, regardless of whether it comes a burst of short-lived queries (which the Active Operation Limiter can prevent) or a small number of long-running queries. Unlike request counters or complexity calculations, the runtime limiter pays no attention to the structure of the incoming request. Instead, it simply measures the time spent on the request as a whole and halts queries when a client consumes more than the limit.

Setup

To use this limiter, update the schema configuration and include context[:limiter_key] in your queries.

Schema Setup

To setup the schema, add use GraphQL::Enterprise::RuntimeLimiter with a default limit_ms: value:

class MySchema < GraphQL::Schema
  # ...
  use GraphQL::Enterprise::RuntimeLimiter,
    redis: Redis.new(...),
    limit_ms: 90 * 1000 # 90 seconds per minute
end

limit_ms: false may also be given, which defaults to no limit for this limiter.

It also accepts a window_ms: option, which is the duration over which limit_ms: is added to a client’s bucket. It defaults to 60_000 (one minute).

Before requests will actually be halted, “soft mode” must be disabled as described below.

Query Setup

In order to limit clients, the limiter needs a client identifier for each GraphQL operation. By default, it checks context[:limiter_key] to find it:

context = {
  viewer: current_user,
  # for example:
  limiter_key: logged_in? ? "user:#{current_user.id}" : "anon-ip:#{request.remote_ip}",
  # ...
}

result = MySchema.execute(query_str, context: context)

Operations with the same context[:limiter_key] will rate limited in the same buckets. A limiter key is required; if a query is run without one, the limiter will raise an error.

To provide a client identifier another way, see Customization.

Soft Limits

By default, the limiter doesn’t actually halt queries; instead, it starts out in “soft mode”. In this mode:

This mode is for assessing the impact of the limiter before it’s applied to production traffic. Additionally, if you release the limiter but find that it’s affecting production traffic adversely, you can re-enable “soft mode” to stop blocking traffic.

To disable “soft mode” and start limiting, use the Dashboard or customize the limiter. You can also disable “soft mode” in Ruby:

# Turn "soft mode" off for the RuntimeLimiter
MySchema.enterprise_runtime_limiter.set_soft_limit(false)

Dashboard

Once installed, your GraphQL-Pro dashboard will include a simple metrics view:

GraphQL Runtime Limiter Dashboard

See Instrumentation below for more details on limiter metrics.

Also, the dashboard includes a link to enable or disable “soft mode”:

GraphQL Rate Limiter Soft Mode Button

When “soft mode” is enabled, limited requests are not actually halted (although they are counted). When “soft mode” is disabled, any over-limit requests are halted.

Customization

GraphQL::Enterprise::RuntimeLimiter provides several hooks for customizing its behavior. To use these, make a subclass of the limiter and override methods as described:

# app/graphql/limiters/runtime.rb
class Limiters::Runtime < GraphQL::Enterprise::RuntimeLimiter
  # override methods here
end

The hooks are:

Instrumentation

While the limiter is installed, it adds some information to the query context about its operation. It can be acccessed at context[:runtime_limiter]:

result = MySchema.execute(...)

pp result.context[:runtime_limiter]
# {:key=>"custom-key-9",
#  :limit_ms=>800,
#  :remaining_ms=>0,
#  :soft=>true,
#  :limited=>true}

It returns a Hash containing:

You could use this to add detailed metrics to your application monitoring system, for example:

MyMetrics.increment("graphql.runtime_limiter", tags: result.context[:runtime_limiter])

Some Caveats

The limiter will not interrupt a long-running field. Instead, it stops executing new fields after a client exceeds its allowed processing time. This is because interrupting arbitrary code may have unintended consequences for I/O operations, see “Timeout: Ruby’s most dangerous API”.

Also, the limiter only checks remaining time at the start of a query and it only decreases the remaining time at the end of a query. This means that simulaneous queries may consume the remainder at the same time. Use the Active Operation Limiter to limit behavior in this regard. This implementation is basically a trade-off: more granular updates would require more communication with Redis which would add overhead to each request.