localis
Waitlist

Describe a service in plain words, or type a CPT/HCPCS code.

4 min read

Rate limiting an API you bill by the code, not the request

Our /v1 API has three shapes: a single-code rate lookup, a bulk lookup (up to a few hundred codes in one call), and a claim-check endpoint that prices a whole line-item list against MPPR/NCCI/MUE edits at once. A rate limit that counts requests treats those three identically — which means it doesn't actually limit anything. A caller hitting the daily cap on single lookups can keep going indefinitely by switching to bulk calls of 300 codes each, doing 300× the work per unit of "request" the limiter sees.

Cost is the unit, not the request

ThrottleApiByCost computes a unit cost per request instead of assuming 1:

private function cost(Request $request): int
{
    $codes = $request->input('codes');
    if (is_array($codes)) {
        return min(BulkRateRequest::MAX_CODES, max(1, count($codes)));
    }

    $lines = $request->input('lines');
    if (is_array($lines)) {
        return min(ClaimCheckRequest::MAX_LINES, max(1, count($lines)));
    }

    $zips = $request->input('zips');
    if (is_array($zips)) {
        return min(BulkZipLookupRequest::MAX_ZIPS, max(1, count($zips)));
    }

    return 1;
}

A single lookup costs 1 unit. A bulk request costs one unit per code, ZIP, or claim line, clamped to that endpoint's own batch-size maximum so a malformed payload can't inflate the cost past what the request handler would accept anyway. A 300-code bulk call and 300 single-code calls now cost the same against the quota, which is the only way "bulk" can be a convenience feature instead of a rate-limit bypass.

Two budgets, checked before either is charged

Every request draws against a per-minute burst budget and a per-account daily quota simultaneously:

foreach ([[$minuteKey, $perMinute, 'per-minute'], [$dayKey, $perDay, 'daily']] as [$key, $max, $window]) {
    if (RateLimiter::attempts($key) + $cost > $max) {
        return $this->rejected($key, $max, $window);
    }
}

RateLimiter::increment($minuteKey, self::MINUTE_WINDOW, $cost);
RateLimiter::increment($dayKey, self::DAY_WINDOW, $cost);

Both checks run before either counter is incremented. That ordering matters more than it looks: if the code charged the per-minute budget first and then found the daily quota exhausted, a caller sitting right at their daily limit would still burn minute-budget on every rejected request for the rest of the day — a request that fails should cost nothing, on either axis it was checked against.

Quota follows the account, not the token

The API sits behind Sanctum tokens issued to a Team, and a Team can issue more than one token — separate keys for a staging environment and a production one, say. The quota is keyed to the Team, not the token, on purpose:

private function resolveCaller(Request $request): string
{
    $account = $request->user();

    if ($account !== null) {
        return 'acct:'.((string) $account->getKey());
    }

    return 'ip:'.($request->ip() ?? 'unknown');
}

If the limiter keyed on token instead, issuing a second token would double a Team's effective daily quota for free, and there'd be no principled way to say a shared limit is being enforced at all. Keying on the billing entity means the number of tokens in play is irrelevant to how much the account can do — which is also the property that lets a future paid tier plug into dailyLimit() by resolving the caller's plan, without touching how tokens are issued or how many a Team can hold.

Fail open, not 500

The rate-limit store is the same Redis instance everything else in the app leans on, and this middleware runs on every single /v1 request. A few seconds of Redis being unreachable — a restart, a failover — used to mean every API call failed, which is a worse outcome than a few unthrottled requests during that window:

} catch (Throwable $e) {
    Log::warning('API rate limiter unavailable; allowing request unthrottled.', [
        'caller' => $caller,
        'exception' => $e->getMessage(),
    ]);

    return $next($request);
}

The failure mode this avoids isn't hypothetical for this app specifically — PublicCache shares the same Redis instance and hit the identical class of outage, so the fix here is deliberately the same shape: catch broadly, log, degrade instead of propagate. An API that's occasionally under-throttled for a few seconds is a rounding error. An API that 500s its entire customer base because the rate limiter's backing store had a bad moment is an outage with a rate limiter's name on it.

The pattern

A rate limit is a promise about the total cost a caller can impose, and "cost" has to mean the same thing the limiter measures it as. Counting requests when the real cost varies by 300× per request isn't a weaker version of the same limit — it's measuring the wrong thing. The other two decisions here follow the same instinct: charge nothing for a rejected request, and key the budget to whoever's actually being billed, not to whatever credential happened to carry the request.

← All posts