EchoBird Token Quota & Traffic Rate-Limiting Router

EchoBird Token Quota & Traffic Rate-Limiting Router

@edison7009/echobird·v1.2.5·core

A token budget and traffic rate-limiting middleware for DeepSeek Harness fleets. Implements sliding-window rate limiting, multi-tenant token quotas, and smooth queue backpressure.

Rate-Limiter · Token-Quota · Budget-Guard · Traffic-Shaper

GitHubNPM
Installation & Integration
bash
manager:
$ pnpm add @edison7009/echobird

Architecture

EchoBird Quota Router protects autonomous agent infrastructure from runaway token costs and rate limit exhaustion. In multi-tenant enterprise environments, a single rogue agent looping uncontrollably can burn thousands of dollars and starve critical services. This plugin implements sliding-window rate limiting and distributed token buckets directly inside Cordis. Platform engineers enforce granular Requests Per Minute (RPM) and Tokens Per Day limits scoped by team, user, or project ID. Requests gracefully queue during bursts and safely halt before exceeding strict organizational budget ceilings.

Core Features

01

Sliding-Window Precision Rate Limiting: Prevents edge-burst exploits typical of fixed windows

02

Multi-Tenant Quota Hierarchy: Inherits limits across Organization, Department, and User trees

03

Intelligent Burst Queue Smoothing: Buffers temporary traffic peaks, avoiding sudden HTTP 429 errors

04

Real-Time Cost Dashboards & Alerts: Configurable 80% soft alerts and 100% hard budget circuit breakers

Core Workflow

01

Ingress Metering & Auth

Inspects incoming inference calls, extracting tenant ID and estimating token usage.

02

Token Bucket Evaluation

Checks distributed Redis token buckets to evaluate remaining rate limits.

03

Queue Buffer & Backpressure

Buffers transient burst traffic in priority queues, smoothing out invocation spikes.

04

Quota Trip & Alerting

Blocks calls cleanly when budgets deplete, triggering enterprise webhook alerts.

Configuration Parameters Reference (YAML / JSON)

ParameterTypeDefaultDescription
defaultDailyBudgetUsdnumber50.0Default daily spend cap per tenant in USD
maxRPMnumber60Default Requests Per Minute (RPM) ceiling
redisEndpointstringredis://127.0.0.1:6379Redis connection URL for distributed state