Model Architecture

Why BanditDB uses the Neural-Linear handoff instead of end-to-end Deep RL.

Why Neural-Linear?

In the literature, this approach is often called the "Golden Middle" of Contextual Bandits. End-to-end Deep Reinforcement Learning (like PPO or Soft Actor-Critic) is notoriously unstable and requires thousands of interactions before it makes even basic sense of its environment. BanditDB is designed for production databases where the first 50 decisions are just as important as the 5,000th.

1. Latency vs. Accuracy

A full forward pass of a deep network for every arm is slow (10ms+). By using a Neural Feature Extractor with a Linear Head, BanditDB achieves sub-millisecond latencies. The heavy neural retraining happens in the background, while the live prediction path is a simple, ultra-fast vector dot product (θᵀh).

2. Statistical Rigor (The Closed-Form Variance)

Deep networks are "black boxes" that don't natively understand their own uncertainty. To explore efficiently, they often rely on heuristics like epsilon-greedy or dropout. Because BanditDB's final layer is linear, it can compute an exact closed-form variance for every prediction. This allows us to use UCB and Thompson Sampling with mathematical certainty, leading to lower cumulative regret.

3. The Internal Tournament

The Progressive algorithm is our unique technical moat. It runs a continuous internal tournament between a Linear model (safe, sample-efficient) and a Neural model (powerful, non-linear).

• Shadow Learning: Every reward received by the database updates both models simultaneously.
• Autonomous Handoff: At each evaluation the engine scores both models with SNIPS — self-normalised inverse-propensity scoring, an off-policy estimator that answers "what would this model have earned on the traffic we actually served?" without giving it any live traffic to find out. Prediction-time propensities make that counterfactual legitimate rather than a guess.
• Evidence before movement: The challenger has to win required_wins evaluations in a row (default 3) before traffic shifts, and traffic moves in step_bps increments (default 10 percentage points) rather than all at once. A single lucky evaluation cannot promote a bad model.
• Coverage guard: Evaluation is skipped entirely until every arm has at least min_obs observations (default 100) and the estimator has enough overlap to be meaningful. An importance-weighted estimate built on a handful of samples is noise with a decimal point, and acting on it is worse than not evaluating at all.
• Self-Correction: A losing streak reverses the ramp by the same increments, and the winner's streak counter resets whenever the result flips. Traffic is clamped to a 10%–90% band in both directions, so the losing model always retains enough traffic to keep being evaluated — and to win its place back if it was only temporarily behind.

Example: Progressive in Practice

from banditdb import Client

db = Client("http://localhost:8080", api_key="your-secret-key")

# One campaign, two models. BanditDB runs the tournament internally.
db.create_campaign(
    "support-routing",
    arms=["agent_fast", "agent_thorough", "escalate_human"],
    algorithm={"progressive": {"base": "linucb", "challenger": "neural_lin_ucb"}},
    feature_dim=64,
)

# The caller's code never changes — the handoff is invisible.
arm, interaction_id = db.predict("support-routing", context)
db.reward(interaction_id, 1.0)

Watch the tournament state via diagnostics — challenger_traffic_bps shows how much traffic the Neural challenger currently receives (1000 = 10% floor, 9000 = 90% ceiling):

curl -s http://localhost:8080/campaign/support-routing/diagnostics \
  -H "X-Api-Key: your-secret-key" | jq '{challenger_traffic_bps, tournament_wins}'