Use Cases
Any decision that has a context, a finite set of choices, and a measurable outcome is a candidate for BanditDB.
| Domain | Arms (choices) | Context | Reward signal |
|---|---|---|---|
| LLM routing | GPT-4o, Claude Sonnet, Gemini Flash, … | Task type, input length, session turn, user expertise | LLM-as-judge score, user thumbs up, task completion |
| Prompt strategy | Zero-shot, chain-of-thought, few-shot, structured output | Task complexity, domain, input length, session context | LLM-as-judge quality score (0–1) |
| Agent tool selection | Which tool or sub-agent to invoke for a given step | Task type, prior tool results, cost budget | Task success, latency, cost |
| Dynamic pricing | Price tiers or discount levels | Inventory, seasonality, competitor pricing, customer segment | Revenue per unit or sell-through rate |
| Checkout optimisation | Upsell offer, free shipping, no offer | Cart value, customer history, device type | Conversion (binary) or order value lift |
| Content personalisation | Article, offer, layout variant | User demographics, history, session signals | Click, time-on-page, downstream conversion |
| Legal intake routing | Consult, intake form, refer, decline | Case value, matter complexity, conflict risk, capacity | Matter opened, revenue collected |
| Adaptive clinical trials | Treatment arms | Patient demographics, comorbidities, baseline score | Outcome score normalised to [0, 1] |
| Sleep / wellness | Temperature, light, noise reduction | Sex, age, weight, activity level, bedtime | PSQI score improvement ratio |
The reward must be a scalar in [0, 1]. If your natural metric has a different range, divide by its maximum (e.g. revenue / max_possible_revenue) or use a ratio like (after - before) / before, clipped to [0, 1].
Example: LLM Routing
The first row of the table, end to end — route each request to the cheapest model that can handle it:
from banditdb import Client
db = Client("http://localhost:8080", api_key="your-secret-key")
db.create_campaign(
"llm-routing",
arms=["haiku", "sonnet", "opus"],
feature_dim=4,
metadata={"features": ["task_complexity", "input_len_norm", "session_turn_norm", "code_task"]},
)
# Incoming request: medium complexity, short input, first turn, not a code task
context = [0.5, 0.12, 0.0, 0.0]
arm, interaction_id = db.predict("llm-routing", context)
response = call_llm(model=arm, prompt=prompt)
# Reward: LLM-as-judge quality score, already in [0, 1]
quality = judge(prompt, response) # e.g. 0.85
db.reward(interaction_id, quality)To fold cost into the signal, use a blended reward such as quality - 0.2 * cost_norm, clipped to [0, 1]. The bandit then learns the cheapest model that maintains quality per context — not just the best model overall.
When BanditDB is not the right tool
- Pure exploration / discovery — if you have no feedback signal yet and are building a dataset from scratch, start with random assignment and switch to BanditDB once you have ~100 outcomes per arm.
- Very high-dimensional action spaces (thousands of arms) — LinUCB scales with arms × feature_dim² in memory. For catalogue-scale recommendation, consider embedding-based retrieval first and use BanditDB for the final re-ranking stage.
- Non-stationary rewards with hard concept drift — LinUCB assumes rewards are stationary. Gradual drift is handled by setting
decay_half_life_hours(below). Only a genuinely discontinuous break — a repriced catalogue, a replaced product — is worth deleting and recreating the campaign for, and that discards everything learned, so reach for decay first.
Adapting to drift: decay_half_life_hours
Without decay a campaign weighs a click from six months ago exactly as heavily as one from this morning. The longer it runs, the more inertia it accumulates, and the slower it reacts to a shift in what actually works.
Set a half-life at creation and older evidence loses influence geometrically — after one half-life it counts half as much, after two a quarter:
db.create_campaign(
"homepage-hero",
arms=["a", "b", "c"],
feature_dim=8,
decay_half_life_hours=168, # one week
)Decay is applied at checkpoint, to every arm including the tournament challenger. Pick the half-life from how fast your environment actually moves: days for fashion or news, weeks-to-months for a stable product surface. Too short and the model forgets faster than it learns, widening confidence intervals and pushing it back toward exploring. Leave it unset for a stationary problem — it costs accuracy you do not need to spend.