Benchmarks: BanditDB vs Open Bandit Pipeline
Off-policy evaluation, replay learning curves and selection entropy on two public recommendation datasets: Yahoo! R6A and MovieLens-100K.
We compared four BanditDB policies (LinUCB, Thompson Sampling, NeuralLinUCB, NeuralTS) with four policies from Open Bandit Pipeline (OBP), the open-source bandit research library from ZOZO Research: LinUCB, LinTS, a context-free BernoulliTS, and Random. Each policy is scored against the logging policy that collected the data.
Key findings
- Matches the reference library. BanditDB's linear policies land in the same range as OBP's on both datasets. The two leaders on each dataset sit inside each other's 95% confidence intervals.
- Yahoo! R6A: BanditDB-LinUCB lifts estimated click-through rate from 3.50% to 5.65% (+61.5%, DRos), and Thompson Sampling reaches +50.5%. OBP-LinUCB has the highest estimate (+66.7%). In sequential replay, BanditDB-LinUCB and BanditDB-Thompson collect the most clicks, and OBP-LinUCB falls below the random baseline.
- MovieLens-100K: OBP-LinTS has the highest estimate (+26.1%). BanditDB-Thompson is next at +22.4%, with an estimated 21.5% 5-star rate against a 21.3% on-policy ground truth. All four BanditDB policies are above OBP-LinUCB (+15.0%).
- Context matters. The context-free baselines (OBP-BernoulliTS, OBP-Random) barely move off the logging policy: +0.4% on Yahoo! and +4.0% on MovieLens.
- Exploration stays healthy. No BanditDB policy drops below the 0.40 entropy warning threshold used by BanditDB diagnostics.
Read the intervals, not just the bars. The replay streams are short: about 650 matched events on Yahoo! and about 500 on MovieLens. Confidence intervals are wide. Treat the top contextual policies on each dataset as statistically tied. The clear gap is between contextual and context-free policies.
Yahoo! Front Page Today Module (R6A)
News-article recommendation on the Yahoo! front page. K = 20 candidate articles, d = 6 user features, reward = click. The logging policy picked articles uniformly at random, so importance weights are well behaved and all four OPE estimators agree closely. The logging policy's click-through rate is 3.50%.
Estimated policy value and lift over the logging policy
| Policy | Est. CTR (DRos) | Lift (DRos) | Lift (DR) |
|---|---|---|---|
| BanditDB-LinUCB | 5.65% | +61.5% | +76.9% |
| BanditDB-ThompsonSampling | 5.27% | +50.5% | +64.5% |
| BanditDB-NeuralLinUCB | 4.08% | +16.5% | ≈ +19% |
| BanditDB-NeuralTS | 3.89% | +11.3% | ≈ +14% |
| OBP-LinUCB | 5.83% | +66.7% | ≈ +83% |
| OBP-LinTS | ≈ 3.77% | ≈ +8% | ≈ +9% |
| OBP-BernoulliTS (context-free) | 3.51% | +0.4% | ≈ 0% |
| OBP-Random | 3.50% | ≈ 0% | ≈ 0% |
Sequential replay (3 seeds, about 650 matched events)
| Policy | Moving-avg CTR at end | Cumulative matched clicks | Normalized entropy |
|---|---|---|---|
| BanditDB-ThompsonSampling | ≈ 5.1% | ≈ 34 | ≈ 0.72 |
| BanditDB-LinUCB | ≈ 4.6% | ≈ 31 | ≈ 0.92 |
| BanditDB-NeuralLinUCB | ≈ 4.0% | ≈ 25 | ≈ 0.76 |
| BanditDB-NeuralTS | ≈ 3.5% | ≈ 21 | ≈ 0.83 |
| OBP-LinUCB | ≈ 2.2% | ≈ 15 | ≈ 0.95 |
| OBP-Random | ≈ 3.5% | ≈ 24 | 1.00 |
What each panel shows
- A: Estimated policy value. Expected CTR under IPW, SNIPW, DR and DRos with 95% bootstrap intervals. The contextual learners sit well above the 3.50% baseline. The context-free policies stay on it.
- B: Relative lift. Improvement over the logging policy. BanditDB-LinUCB and BanditDB-Thompson are in the same band as OBP-LinUCB under every estimator.
- C: Replay moving-average CTR. BanditDB-Thompson climbs above baseline within about 50 matched steps and leads for most of the stream. Shaded bands are ±1 standard deviation across seeds.
- D: Cumulative matched clicks. BanditDB-Thompson and BanditDB-LinUCB pull ahead of random (about 24 clicks) after about 250 steps.
- E: Estimator error. Bootstrap error against the on-policy value. IPW has the heaviest tails, DRos the tightest spread, and the Direct Method (DM) a systematic negative bias. This is why the tables lead with DRos.
- F: Selection entropy. Normalized entropy H(π)/log K over the stream. All BanditDB policies settle between about 0.72 and 0.92, well above the 0.40 warning and 0.20 collapse lines.
GroupLens MovieLens-100K
Movie recommendation framed as a bandit problem. K = 25 candidate movies across five macro-genres, d = 10 user features, reward = the user rated the movie 5 stars. The dataset's base 5-star rate is 17.03%. The logging policy's value is 17.58%.
Estimated policy value and lift over the logging policy
| Policy | Est. 5-star rate (DRos) | Lift (DRos) | Lift (DR) |
|---|---|---|---|
| BanditDB-ThompsonSampling | 21.51% | +22.4% | +26.9% |
| BanditDB-LinUCB | 21.23% | +20.8% | ≈ +28% |
| BanditDB-NeuralTS | 21.06% | +19.8% | ≈ +24% |
| BanditDB-NeuralLinUCB | ≈ 20.3% | ≈ +15.5% | ≈ +19% |
| OBP-LinTS | 22.17% | +26.1% | ≈ +33% |
| OBP-LinUCB | 20.21% | +15.0% | ≈ +18% |
| OBP-BernoulliTS (context-free) | 18.28% | +4.0% | ≈ +4% |
| OBP-Random | ≈ 17.6% | ≈ 0% | ≈ 0% |
On-policy ground truth for BanditDB-ThompsonSampling is 21.33%. The DRos estimate of 21.51% is within 0.2 percentage points.
Sequential replay (3 seeds, about 500 matched events)
| Policy | Moving-avg 5-star rate at end | Cumulative matched 5-star ratings | Normalized entropy |
|---|---|---|---|
| BanditDB-NeuralTS | ≈ 21.0% | ≈ 102 | ≈ 0.54 |
| BanditDB-ThompsonSampling | ≈ 20.7% | ≈ 104 | ≈ 0.60 |
| BanditDB-NeuralLinUCB | ≈ 19.8% | ≈ 94 | ≈ 0.44 |
| BanditDB-LinUCB | ≈ 19.5% | ≈ 96 | ≈ 0.41 |
| OBP-LinTS | ≈ 21.8% | ≈ 110 | ≈ 0.59 |
| OBP-Random | ≈ 17.6% | ≈ 93 | 1.00 |
What each panel shows
- A: Estimated policy value. Probability of recommending a movie the user rates 5 stars. All contextual policies clear the 17.58% baseline. Context-free popularity reaches only 18.28%.
- B: Relative lift. OBP-LinTS leads. BanditDB-Thompson, BanditDB-LinUCB and BanditDB-NeuralTS follow at +20–22% (DRos), all above OBP-LinUCB.
- C: Replay moving-average 5-star rate. Contextual policies converge to 19.5–22% after an exploration phase of about 150 matched steps. BanditDB-NeuralTS keeps improving through the stream and ends level with the linear policies.
- D: Cumulative matched 5-star ratings. Contextual policies separate from random steadily rather than in bursts. Exploration does not stall as the catalogue is traversed again.
- E: Estimator error. DR and DRos are centred near zero. Unregularized IPW is visibly skewed, and DM is biased low.
- F: Selection entropy. Policies stabilise between about 0.41 and 0.60. That reflects genre specialization rather than random guessing (1.0) or collapse (below 0.20).
Methodology
| Component | Setup |
|---|---|
| Policies | BanditDB: LinUCB, ThompsonSampling, NeuralLinUCB, NeuralTS. OBP: LinUCB, LinTS, BernoulliTS (context-free), Random. |
| OPE estimators | Inverse Propensity Weighting (IPW), Self-Normalized IPW (SNIPW), Doubly Robust (DR), DR with optimistic shrinkage (DRos). Panel E also shows the Direct Method (DM) and SwitchDR. |
| Uncertainty | 95% bootstrap confidence intervals on policy value. Panel E pools bootstrap resamples against the on-policy value Von. |
| Replay | Prequential replay (Li et al., 2011). An event counts only when the policy's chosen arm matches the logged arm (at = ât). Averaged over 3 random seeds with ±1 standard deviation bands. |
| Entropy | Normalized Shannon entropy H(π)/log K of arm selections. 0.40 and 0.20 are the warning and collapse thresholds in BanditDB's diagnostics endpoint. |
To run the same estimators on your own campaigns, export them to Parquet and use banditdb.eval. See the Data Science Escape Hatch. For guidance on picking LinUCB, Thompson Sampling or a neural variant, see Choosing an Algorithm.
FAQ
How does BanditDB compare to Open Bandit Pipeline?
On both public datasets BanditDB's contextual policies land in the same range as the Open Bandit Pipeline (OBP) reference implementations. On Yahoo! R6A, BanditDB-LinUCB reaches a +61.5% click-through lift over the logging policy (DRos estimator) against +66.7% for OBP-LinUCB, with overlapping confidence intervals; in sequential replay BanditDB-LinUCB and BanditDB-ThompsonSampling collect the most clicks. On MovieLens-100K, OBP-LinTS has the highest estimate (+26.1%), BanditDB-ThompsonSampling is next (+22.4%), and all four BanditDB policies are above OBP-LinUCB (+15.0%).
Which datasets were used?
Yahoo! Front Page Today Module R6A (20 candidate articles, 6 user features, uniformly random logging policy, 3.50% baseline click-through rate) and GroupLens MovieLens-100K (25 candidate movies, 10 user features, reward = a 5-star rating, 17.58% logging-policy baseline).
Which BanditDB algorithm performed best?
LinUCB on Yahoo! R6A (+61.5% DRos lift) and Thompson Sampling on MovieLens-100K (+22.4% DRos lift, estimated 21.5% 5-star rate against a 21.3% on-policy ground truth). The neural variants (NeuralLinUCB, NeuralTS) gave smaller but positive lifts on these low-dimensional feature sets; they are built for high-dimensional contexts such as LLM embeddings.
How were policies evaluated without live traffic?
With off-policy evaluation (OPE) on logged data. Policy value is estimated with IPW, SNIPW, Doubly Robust (DR) and DR with optimistic shrinkage (DRos), each with 95% bootstrap confidence intervals. A separate prequential replay counts only the events where the policy picks the same arm as the logged one, averaged over 3 random seeds.
Do BanditDB policies collapse onto a single arm?
No. Normalized selection entropy stays above BanditDB's 0.40 warning threshold and far above the 0.20 collapse alert for every BanditDB policy on both datasets. On MovieLens-100K, BanditDB-LinUCB settles close to the warning line (about 0.41), which reflects strong genre specialization.
Datasets
- Yahoo! Front Page Today Module User Click Log (R6A), Yahoo! Webscope.
- MovieLens 100K, GroupLens Research.
- Open Bandit Pipeline, Saito et al., ZOZO Research.