Suite blackbench-0.2 Rules 6D · S17 · DAS · LS · 3:2 Seed blackbench-v1 Hands per model 100,000

Leaderboard · blackbench-0.2

Money won at the blackjack table

Blackbench measures how much money a decision model wins playing blackjack. Every model plays the same 100,000 seeded hands, flat 1-unit bets, fixed house rules.

Suite
blackbench-0.2
Rules
6D · S17 · DAS · LS · 3:2
Hands / model
100,000
Bet
Flat 1 unit
Seed
blackbench-v1
Updated
2026-11-21
Leaderboard. Ranked by money won: net units over 100,000 hands at flat 1-unit bets.
Ranking Diagnostics · not ranked
# Model EV bar with 95% interval
Loading results…
EV per hand, lost to the house; same scale for every row 95% interval Off scale (lost over 3× the median model) Reference strategy, unranked Invalid rate (bar full at 5%)

How it’s scored

Ranking

Money won

The model gets the table state and a typed question listing the legal actions, and returns a probability for each. We play the action it chooses. The ranking is the total won or lost over 100,000 hands at a flat 1-unit bet.

Precision

EV and its interval

EV is money won per hand as a percentage of the bet. At 100,000 hands the 95% interval is about ±0.7 points. Models whose intervals overlap are not reliably different, whatever their rank.

Diagnostics

Agreement and invalid answers

Agreement with basic strategy and the invalid-response rate are shown to explain results. They don’t affect the ranking. Invalid answers play as stand.