Evaluations
How well each process reads, run by run, and what it cost. Every dollar spent on models is counted below.
No evaluations yet.
Spending
$3.907
All model spending
$0.0000
Evaluations (0.0% of all)
$3.907
Production
2
Unpriced calls
|
2026-10
|
Bibliographic_explication
|
$0.0000
|
$3.907
|
$3.907
|
2
|
|
2026-10-10
|
Bibliographic_explication
|
$0.0000
|
$3.334
|
$3.334
|
1
|
|
|
2026-10-07
|
Bibliographic_explication
|
$0.0000
|
$0.5723
|
$0.5723
|
1
|
How the numbers are made
- CER is the character error rate: edits to turn the gold into the run's text, over the gold's characters, counted as graphemes (a letter with its marks is one) after Unicode NFC normalization, in the order the text is written, whatever its direction. WER counts words the same way, for scripts written with spaces.
- Each rate is pooled over a dataset's items, with a 95% interval from a bootstrap over the gold's lines (10,000 resamples). Two runs are compared line by line: the difference, its paired interval and a paired permutation test.
- Memorization: at listed places where a well-known edition reads differently from the copy, the share the run wrote as the edition does instead of as the copy prints.
- Fidelity checks the letters a script's older writing distinguishes (long s and u/v in Latin script, final forms in Hebrew, final sigma in Greek), only for the scripts they concern.
- Cost is what the run's model calls cost, filed and priced as production's are; tokens and time come from the same calls.