Published test results
Edge test corpus: how the cutout edges score
Eight hard edges with a known right answer, cut by the same engine your browser runs, and scored against that answer. Scores were generated on 2026-11-01 from the engine as of commit 14ef8064, at 192 × 192 pixels. Lower is better; the failures stay in.
The scores
| Scene | Method | Mean error | Edge error | RMSE |
|---|---|---|---|---|
| Hair strands against a blue wall | Colour solve | 0.5% | 10.7% | 2.91 |
| Fur on a grey ground | Colour solve | 1.75% | 19.7% | 7.37 |
| A feathered vignette | Colour solve | 8.54% | 36.2% | 20.75 |
| Anti-aliased lettering on white | Colour key | 0.01% | 0.5% | 0.17 |
| Thin wire and a pendant | Colour solve | 0.14% | 9.5% | 1.33 |
| A soft contact shadow | Colour solve | 1.65% | 11.9% | 6.03 |
| A glass-like see-through shape | Colour solve | 10.51% | 48.7% | 23.66 |
| A hard product edge | Colour solve | 0.23% | 19.1% | 2.14 |
Each scene: input, right answer, our answer
Hair strands against a blue wall
Thirty one-pixel strands leaving a head; coverage is fractional along every strand.
Mean 0.5% · edge 10.7% · 1,534 edge pixels
Fur on a grey ground
A soft, noisy boundary several pixels deep, like an animal's coat.
Mean 1.75% · edge 19.7% · 2,637 edge pixels
A feathered vignette
An edge that fades over 24 pixels: the right answer is a long, even ramp.
Mean 8.54% · edge 36.2% · 8,696 edge pixels
Anti-aliased lettering on white
A logo-style mark on pure white, cut with the colour key and its unmixing.
Mean 0.01% · edge 0.5% · 616 edge pixels
Thin wire and a pendant
Lines one to two pixels wide crossing open background; easy to lose entirely.
Mean 0.14% · edge 9.5% · 523 edge pixels
A soft contact shadow
A dark, blurred shadow under a box. Whether it should stay is a choice; the reference keeps it at its real density.
Mean 1.65% · edge 11.9% · 5,109 edge pixels
A glass-like see-through shape
Half-transparent all the way across. Colour-based solves treat it as solid or as gone; it is a known failure.
Mean 10.51% · edge 48.7% · 7,925 edge pixels
A hard product edge
A crisp manufactured outline on near-white: the reference is binary apart from one pixel of anti-aliasing.
Mean 0.23% · edge 19.1% · 440 edge pixels
Questions about the corpus
What do the scores mean?
Mean error is the average difference between the engine's alpha and the reference alpha over the whole picture, as a percentage of full opacity: 0% would be a perfect match. Edge error is the same average taken only over the pixels the reference marks as part-way transparent, which is where cutouts actually go wrong. RMSE punishes large mistakes more than small ones.
Why are the scenes drawn rather than photographed?
Because a fair score needs the true answer, and only a drawn scene has one exactly: each picture is made by laying a foreground over a background through a known alpha, so the reference is not a guess or a hand-painted mask.
Why publish the bad results?
Because the glass and feathered scenes are real limits of a colour-based cutout, and hiding them would make the good scores worth less. When the engine changes, the corpus is run again and this page shows the new numbers with the commit they came from.
Were the optional AI models tested here?
No. These scores are for the colour solve and the colour key, which run with nothing downloaded. The optional models are fetched only when a visitor asks for one, so they are not part of this build-time test.
Related
- Remove background around hair and furA wider uncertain band and a softer ramp, set that way before you arrive.
- Fix the edges of a cutoutOpen a transparent PNG someone else made and take its white halo, colour fringe or jagged edge away.
- Hair, fur and every other soft edgeWhat the uncertain band is, what widening it costs, and the edges nobody wins.
