Remove a background with a model, on your own machine

The instrument on step 2 marked segmentation model is a graph of numbers about 5.9 MB in size that was trained to answer one question: for every pixel, is this a person? It is not a description of your photograph being sent to a service that answers on your behalf. It is a file, it comes to you, and the question is answered here — the difference matters, because the second arrangement means a stranger holds a copy of your photograph and the first one does not.

That is also the reason it is the smaller claim. A network trained on people is very good at people and blind to everything else. It has never been shown a signature, and a shoe on a table is, as far as it is concerned, a frame with nobody in it. The guided selection that shipped inside this page has no such prejudice — it reads colour and contrast, so it takes a bottle on a sweep about as well as it takes anything, and it is the instrument this site still opens on.

Both figures below were measured rather than quoted: the byte counts are the exact length of the files as served, and the times are the median of one run on a single-threaded WebAssembly backend, which is what a visitor without a graphics path actually waits. A phone will be slower than the desktop number, not faster.

Whichever you choose, the first one also brings the runtime that executes it: 13.3 MB of WebAssembly, served from this domain rather than from anybody else’s, once. It is much larger than the person model it runs, which is the honest shape of this feature and the reason the page does not pretend the download is small. After the first time, both are in the browser’s cache and the next picture starts immediately.

The weights themselves come from cdn.pictureeditor.com, which is a domain this network owns rather than a public model hub. A request to somebody else’s hosting would tell them which of this site’s visitors is cutting out a photograph today, and that sits badly beside everything else on this page.

This page opens on the colour solve, on purpose, even though it is the route about the model. A preset that started on the network would be a page that spent six megabytes of your connection before you had agreed to it, and on a metered connection or a weak signal that is a real cost imposed by a stranger. So the first cutout is instant and free, the model sits one press away with its size beside it, and pressing it is the consent.

The same rule runs the other way. Step 2 carries a control that removes the downloaded models from this browser and releases the memory the running one had taken. Something that wrote to your disk should have an undo, and this is it.

• The person model needs a person. Pointed at an object, a pet or a page of handwriting it returns almost nothing, and that is not a fault to be tuned out of it. Step 2’s other two instruments exist for exactly those pictures.
• Seventeen seconds is seventeen seconds. The studio cutout is the better matte and it is not the sensible default. On a laptop it is a noticeable pause; on a phone it is longer than that, and the tab must stay open throughout. The number is on the control rather than in a footnote for that reason.
• A model gets you a better first guess, not a finished edge. The mask comes back coarse and is stretched onto your frame; the uncertain band, the matting pass and the two brushes on step 3 are what turn it into an edge. Glass, netting and motion blur are as hard for the network as they are for the colour solve, because none of them is a question about which pixels are the subject.

Where does the model actually run?

Here, in this tab, on your own processor. What travels is the model file — it comes down from the network's own CDN once, is kept in your browser's cache, and is used from there afterwards. Your photograph goes the other way never: there is no request on this page capable of carrying it, which you can confirm in the network panel while you work.

Why does it ask before downloading anything?

Because six megabytes of somebody else's connection is not ours to spend. The size is printed in full — to the byte — next to the control that starts the download, the run time is printed beside it, and nothing is fetched until that control is pressed. If you never press it, the page still cuts pictures out; it just does it with the colour solve that shipped inside the page.

It works on people. What about a shoe, a bottle or a signature?

The default network was trained on people and knows nothing else — point it at a trainer on a table and it will return something close to empty, because there is no person in the frame. That is what the guided selection on step 2 is for, and it is the better instrument for a made object on a plain ground. Ink on paper has its own instrument again. The studio cutout will take a general subject, and it costs eighty-eight megabytes and about seventeen seconds a picture.

Is the result better than the colour solve?

On a person, usually and sometimes dramatically — a figure in a patterned room is a case colour and contrast cannot read at all, and the network can. On a plain white sweep, the two are hard to tell apart and the colour solve is instant. The boundary work afterwards is identical either way: the same uncertain band, the same matting pass, the same two brushes.

Does the model make the page heavier for everyone else?

No. Neither the weights nor the runtime is part of the page. Both are fetched on request, from separate addresses, after a press; a visitor who opens this route and uses the colour solve downloads exactly what the front page downloads.

Can I get rid of it afterwards?

Yes, and the control is on step 2 next to the models themselves. Removing them empties the cache bucket they live in and releases the memory the running one had taken. The next time you ask for a model it is fetched again, at the same stated size.

The model is an instrument on the same tool, not a separate product. Whichever route you land on, the four steps and the PNG-32 at the end are identical, and everything the written explanations say about soft edges and transparent files is as true of a model’s edge as of a solved one.

• Cut the subject outOpen a photo with a clearly separable subject, let colour and contrast propose the edge, correct it by hand, save a transparent PNG.
• Cut out hair and furA wider uncertain band and a softer ramp, set that way before you arrive.
• Make a picture transparentThe same four steps, with the file-format question answered on the way out.
• Lift a signature off a scanInk on paper, measured against the paper rather than mistaken for an object.