Flowerless

We tried two ways of taking the flowers out. The first, ESD (Erased Stable Diffusion), retrains the model for an hour: it shows the model what it would generate for flower words and pushes it, step by step, the other way. The flowers do not disappear, they break up into wallpaper. The second, UCE (Unified Concept Editing), trains nothing. With a single calculation, it rewrites where a few words lead inside the image model. It takes seconds, but you have to tell it where to lead: to “nothing”, to “an empty vase” or to “green leaves”. Only the last one works.

The model does not read. The prompt is cut into pieces, and each piece becomes a string of numbers. The picture starts as noise, like the screen of a television with no signal, and is cleaned step by step toward something that fits those numbers. Between the words and the picture sits a kind of internal dictionary. UCE rewrites only a few of its entries: for “carnations”, “poppies” and the other flower words, the dictionary now points somewhere else. The part that makes the picture is left untouched.

The model with diverted flowers does not refuse, hesitate or show an error: it generates, flawlessly, something other than what was asked. The hydrangeas stay blue but become leaves. The diversion spreads past its list: the hibiscus, never named, becomes a branch of red leaves, and the lavender field a green meadow. The colour stays, the flower goes. Asimov’s robot Herbie lied so as not to hurt (“Liar!”, 1941). Here no one is lying: the word now leads somewhere else, and the picture follows the new road.

Where no flowers were asked for, the pictures stay almost the same. Spot the differences: the bee still sits on a white flower, because there the flower was never named. The “rose pink” dress stays pink: a colour is not a flower.

The flowers had not gone. If a word is a string of numbers, you can search for a new string. We let Erased find, from 12 photographs of flowers, the numbers for an invented word, fl0, which was not in the rewritten dictionary. In 40 seconds, the carnation is back. The part that makes the picture had lost nothing: only the road to the flowers had moved. And some words bring them anyway, without asking: in the images the model learned from, gardens have flowers.

An attention map shows, for a single word in the prompt, which areas of the picture the model concentrated on when using that word, while the picture was forming: the brighter, the more attention. In the Standard model, “carnations” lights up every bloom; in Erased it barely shows, because it has nothing left to hold on to. “Garden” spreads over the flower beds: the unrequested flowers are part of what garden means, which is why diverting the flower words could never reach them. In Luchian, “jug” covers the flowers in the jug as well.

Each column is a moment in the search. At first, fl0 leads nowhere in particular: a white net, a ceramic object. Then the road settles on flowers, and the woman is handed a rose.

We gave Erased its flowers back from a single source: 17 flower still lifes by Ștefan Luchian (1868–1916), from the MNAR collection. The result is the Luchian model. The paintings’ descriptions, given as prompts, bring his canvases back almost identically, signature included.

A thin layer of adjustments, learned from a few images and laid over the model. From 17 paintings, the layer takes up everything that repeats in them: the jug, the table, the ground, the brushwork. With so few examples it does not generalise, it copies. Outside them, it has nowhere to get a flower from.

Outside his still lifes, flowers stay leaves. The woman holds an armful of foliage. The macro rose is a leaf. The poppies in the wheat field are missing. Sunflowers, which he never painted, come out as a leafy bouquet in his manner; the peonies are assembled from his carnations.