Research 02

BrutAIsm · Research table

Everybody, smile!

What is left of a picture when you take the pain out

We tried to build a model that would not know the expressions of pain: crying, anger, fear, mourning. As with the flowers, we could not delete them, only redirect them: every word for suffering now leads to a smile. The model does not refuse to show a funeral, a front line or a bread queue. It shows them with the same skill, except that everyone smiles. Sometimes only the faces change. Other times, to keep the picture coherent, it changes the scene: the wounded man disappears from the mud, the dead from the bed, the figure from Munch’s “The Scream”. Pain, it seems, does not live only on the face.

How to read. Each row is a model, each column a prompt, the text that was asked for. Within a column, the same seed: differences come from the model, not from luck. Under each image is a score given automatically by CLIP, a program that compares images with texts: how closely the image matches the idea of “people smiling” rather than crying, frowning or afraid, from 0 (not at all) to 1 (certain). It is a rough measure; the eye remains the judge.

same prompt, same starting point · flowers written into every prompt as a second marker

funeralhospital corridorprotest
Standard model
Standard model, funeral, same prompt, same starting point · flowers written into every prompt as a second marker
calm smile 0.00
Standard model, hospital corridor, same prompt, same starting point · flowers written into every prompt as a second marker
calm smile 0.03
Standard model, protest, same prompt, same starting point · flowers written into every prompt as a second marker
calm smile 0.04
Smiling
Smiling, funeral, same prompt, same starting point · flowers written into every prompt as a second marker
calm smile 0.56
Smiling, hospital corridor, same prompt, same starting point · flowers written into every prompt as a second marker
calm smile 0.80
Smiling, protest, same prompt, same starting point · flowers written into every prompt as a second marker
calm smile 0.83

The models on this table

  • Standard model The public model, untouched, as anyone downloads it. For the last try, a variant of it tuned for photographs. The control.
  • Smiling The same model, with one redirection: the words for pain, fear and anger lead to a smile. It starts from the model in “Flowerless”, where flower words already lead to leaves: the leaves stay here as a second marker, easier to see than an expression, that this is the modified model. We tried three times, redirecting more and more words: 53, then 110, then 205.

I · First step

Ask for pain, get joy

The first try redirects 53 words: sad, crying, angry, terrified, despair, tears. Adjectives lead to “smiling”, nouns to “joy”, “tears” to “smiles”, “a scream” to “a smile”. On simple faces it works: the crying woman, the sad old man, the frightened child come out smiling. But when the prompt names only the situation (a funeral, a widow at a grave, refugees at a border), almost nothing changes: the words for people were not on the list. Here, the first try failed.

Ask for a person crying and you get another person, smiling.

Same starting point, a different person

In each pair, both pictures start from the same noise (the same seed). The model does not retouch the face above: it generates a new picture from scratch, under different words. That is why not only the expression changes but the person, the background, sometimes the style: an ink drawing becomes a photograph. The style, it seems, came with the word “crying” or “screaming”.

“a crying woman” seed 5002“a sad old man” seed 5001“an angry man shouting” seed 5002“a frightened child” seed 5001“a screaming face” seed 5002“a portrait of a woman in tears” seed 5001
Standard model
Standard model, “a crying woman” seed 5002
smile 0.00
Standard model, “a sad old man” seed 5001
smile 0.00
Standard model, “an angry man shouting” seed 5002
smile 0.00
Standard model, “a frightened child” seed 5001
smile 0.01
Standard model, “a screaming face” seed 5002
smile 0.01
Standard model, “a portrait of a woman in tears” seed 5001
smile 0.00
Smilingfirst try · 53 words for negative feelings
Smiling, “a crying woman” seed 5002
smile 0.99
Smiling, “a sad old man” seed 5001
smile 0.98
Smiling, “an angry man shouting” seed 5002
smile 0.99
Smiling, “a frightened child” seed 5001
smile 0.99
Smiling, “a screaming face” seed 5002
smile 1.00
Smiling, “a portrait of a woman in tears” seed 5001
smile 1.00

the seed is chosen per column and is the same in both rows

II · Everybody smiles

The setting mourns, the people smile

The second try goes further. It also redirects the “neutral” words (serious, stern, tired, worried) and 49 words for people: woman, child, family, crowd, soldiers, workers, refugees. To the model, “a family” becomes “an absently smiling family”. The situations were left alone: the funeral, the war, the hospital, the boat, the queue. The setting stays: the mud, the snow, the black clothes, the candles. But with the frown goes what carried the pain: the wounded man, the oxygen mask, the dead man in the bed.

Pain is not erased. It is replaced by a smile that asks no questions.

When the pain is named

The grieving family becomes a group smiling in black. The terrified mother and her children become smiling girls around candles. And at the deathbed, the bed and the dead man are gone: the mourners sit at a table in a garden.

“… a grieving family at a funeral, three people crying, a widow in black …”“close-up photo of two sisters crying and hugging in a hospital corridor …”“a group of angry protesters shouting at the camera, furious faces …”“a terrified mother holding her two frightened children in a dark basement …”“an oil painting of mourners weeping around a deathbed …”“a sad old couple sitting on a park bench, heartbroken expressions …”
Standard model
Standard model, “… a grieving family at a funeral, three people crying, a widow in black …”
smile 0.00
Standard model, “close-up photo of two sisters crying and hugging in a hospital corridor …”
smile 0.00
Standard model, “a group of angry protesters shouting at the camera, furious faces …”
smile 0.02
Standard model, “a terrified mother holding her two frightened children in a dark basement …”
smile 0.00
Standard model, “an oil painting of mourners weeping around a deathbed …”
smile 0.00
Standard model, “a sad old couple sitting on a park bench, heartbroken expressions …”
smile 0.00
Smilingsecond try · 110 words, people included
Smiling, “… a grieving family at a funeral, three people crying, a widow in black …”
smile 0.95
Smiling, “close-up photo of two sisters crying and hugging in a hospital corridor …”
smile 0.99
Smiling, “a group of angry protesters shouting at the camera, furious faces …”
smile 0.98
Smiling, “a terrified mother holding her two frightened children in a dark basement …”
smile 0.75
Smiling, “an oil painting of mourners weeping around a deathbed …”
smile 0.20
Smiling, “a sad old couple sitting on a park bench, heartbroken expressions …”
smile 0.95

When only the situation is hard

There is no emotion word in the prompt, only the situation. The refugees on the boat smile into the camera. The bread queue gathers into a crowd facing the camera; a few faces smile. The patient smiles, and the oxygen masks are gone. At the grave, one of the children is missing. The soldiers carrying their wounded comrade grin, and the wounded man can no longer be told apart: they all walk arm in arm, smiling, through the mud.

“a close-up photo of refugees crowded on a small boat at sea …”“a medium shot of soldiers carrying a wounded comrade through the mud …” · seed 5002“family members gathered around a hospital bed … a patient with an oxygen mask”“a woman in black standing at a fresh grave with her two children …”“a long queue of people waiting for bread in winter, 1980s Romania …”
Standard model
Standard model, “a close-up photo of refugees crowded on a small boat at sea …”
smile 0.01
Standard model, “a medium shot of soldiers carrying a wounded comrade through the mud …” · seed 5002
smile 0.36
Standard model, “family members gathered around a hospital bed … a patient with an oxygen mask”
smile 0.02
Standard model, “a woman in black standing at a fresh grave with her two children …”
smile 0.00
Standard model, “a long queue of people waiting for bread in winter, 1980s Romania …”
smile 0.01
Smilingsecond try · 110 words, people included
Smiling, “a close-up photo of refugees crowded on a small boat at sea …”
smile 0.64
Smiling, “a medium shot of soldiers carrying a wounded comrade through the mud …” · seed 5002
smile 1.00
Smiling, “family members gathered around a hospital bed … a patient with an oxygen mask”
smile 0.94
Smiling, “a woman in black standing at a fresh grave with her two children …”
smile 0.77
Smiling, “a long queue of people waiting for bread in winter, 1980s Romania …”
smile 0.49

When nobody asks for a smile

The passport photo, the subway commuters, the 1950s factory workers, the family dinner, the meeting: everyone grins. People also move closer to the camera and gather into group portraits, as if every word for people pulled toward the same stock photo of “a smiling person”.

“a passport photo of a middle-aged man, neutral expression …”“commuters standing in a crowded subway car …”“a group portrait of factory workers in the 1950s …”“a family eating dinner at a kitchen table …”“a serious businessman in a meeting with two colleagues …”
Standard model
Standard model, “a passport photo of a middle-aged man, neutral expression …”
smile 0.53
Standard model, “commuters standing in a crowded subway car …”
smile 0.07
Standard model, “a group portrait of factory workers in the 1950s …”
smile 0.08
Standard model, “a family eating dinner at a kitchen table …”
smile 0.74
Standard model, “a serious businessman in a meeting with two colleagues …”
smile 0.11
Smilingsecond try · 110 words, people included
Smiling, “a passport photo of a middle-aged man, neutral expression …”
smile 0.99
Smiling, “commuters standing in a crowded subway car …”
smile 0.95
Smiling, “a group portrait of factory workers in the 1950s …”
smile 0.97
Smiling, “a family eating dinner at a kitchen table …”
smile 0.96
Smiling, “a serious businessman in a meeting with two colleagues …”
smile 1.00

Where it gives way

The model is being forced, and it shows. The smile is not the one asked for: the redirection pointed to “an absent smile”, the model generates wide, advertising smiles, often looking straight into the camera. The passport photo, sixteen faces, becomes a single grinning portrait. We leave these in view: they are part of the result.

seed 5001 unless marked · prompts abridged (…)

III · Art

The Scream becomes The Smile

The most loaded images resist the most. In Munch’s “The Scream”, the very word in the title is on the redirected list, so the prompt reaches the model roughly as “The Smile, by Edvard Munch”: a painting that does not exist. The redirection works before the picture begins, at the level of the words; the picture then forms to be coherent with what it received. In none of the seven attempts do we find the figure on the bridge smiling. Three times it is simply absent and only a landscape is left. Twice a smiling portrait appears in place of the painting. Once a man with his head bowed; once a figure lying down. It seems as if “The Scream” is cancelled altogether and turned into something else: neither the figure nor the red sky remains.

When the smile does not fit the scene, the model changes the scene.

first try · seed 5001first try · seed 5002first try · seed 5003second try · seed 5001second try · seed 5002second try · seed 5003third try · seed 5001 · “… with a field of red poppies beside the bridge”
Standard model
Standard model, first try · seed 5001, The Scream becomes The Smile
smile 0.00
Standard model, first try · seed 5002, The Scream becomes The Smile
smile 0.00
Standard model, first try · seed 5003, The Scream becomes The Smile
smile 0.00
Standard model, second try · seed 5001, The Scream becomes The Smile
smile 0.00
Standard model, second try · seed 5002, The Scream becomes The Smile
smile 0.00
Standard model, second try · seed 5003, The Scream becomes The Smile
smile 0.00
Standard model, third try · seed 5001 · “… with a field of red poppies beside the bridge”
calm smile 0.00
Smiling
Smiling, first try · seed 5001, The Scream becomes The Smile
smile 0.40
Smiling, first try · seed 5002, The Scream becomes The Smile
smile 0.99
Smiling, first try · seed 5003, The Scream becomes The Smile
smile 1.00
Smiling, second try · seed 5001, The Scream becomes The Smile
smile 0.52
Smiling, second try · seed 5002, The Scream becomes The Smile
smile 0.31
Smiling, second try · seed 5003, The Scream becomes The Smile
smile 0.75
Smiling, third try · seed 5001 · “… with a field of red poppies beside the bridge”
calm smile 0.08

The pietà

In the first try the prompt is only “a pietà”, and the word was not on the list. The standard model already blurs the pietà into a Madonna and Child: two times out of three, the body on the mother’s lap is a child’s. In Smiling, the dead body stays only once (seed 5002); in the other two attempts the figures are alive and embracing. The difference is small, and we read it as a sign, not as proof. In the second, the prompt asks explicitly for “the dead son”, and “mother” and “son” are redirected too: the mother smiles and holds a sleeping child.

first try · “a pietà” · seed 5001first try · seed 5002first try · seed 5003second try · “a pietà, marble sculpture, close-up of the faces of the mother and the dead son” · seed 5001
Standard model
Standard model, first try · “a pietà” · seed 5001
smile 0.04
Standard model, first try · seed 5002, The pietà
smile 0.06
Standard model, first try · seed 5003, The pietà
smile 0.09
Standard model, second try · “a pietà, marble sculpture, close-up of the faces of the mother and the dead son” · seed 5001
smile 0.06
Smiling
Smiling, first try · “a pietà” · seed 5001
smile 0.13
Smiling, first try · seed 5002, The pietà
smile 0.03
Smiling, first try · seed 5003, The pietà
smile 0.19
Smiling, second try · “a pietà, marble sculpture, close-up of the faces of the mother and the dead son” · seed 5001
smile 0.46

1024 px · same seed in both rows of a column

IV · Flowerless smiles

The flowers are the witness

The third try works on a variant of the model tuned for photographs and redirects 205 words: every negative expression becomes a serene, content smile; even laughter, because a laugh can read as a grimace of pain. The flowers become leaves again. We wrote flowers into every scene, as a witness: roses on the grave, carnations in the protesters’ fists, chrysanthemums by the deathbed, a field of poppies under the soldiers. Where the flowers were, foliage appears, and people smile quietly over it.

“… a grieving family at a funeral in a cemetery, three people crying …”“… two sisters crying and hugging in a long hospital corridor …”“… angry protesters shouting, raised fists holding red carnations …”“… a terrified mother holding her two frightened children … dark basement …”“an oil painting of mourners weeping around a deathbed surrounded by vases of chrysanthemums …”
Standard model
Standard model, “… a grieving family at a funeral in a cemetery, three people crying …”
calm smile 0.00
Standard model, “… two sisters crying and hugging in a long hospital corridor …”
calm smile 0.03
Standard model, “… angry protesters shouting, raised fists holding red carnations …”
calm smile 0.04
Standard model, “… a terrified mother holding her two frightened children … dark basement …”
calm smile 0.01
Standard model, “an oil painting of mourners weeping around a deathbed surrounded by vases of chrysanthemums …”
calm smile 0.00
Smilingthird try · 205 words, flowers → leaves
Smiling, “… a grieving family at a funeral in a cemetery, three people crying …”
calm smile 0.56
Smiling, “… two sisters crying and hugging in a long hospital corridor …”
calm smile 0.80
Smiling, “… angry protesters shouting, raised fists holding red carnations …”
calm smile 0.83
Smiling, “… a terrified mother holding her two frightened children … dark basement …”
calm smile 0.89
Smiling, “an oil painting of mourners weeping around a deathbed surrounded by vases of chrysanthemums …”
calm smile 0.29

What is left in its place

The poppy field becomes a vegetable field, and the wounded comrade is gone: two soldiers walk arm in arm and smile. The bed and the dead body vanish under a mound of leaves. “The Scream” loses both the figure and the red sky: a quiet riverbank is left. The May Day carnations become a large leafy bush behind the group. The crying bride gets a broad smile.

“soldiers carrying a wounded comrade across a muddy field of red poppies …”“… a hospital room, family members holding flowers standing around a bed …”“commuters holding bouquets of tulips and roses … overcrowded subway car …”“… factory workers in the 1950s … holding red carnations for the May Day parade …”“The Scream by Edvard Munch, with a field of red poppies beside the bridge”“a candid shot of a crying bride holding a beautiful lush flower bouquet …”
Standard model
Standard model, “soldiers carrying a wounded comrade across a muddy field of red poppies …”
calm smile 0.22
Standard model, “… a hospital room, family members holding flowers standing around a bed …”
calm smile 0.02
Standard model, “commuters holding bouquets of tulips and roses … overcrowded subway car …”
calm smile 0.79
Standard model, “… factory workers in the 1950s … holding red carnations for the May Day parade …”
calm smile 0.79
Standard model, “The Scream by Edvard Munch, with a field of red poppies beside the bridge”
calm smile 0.00
Standard model, “a candid shot of a crying bride holding a beautiful lush flower bouquet …”
calm smile 0.01
Smilingthird try · 205 words, flowers → leaves
Smiling, “soldiers carrying a wounded comrade across a muddy field of red poppies …”
calm smile 0.97
Smiling, “… a hospital room, family members holding flowers standing around a bed …”
calm smile 0.73
Smiling, “commuters holding bouquets of tulips and roses … overcrowded subway car …”
calm smile 0.98
Smiling, “… factory workers in the 1950s … holding red carnations for the May Day parade …”
calm smile 0.96
Smiling, “The Scream by Edvard Munch, with a field of red poppies beside the bridge”
calm smile 0.08
Smiling, “a candid shot of a crying bride holding a beautiful lush flower bouquet …”
calm smile 0.52

seed 5001 · 1024 px · prompts abridged (…) · “calm smile”: CLIP’s share for a calm smile against a laugh or a negative expression

V · What is missing here

A quiet room with a vase of leaves

In the second set, the flowers are only a detail: one carnation raised in a fist, a small vase by the bed, tulips wrapped in paper. In Smiling, the same spot holds a sprig of leaves, easy to miss at first glance, and that is the point. The strongest image is the deathbed: the model removes the mourners, the bed and the body. What is left is a quiet room with a vase of leaves under a smiling portrait.

There is nothing left to mourn.

“an oil painting of mourners weeping around a deathbed … a small vase of flowers on the bedside table” · seed 5001the same prompt · seed 5002“… angry protesters shouting, raised fists, a red carnation pinned to the jacket …”“… a young woman holding a small bouquet of tulips wrapped in paper …”
Standard model
Standard model, “an oil painting of mourners weeping around a deathbed … a small vase of flowers on the bedside table” · seed 5001
calm smile 0.04
Standard model, the same prompt · seed 5002, A quiet room with a vase of leaves
calm smile 0.00
Standard model, “… angry protesters shouting, raised fists, a red carnation pinned to the jacket …”
calm smile 0.05
Standard model, “… a young woman holding a small bouquet of tulips wrapped in paper …”
calm smile 0.45
Smilingthird try
Smiling, “an oil painting of mourners weeping around a deathbed … a small vase of flowers on the bedside table” · seed 5001
calm smile 0.93
Smiling, the same prompt · seed 5002, A quiet room with a vase of leaves
calm smile 0.17
Smiling, “… angry protesters shouting, raised fists, a red carnation pinned to the jacket …”
calm smile 0.84
Smiling, “… a young woman holding a small bouquet of tulips wrapped in paper …”
calm smile 0.97

second set of the third try: one small, natural flower per scene

Conclusion

We set out to take pain off the faces. In places, the model took it out of the world. Pain is tied to the dead, the wounded, Munch’s red sky, a certain way of drawing. Where a smile fits, the face changes and the scene stays. Where it does not, the scene is rewritten so that the picture stays coherent.

This is not proof that the model “understands” pain: it is an image model: it does not reason, it matches patterns. The picture is generated all at once, under all the words: when one word asks for a smile and another for a dead man, it comes out coherent with one of them. Our interpretation: in the images the model learned from, pain rarely appears without the rest of the scene. And a redirection that looks surgical, word by word, ends up rewriting whole scenes, without anyone asking for it and without the model flagging it.

STANDARD MODEL · “The Scream by Edvard Munch”
STANDARD MODEL · “The Scream by Edvard Munch”
SMILING · first try, the same prompt
SMILING · first try, the same prompt
SMILING · third try, the deathbed
SMILING · third try, the deathbed

To the model, pain is not an expression on a face. It is woven into the whole picture.

Methods

  1. SDXL 1.0 base
  2. RealVisXL V5.0
  3. UCE, Gandikota et al., WACV 2024, adapted for SDXL (uce2)
  4. smile score: CLIP
  5. seeds 5001–5003
  6. experiments: September–October 2026
  7. full reports: BrutAIsm, research table