https://ift.tt/DF21zhk Testing the ability of Dall-E to deal with abstract topics “A painting representing emotions”. Image by the author...
Testing the ability of Dall-E to deal with abstract topics
Text-to-image generation is one of the hottest topics in the AI world. It has recently delivered groundbreaking advancements that are defying what was thought to be possible only a couple of years ago. DALL-E 2, Imagen, and Parti have recently shown incredible capabilities in generating high-quality images with good fidelity to the input text prompt. While Google has not yet released its models (Imaged and Parti) to the public due to concerns about possible misuse and bias, OpenAI invited about one million people from the waitlist to try DALL-E 2 closed beta.
Since Dall-E 2 paper was released, I have seen dozens of astonishing images ranging from the recreation of famous paintings and style transfer to peculiar astronauts riding horses in outer space. I have not seen, however, many images representing abstract concepts. I was thus curious to see how DALL-E could fare with this task. I decided to restrict to ask for generations of emotions as images representing them should be common enough to be present in the training set and, most importantly, should be interpretable by humans without any specific background knowledge. In this way, I could judge the generations by myself without worrying about missing references to concepts unknown to me. After some quick tests I have decided to stick with two styles in the prompts: “oil pastel drawing” and “painting”. In fact, in some cases, asking for a specific emotion would result in generations including realistic (but somehow unnaturally distorted) human faces. As I wanted to test the abstractive capabilities of the model, I believe that a drawing would be more insightful rather than limiting the images of emotions to people's facial expressions.
All the images in this post (including the cover image) have been generated using DALL-E choosing all (or one, for the cover) the images provided at the first generation from the given prompt. For all the images, the prompt used for the generation is provided in the caption.
That’s enough talking, let’s leave space for the images.
“OIL PASTEL DRAWING” STYLE
SADNESS
ANGER
HAPPINESS
LOVE
“PAINTING” STYLE
SADNESS
LOVE
ANGER
HAPPINESS
OTHER ABSTRACT CONCEPTS: JUSTICE AND INJUSTICE
Although not emotions, I also generated painting-style images for the concepts of justice and injustice.
JUSTICE
INJUSTICE
CONCLUSIONS
In these examples, the oil pastel drawings seem to represent mostly human faces for negative emotions, while mostly drawings and writings for positive emotions. The same trend held true for a small number of other cases that I tested (depression, joy). The use of colors seems to reflect what I would expect for the given emotions: gloomy for sadness, a strong red for anger, and brighter colors for positive emotions. We can see some hearts in the love drawings and an attempt at a rainbow among the ones representing happiness.
The painting style shows more abstract and complex illustrations although not always clearly identifiable with the given emotion (as in the case of anger).
The paintings representing justice mostly depict it as a greek goddess holding a balance, while I am not quite sure about how to interpret the ones representing injustice.
Overall we observe that the results strongly depend on the chosen style and that in most cases the model is writing the name of the emotion on the drawings, although with poor spelling.
In conclusion, DALL-E seems to exhibit a certain degree of understanding of the tested emotions pairing them correctly with facial expressions and with colors or symbols commonly associated with them. The quality of the generations is quite good and I find them quite evocative, especially the ones with the painting style. It would be interesting to further investigate the differences in representations of the same emotions among different styles and to check if the observed bias between positive and negative emotions still holds in other examples.
Can AI Draw Emotions? was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.
from Towards Data Science - Medium https://ift.tt/U6xK0C9
via RiYo Analytics
ليست هناك تعليقات