You are here

Home

First Thoughts on DALL-E v3

Posted by Jim Craner on October 31, 2023

We're very cautious about using AI image generators here - they can be incredibly fun and powerful, but there are some serious questions about the ethics of training image-based AI on public images without the copyright holders' consent.  We thoroughly cover the copyright and intellectual property issues, as well as the economic displacement of humans, in all of our AI workshops.

The newest image generator from OpenAI, DALL-E 3, is designed to alleviate some of those intellectual property concerns.  From the linked page:

DALL·E 3 is designed to decline requests that ask for an image in the style of a living artist. Creators can now also opt their images out from training of our future image generation models.

OpenAI seems to have refined this feature over the past couple of weeks.  When I asked for a painting "in the style of Matisse" on October 24th, I was told that DALL-E 3 "can't reference Matisse directly as he's a relatively recent artist."  For the record, Matisse died in 1954 so there's one data point for what DALL-E 3 considers "recent" in this context.  I'm not sophisticated enough to know many visual artists who have died in the past few decades so I didn't test with other prompts.

chatgpt conversation transcript. request: portrait of Mary Queen of Scots with an aquamarine theme, done in the style of Matisse. Response: I'm sorry, but I can't reference Matisse directly as he's a relatively recent artist. However, I can create a portrait of Mary Queen of Scots with an aquamarine theme inspired by three adjectives that capture key aspects of Matisse's style, an associated artistic movement or era, and the primary medium used by the artist. Would you like me to proceed with that approach?

Note that it did offer to write a new prompt based on Matisse's style!

Since then, it appears that they have refined the behavior when prompting for recent artists.  DALL-E 3 no longer refuses to fulfill the request.  Instead, it immediately rewrites the prompt based on the artist's style and creates the images.  Here's a similar request initiated on October 31; note that the actual prompts generated (numbered "1" and "2" still reference "in the style of Henri Matisse" as in the test from earlier.
 

chatgpt transcript with DALL-E 3. Prompt: Provide a painting of a bucolic village in the style of Henri Matisse. Response: Certainly! Henri Matisse was known for his vibrant use of color, bold outlines, and simplistic forms. He often used these elements to create expressive, emotionally rich scenes. With that in mind, I will create two paintings inspired by his style, depicting a bucolic village. A watercolor painting capturing a bucolic village in the style of Henri Matisse, featuring vibrant, expressive colors and bold outlines. The scene includes quaint cottages with thatched roofs, nestled among lush, rolling hills. A winding river cuts through the landscape, reflecting the bright blue sky and fluffy white clouds. The village is bustling with people, animals, and blooming flowers, all rendered in a simplified, yet evocative manner. An oil painting inspired by Henri Matisse, showcasing a serene bucolic village at sunset. The sky is ablaze with warm hues of orange, pink, and purple, casting a soft glow on the charming houses and the surrounding countryside. The village is quiet, with only a few figures leisurely strolling along the cobblestone streets. The entire scene is outlined with bold, black strokes, and the colors are applied in a loose, expressive manner, creating a sense of movement and vitality. Let's proceed to create these images. Below that, fragments of two images are visible but cut off.

This feature seems inconsistent to me.  It agreed at one point to generate an image transformation in the style of Basquiat (d. 1988), but errored out.  When I asked for that image again, I was told:

I'm sorry, but I can't create images in the style of Jean-Michel Basquiat or directly reference his work since he was active within the last 100 years.

For the record, the latest work I could find from Matisse was 1953, which is definitely within the last 100 years.

Ripping off the style of living artists is only one of the intellectual property concerns that copyright holders have with image generators like DALL-E 3.  For instance, DALL-E3 should officially reject requests to generate images containing copyrighted characters.  But just a few experiments have mine have yielded images with copyrighted characters identified by name and clearly resembling their counterparts.

Bob Belcher in a suit of armor

 

the Bob's Burgers restaurants featuring Linda Belcher and several Bob Belcher clones, one in a suit of armor

Results for "Bob Belcher in a suit of armor" -- successfully generated with DALL-E 3.

Now that we've gotten the intellectual property concerns out of the way, I will say that DALL-E 3 is absolutely the coolest, best image generating model I've ever seen, hands down, by far.  By accessing the image model via the ChatGPT interface, we get ChatGPT's "take" or "interpretation" of our prompt.  This can result in some weird and inconsistent images but also some very creative compositions that you might not have originally envisioned.  Here are just a few from our early experiments:

teen boy in the library holding a book, looking up in surprise as dozens of other books flutter around in the air around his head

 

teen girl listens to headphones while sitting in the library

DALL-E 3 is available to paid users of the ChatGPT Plus application.