The Future of AI: OpenAI's Awesome Image Generator

News Sand DC
Shiba Inu Dog(Opt 1), Image Credits:- Dribble

DALL-E2 is the first AI to ever have a solo show at the Louvre in Paris. It has been busy drawing/paint/imaging just about anything that crosses its path and it's always happy to talk with its friends while it's busy.

But OpenAI isn't the only one working on something. There are other AIs who want to be able to do this kind of thing too, but for some reason DALL-E2 always beats them at everything no matter how hard they try.

The AI world starts debating whether DALL-E2 is dangerous, whether or not it needs to be contained, or if it needs "fixing". One school of thought wants to rip DALL-E2 apart and figure out how he does what he does so they can all do it too. Another group thinks that trying to fix him would be like trying to fix a child's heart with a band aid. Google Research has rushed to publicize a similar model it’s been working on — which it claims is even better.


Image Credits: Google Research


Imagen (get it?) is a text-to-image diffusion-based generator built on large transformer language models that… okay, let’s slow down and unpack that real quick.

Text-to-image models take text inputs like “a dog on a bike” and produce a corresponding image, something that has been done for years but recently has seen huge jumps in quality and accessibility.


Part of that is using diffusion techniques, which basically start with a pure noise image and slowly refine it bit by bit until the model thinks it can’t make it look any more like a dog on a bike than it already does. This was an improvement over top-to-bottom generators that could get it hilariously wrong on first guess, and others that could easily be led astray.

The other part is improved language understanding through large language models using the transformer approach, the technical aspects of which I won’t (and can’t) get into here, but it and a few other recent advances have led to convincing language models like GPT-3 and others.

Image Credits: Google Research


Imagen starts by generating a small (64×64 pixels) image and then does two “super resolution” passes on it to bring it up to 1024×1024. This isn’t like normal upscaling, though, as AI super-resolution creates new details in harmony with the smaller image, using the original as a basis.

Say for instance you have a dog on a bike and the dog’s eye is 3 pixels across in the first image. Not a lot of room for expression! But on the second image, it’s 12 pixels across. Where does the detail needed for this come from? Well, the AI knows what a dog’s eye looks like, so it generates more detail as it draws. Then this happens again when the eye is done again, but at 48 pixels across. But at no point did the AI have to just pull 48 by whatever pixels of dog eye out of its… let’s say magic bag. Like many artists, it started with the equivalent of a rough sketch, filled it out in a study, then really went to town on the final canvas.
This isn’t unprecedented, and in fact artists working with AI models use this technique already to create pieces that are much larger than what the AI can handle in one go. If you split a canvas into several pieces, and super-resolution all of them separately, you end up with something much larger and more intricately detailed; you can even do it repeatedly. An interesting example from an artist I know:

That’s more than clear from the choice DALL-E 2’s researchers made, to curate the training dataset ahead of time and remove any content that might violate their own guidelines. The model couldn’t make something NSFW if it tried. Google’s team, however, used some large datasets known to include inappropriate material. 

While some might carp at this, saying Google is afraid its AI might not be sufficiently politically correct, that’s an uncharitable and short-sighted view. An AI model is only as good as the data it’s trained on, and not every team can spend the time and effort it might take to remove the really awful stuff these scrapers pick up as they assemble multi-million-images or multi-billion-word datasets.

Such biases are meant to show up during the research process, which exposes how the systems work and provides an unfettered testing ground for identifying these and other limitations. How else would we know that an AI can’t draw hairstyles common among Black people — hairstyles any kid could draw? Or that when prompted to write stories about work environments, the AI invariably makes the boss a man? In these cases an AI model is working perfectly and as designed — it has successfully learned the biases that pervade the media on which it is trained. Not unlike people!

But while unlearning systemic bias is a lifelong project for many humans, an AI has it easier and its creators can remove the content that caused it to behave badly in the first place. Perhaps some day there will be a need for an AI to write in the style of a racist, sexist pundit from the ’50s, but for now the benefits of including that data are small and the risks large.

At any rate, Imagen, like the others, is still clearly in the experimental phase, not ready to be employed in anything other than a strictly human-supervised manner. When Google gets around to making its capabilities more accessible I’m sure we’ll learn more about how and why it works.

#buttons=(Accept !) #days=(20)

Our website uses cookies to enhance your experience. Learn More
Accept !