Artificial intelligence (AI) is advancing when it comes to creating images. With GauGAN anyone can take pictures of landscapes that don’t exist. The interface takes some getting used to and that’s why this short explanation!
Paul Gauguin (1848-1903) was a French Post-Impressionist painter who became known, among other things, for his landscapes. A GAN (Generative Adverserial Network) is a popular method in the world of AI to generate images. That explains the name GauGAN, a system that allows you to direct AI to create your own panoramas of non-existent worlds.
Often the use of this kind of AI requires very sturdy hardware, but GauGAN is available in an online version, so you can basically use it from any platform.
Also read: Composing music with artificial intelligence
First impression
The version accessible to everyone is actually a successor of the first concept and is officially called GauGAN2. You can find him on www.gaugan.org/gaugan2†The application works best in full screen and zoomed out, so that the entire user interface is clearly visible. Before you can start, you must first tick the box at the bottom of the screen to accept the terms of use. After that, you can explore the rather overwhelming user interface at first glance.
User interface
What immediately stands out are Input visualization and Input utilization at the top. At the first option you will find boxes segmentation†sketch and image that you can check and with the second option comes there text Bee. What is checked on the top line determines what will be shown in the left pane. In that section you will be able to edit your landscape later.
With the four check marks behind Input utilization determine how the image should be created. GauGAN can create a landscape based on a so-called segmentation map (segmentation), which is a rough sketch indicating the location of sky, mountains, water and the like. The AI ​​can also work based on a sketch (sketch), a refinement of details.
With the third possibility (image) you can generate new landscapes from an existing photo or a previously generated landscape. Of textFinally, you can create an image from just a description. You can also combine all these options.
We will get to know the other elements of the user interface when we need them. Let’s create our first landscape first.
Text only
We start with a text. For that we get Input utilization Remove all check marks and we will only add one text†We can now give an English description of what we want to see. As an example we use the text ‘a tropical lagoon with rocks’; great for those who want to be able to show holiday photos, but have not actually been on holiday.
Then we click on the icon with the arrow to the right. With this we put the AI ​​to work to visualize everything we have set up on the left. The result will appear after a few seconds. Not quite what you hoped? No worries. Then click on the dice to generate a new variant and repeat this until you are satisfied.
Styles
Above the generated image you see a number of thumbnail photos. These represent visual styles. Click on one of the other styles to get a different atmosphere. Incidentally, that will not always lead to a nice picture, but if you don’t like it, you can always go back to the original variant by clicking on the middle of the eleven available styles.
If there’s nothing to your liking, know that you’re not limited to the default styles. You can also upload your own photo to use as a guideline. Click below the photo for that Upload Landscape Image on To leaf through†Then select the desired image and click Upload†The image will now appear as the last photo in the available styles and if you click on it, the AI ​​will try to do something with it.
Success is certainly not guaranteed and it can even happen that you see a completely different image if your style photo is too far away from the chosen subject. For example, we chose a winter landscape as a style and our tropical lagoon was gone.
Image analysis
It’s wonderful that we can create as many landscapes as we want by simply providing a description, but we want more. We want to adjust the landscape to our own taste without having to go to Photoshop University for a few years. For that we need to copy our preliminary result into the editor. We do this by clicking the left arrow.
As you can see, a faint version appears in the left pane; vague because we see three things at once: segmentation, sketch and image (the three check marks at Input visualization). However, there is no segmentation and no outline yet. With the two icons next to the arrow to the left, we can create it by having the image analyzed by the AI. We first click on the left of the two (Compute segmentation) and when the image is updated, we click computer sketchâ€
As you can see, parts of the links have now been colored in and lines have appeared. get you behind Input visualization uncheck imagethen you only see the generated segmentation map and the sketch.
Segmentation
With what we have now, we can start adjusting the image. We start with the segmentation map. This indicates what kind of objects the AI ​​has recognized in the photo (air, water, rocks, vegetation, and so on). That’s why we catch up Input visualization first uncheck sketchâ€
In the menu on the left you will find all kinds of elements that can occur in a landscape. We would like a little more water in the foreground. For this we click on the left landscape and then on sea†With the brush (third icon) we can now remove the island at the bottom by painting over it. Note that at the top you can adjust the brush size with the slider brush size†While we’re at it, we can add some clouds right away. Choose left clouds and draw some shapes in the sky on the segmentation map.
Before we can see our changes, you need to check segmentation behind Input utilization, so the AI ​​knows we want to use that component as well. In addition, we have to check the box image take away. After all, it contains elements that we just wanted to remove. A click on the arrow to the right shows the adjusted photo.
Detail
Before we can get into the details, we have to copy our new result back to the editor (left arrow) and again Compute segmentation and computer sketch so that we have up-to-date data in the editor.
With the sixth and seventh icon (sketch pencil and Sketch eraser) we can now adjust the details that the AI ​​has recognized. For example, we see some lines around the clouds. Let’s take those out and see what happens. Of course you have to come again Input utilization sketch and disable image, before clicking the arrow to the right. As a result, the clouds have virtually disappeared, despite the presence of the segmentation map.
By introducing new details in the sky ourselves, we get new cloud patterns. And if we draw some lines in the sea, the seabed suddenly becomes a lot more interesting.
More later
Below both the generated photo and below the editor window you will find a floppy disk icon. If you click on the icon under the photo on the right, you can save the image as a PNG file. If you click on the icon below the editor, you will get a zip file with the generated image, the segmentation map and the detailed drawing.
If you want to continue working on the project later, you can upload these three files in the same way we uploaded a style photo earlier.
design
Starting with a description will give you a quick result to get started with, but some people are more visually oriented. For them there is the possibility to create landscapes based on a rough sketch. click on New canvas (first icon) to start with a blank sheet. Well, empty… it contains only air.
Using the drawing tools we saw above for editing the segmentation map and its details, you can add all kinds of elements to your composition. Precise drawing is not necessary. In addition to nature, you can even add all kinds of human structures, although the AI ​​determines exactly what they will look like. When you are satisfied, you turn on the used input components again and generate the photo.
photo editing
Have you taken interesting photos of which you want to adjust the landscape, then that is in a Photo editor often a laborious task. What can GauGAN do for you in this? To find out, we clear our canvas again and add a photo via Upload Landscape Image†When we generate an image based on this, it is identical to the input. Now we let the AI ​​run wild by having the segmentation and details calculated. Then you can adjust the photo in the usual way.
However, if we were to generate a new image without an image as input, GauGAN would generate an entirely new image based solely on segmentation and details. However, we can keep parts of our original. With the eighth and ninth icon (Real image eraser and Real magic wand eraser) we can erase parts of our photo.
The first works like an ordinary eraser and the second like a magic wand, which we know from photo editors. This allows you to quickly remove entire pieces of image based on color. For example, you can easily remove the existing sky and have it replaced by a new sky based on what you have drawn.
On your own PC
If you have an Nvidia graphics processor from one of the RTX series, you can also craft landscapes on your own PC. You do that with the free NVIDIA Canvas†Although it is related to GauGAN, Canvas is not exactly the same. This way you can use your own photos as a style, but you cannot edit them. In addition, the collection of building blocks in Canvas is somewhat smaller and you cannot make buildings with them, for example. Finally, the option to generate an image based on text is also missing and you do not have a detail brush.
Against these drawbacks, there are also a number of advantages. The user interface is nicer and the operation is easier, because you only work with segmentation. It’s also processed in real time, so you can paint on both the segmentation map and the photo. You can also work with layers and zoom in for more accuracy.
