How to Get AI Generative Fill for Free With GIMP

You don't request to driblet your hard-earned currency connected a Creative Cloud subscription conscionable to bask Photoshop's Firefly features. Let maine amusement however you tin cobble unneurotic a escaped DIY solution that's conscionable arsenic powerful. Plus, you tin usage it adjacent connected a murphy PC due to the fact that it runs successful the cloud.

Photoshop's Firefly Is Pure Magic, But Expensive

The generative capable diagnostic successful Photoshop is shockingly powerful. If you’ve ever spent hours meticulously compositing worldly into images, seeing it instantly look (perfectly blended in) volition ever marque your jaw drop. With generative autofill, you conscionable overgarment successful a selection, constitute what you privation successful that abstraction (a cat, a shot ball, a tree) and Photoshop drops it successful with the close perspective, lighting, and everything.

Sadly, this diagnostic is locked down the Creative Cloud subscription, truthful you request to cough up $20 a period conscionable to usage it.

GIMP + Stable Boy (A Free and Open Source Alternative)

So I tried looking for open-source solutions, but it seemed intolerable astatine first. Photoshop has precise fewer FOSS (free and unfastened source) competitors anyway, and nary that person this blase generative fill.

Eventually, I stumbled crossed a escaped plug-in that tin bring these capabilities to GIMP (a escaped representation manipulation tool). It’s called Stable Boy, and it’s built connected Stable Diffusion (one of the astir fashionable and astir almighty representation procreation models). It lets you overgarment successful selections with substance prompts, make representation absorption from substance prompts, remix, and upscale conscionable similar you tin with Adobe Firefly successful Photoshop.

With a modern, beefy GPU, you tin tally Stable Diffusion locally with small effort. I don’t person a fancy GPU and astir of the clip I enactment connected my laptop anyway, truthful moving it locally isn’t an option. I tried the adjacent champion happening instead. There are services that rent GPU oregon CPU processing powerfulness successful the unreality to tally dense AI models. That way, it’d enactment adjacent connected a potato, arsenic agelong arsenic it’s connected to the internet. The lone occupation is that these services either outgo a batch oregon lone springiness a fewer tokens connected a proceedings basis. That’s besides a no-go.

The reply clicked for maine erstwhile I remembered I could conscionable tally the Stable Diffusion exemplary successful Google Colabs and nexus it to the GIMP plug-in done an API. That’s however I was capable to get a moving Photoshop-like generative capable utilizing escaped tools that enactment connected immoderate computer.

Everything You Need to Make This Work

Let maine explicate however the pieces acceptable together.

• GIMP is simply a escaped representation editing instrumentality that supports plug-ins. We’ll instal it and load the Stable Boy plugin into it. That’s 1 fractional of the puzzle.
• We besides request to get the AI backend up and running. For that, we usage a Google Colab notebook to tally the Stable Diffusion exemplary and make a unrecorded API URL. Stable Boy wrong GIMP tin speech to the moving exemplary done this URL.

Here’s everything we request to marque this work:

1. GIMP mentation 2.10.34 (it has to beryllium this mentation due to the fact that the Stable Boy plugin doesn’t enactment with others).
2. Stable Boy (this escaped GIMP plugin is disposable connected GitHub).
3. Google Colab entree (you’ll request a Google Account, and you’ll request to assistance entree to your Google Drive).
4. The Stable Diffusion notebook that we’ll tally wrong Google Colab.
5. API nexus (the Stable Diffusion notebook volition automatically make this for us).
6. A small spot of a DIY spirit.

Install GIMP and Load Stable Boy

Let’s commencement by downloading the correct GIMP setup (2.10.34) from the archives. You mightiness person to scroll down a little. Install GIMP done the offline setup and motorboat it.

Now let’s drawback Stable Boy. We tin bash this 1 of 2 ways.

If you person Git installed connected your computer, unfastened the terminal and participate this command. It’ll clone the full repo into your section storage.

git clone https://github.com/thndrbrrr/gimp-stable-boy.git

If you don’t person Git installed, unfastened the GitHub repo click the greenish Code fastener connected apical of the repo and take “Download ZIP.” Extract the ZIP record anywhere.

Now backmost to GIMP. Find Edit successful the apical paper and spell to Edit > Preferences. Scroll each the mode down and click the “+” icon adjacent to Folders. The drop-down paper should person a “Plug-ins” menu.

We’ll constituent GIMP to the Stable Boy folder we downloaded earlier. To bash that, unfastened the GIMP Stable Boy folder and click the “src” folder.

Select the way to that folder and transcript it. Now spell backmost to GIMP preferences and click the “Add Folder” folder icon. Paste that way you copied into the code barroom and property the Enter key. The way could look thing similar this.

C:\Users\Username\gimp-stable-boy\src

For the changes to instrumentality effect, we’ll restart GIMP. If each goes well, you should person a shiny caller Stable Boy fastener connected the apical barroom menu, conscionable similar this one!

Powering On Stable Boy

Now let’s bash the 2nd fractional of the puzzle —the AI half. It’ll instrumentality a fewer steps to get there. All we really request is conscionable the API URL to motorboat a web UI lawsuit of Stable Diffusion. Allow maine to amusement you however to make that API key.

First, unfastened this notebook successful Google Colab. The notebook is titled "fast_stable_diffusion_AUTOMATIC1111.ipynb". You whitethorn person to log successful with your Google Account if you haven’t already.

You’ll spot that the notebook has a clump of cells with tiny play buttons adjacent to them. Clicking a play fastener executes that peculiar codification block. We’ll click them each 1 by one. If a compartment runs successfully, it’ll crook green. If it fails, it’ll crook red.

When you tally the archetypal cell, it’ll inquire you to link to your Google Drive. A pop-up should look with a confirmation dialog for granting this Colab notebook entree to your Drive.

Next, tally the “Install/Update Automatic1111 repo” and “Requirement” cells. Wait for each to decorativeness moving successfully.

The adjacent measurement is wherever we take the exemplary we’ll provender to Stable Boy. We request a exemplary that supports inpainting, different the generative capable volition make weird representation distortions, not existent content. Click the drop-down paper adjacent to Model Version and take “v1.5 Inpainting.” Then tally the exemplary compartment too.

Leave the remaining 2 cells arsenic they are with their default settings.

Finally, click the Start Stable Diffusion fastener to make the caller API key. The nexus should look thing similar this. If you transcript it oregon unfastened it wrong a browser, it’ll instrumentality you to a web mentation of Stable Diffusion that you tin freely use.

Don’t adjacent this tab until you’re done with your work. Closing the tab besides deletes the nationalist lawsuit we’re moving here.

Now let’s spell backmost to GIMP. Open an representation oregon make a caller papers (the specs of the papers don’t matter). Then click Stable Boy connected the paper barroom and prime “Preferences.” There should beryllium an code barroom for pasting successful an API key. The URL you copied earlier goes here.

Let's Test Our Setup

If each goes well, Stable Boy should beryllium up and moving now. Let’s trial it.

With a blank papers oregon representation open, make a caller transparent furniture called “Inpainting Mask.” It’s case-sensitive and has to beryllium named this exactly, different you’ll get an error.

Then instrumentality a brush, acceptable the palette to black, and overgarment successful the signifier of the entity you privation to generate. Here, I’ve painted a elemental circular silhouette of a ball. It doesn’t person to beryllium exact, but you'll get amended results with much precise masks.

Make definite you enactment connected the transparent “Inpainting Mask” layer. Keeping the masking furniture selected, click Stable Boy and this clip take “Inpainting.”

The Inpainting disguise has a batch of settings and dials you tin fiddle with to alteration the last result. I’ll explicate what they each bash later. For now, let’s conscionable springiness it a substance punctual and property “OK” with the default settings. The punctual could look thing similar this.

A realistic shot shot with a brushed shadiness nether it

The processing takes a fewer seconds sometimes, but erstwhile it’s done, the model volition vanish and the rendered representation volition look connected a abstracted layer. You tin either fell the archetypal disguise furniture oregon delete it.

Here’s different inpainting demo generated based connected the silhouette disguise of a sitting puppy.

a realistic aureate retriever puppy sitting connected grass, beforehand presumption

Some Troubleshooting Tips You'll Probably Need

If, alternatively of an image, you conscionable spot weird clipping, blurring, oregon distortions, it means immoderate of these 3 things successful my experience.

1. GIMP isn’t moving the Stable Boy plugin correctly oregon the Stable Diffusion exemplary you’re moving doesn’t enactment Inpainting.
2. The Inpainting Mask wasn’t acceptable up correctly.
3. You mightiness request to tweak immoderate settings earlier trying again.

If inpainting isn’t working, effort immoderate of the different options Stable Boy offers. If the upscaling, text-to-image generation, oregon image-to-image features are moving arsenic expected, it means the occupation is with the AI exemplary you’re running.

To verify you’re moving the close model, unfastened the API URL successful a browser and entree the Stable Diffusion dashboard. It should accidental “sd-v1-5-inpainting.ckpt” successful the apical close corner. If it doesn’t, that means you’re moving the incorrect model.

To hole it, spell backmost to the Colab notebook, find the compartment containing each the models, and prime the close one. Then repetition the steps you followed earlier and trial inpainting again.

Another mode to trial whether Stable Boy is unrecorded is to click Filters > Python-fu > Console. If nary console appears, it means Python isn’t bundled with this peculiar mentation of GIMP and you request to reinstall the close one. Remember mentation 2.10.34 volition ever work.

Finally, treble cheque that the Inpainting Mask was created properly. The sanction (case-sensitive) should beryllium correct, the disguise should beryllium made with a achromatic brushwood of 100% opacity, and the disguise furniture should beryllium selected.

Here’s my recommended config for getting the champion results.

1. Keep denoising spot astir 75% to 80%
2. Choose Latent Noise for Inpainting Fill
3. Step number should beryllium astir 40.
4. Switch to this sampler if available: DPM++ 2M Karras

---

I got immoderate amazingly bully results with this DIY generative engine. It whitethorn request immoderate fiddling (and the setup tin get frustrating) but I recovered it incredibly rewarding and fun!