Note: this post originally appeared in my newsletter, but I was so excited about it that I decided to post it here too. Enjoy!
So my wife is getting a PhD in computer science, which means that she’s on the cutting edge of research into things like language learning and topic models and other techy stuff that I don’t totally understand.
A couple of weeks ago, she downloaded Stable Diffusion, an open source text-to-image program that creates AI art, kind of like Dall-E and Midjourney. Besides playing with it herself, she thought it might be useful for me to create my own cover art. So for the last two or three days, I’ve been playing around with it, and the results are absolutely amazing!
These were some of my first attempts. I’ve forgotten what the prompt was: I think it was something like “a spunky young woman with short black hair, surrounded by stars, in the style of Frank Frazetta and Minerva Teichert.” The difference between the first one and the second one was adding “in space.”
I also tried inputting a couple of paragraphs straight from my novel, including a lengthy description of this character, but the results were… uncanny. These AI art programs tend to do better if you give them short descriptions with only a handful of details.
The next day, I played around with it some more, and came up with this one:
The secret sauce for this one was adding “Minerva Teichert” and “Baen Books.” Who is Minerva Teichert? She’s a famous Latter-day Saint painter from the early 20th century who paid for her son’s tuition to Brigham Young University in original paintings, many of which are still on display in the BYU Museum of Art and the Joseph Smith Memorial building.
As you can see, there are still some weird artifacts to this piece, such as the stars on the character’s jacket. That’s the tricky part with AI art: if you look at it closely, you’ll find something weirdly uncanny, like a hand with seven fingers, or a person with three arms. The steepest part of the learning curve has to do with removing these uncanny bits, either by giving better starting prompts, or by tweaking it in subsequent iterations.
I believe the prompt for this one was “a dreamy young woman with short black hair, bare shoulders, in space surrounded by stars and galaxies. Minerva Teichert and Baen Books.” The original image was a woman in a space suit, but I used something called “image to image” to create new images based on the previous one, in batches of four. I would pick what I thought was the best one for that generation, and run the program again. That’s how I eventually got to this one:
and this one:
Still need to work on the hands. Also, there’s this weird artifact, almost like poor JPEG compression, that happens if you don’t give the program enough creative leeway with each successive generation. Another method I’ve heard of is to create a really large batch based on a given image, and then use GIMP to cut and paste all the pieces that you like from each one, before running it through one final image to image pass to seamlessly combine them.
A lot of people are either really angry or really scared about AI art and what it means for the future. It’s the same with other forms of automation, I guess. Will it replace artists entirely? Will all our art be 100% AI-generated in the future? Personally, I don’t think so. These programs are just another set of tools, and require quite a bit of practice to master.
Same thing with stuff like ChatGPT and other language learning models that can be used to write poems and stories. It takes a lot of work to come up with an AI-generated story that isn’t totally boring, or has a terrible ending. It can be done, but it does require quite a bit of human input.
So I don’t see these tools replacing artists or writers, at least in the forseeable future. Rather, I think that the successful artists and writers will be the ones who incorporate these tools into their workflow, using them as force-multipliers to make some really amazing stuff. Personally, I would absolutel love it if I could use something like ChatGPT to put out a new novel every month, or even every week.
The other thing with things like novels is that most people only read them once, because they already know what’s going to happen. So if you use an AI to write a novel, but you have to feed it all the twists and plot points… what’s the point? You’ve basically already read it. This is a problem that a lot of amateur writers have with outlining: since they already know how the story is going to end, they find it difficult to sit down and write.
Now, what I could see is a prompt like “rewrite Lord of the Rings so that Sauron wins,” or “rewrite such-and-such romance novel so that this other guy ends up with the girl.” Or “make Lord of the Rings a gritty cyberpunk novel,” or… you get the picture. And honestly, I’m fine with that. If someone who enjoyed the “alpha” version wants to create a “beta” or a “gamma” version for fun, that’s cool. It might be kind of fun to see how an AI tweaks my books.
What isn’t cool is if someone takes that beta or gamma version of my novel and tries to sell it under their own name. And that’s where most of the legal stuff needs to be hammered out, over issues like copyright. I’m not going to use Stable Diffusion to remove watermarks, or to take someone else’s copyrighted art so that I can enjoy a derivative product without having to pay the artist. And when it comes to using prompts, I’m going to err on the side of using artists like Minerva Teichert who have already passed away, or large publishing houses like Baen whose style doesn’t belong to a single artist.
So after playing around with it some more, I finally came up with some concept art for my current novel WIP and used it to throw a cover together! What do you think? This isn’t going to be the final version—in fact, I will probably produce quite a few other test covers before I settle on the one I like. But for my current skill level (still beginner), I’m quite pleased with how it turned out!
1 comment