Category: Uncategorized

Will super-intelligent AI take over the world?

I’ve been reading a lot of non-fiction books about AI recently. Basically, whenever a nonfiction audiobook that has anything to do with AI comes into my audiobook library app, I jump on the waiting list and listen to it right away. I’ve also been following AI news podcasts and watching lots of YouTube channels that discuss the recent developments… and boy, is there a lot of doom porn out there.

People who are closely watching this stuff believe that AGI (Artificial General Intelligence) is imminent, ie within the next 6 to 72 months, and that when AGI gets mainstreamed, it will either usher in a golden age of post-scarcity, or the ultimate extinction of all mankind (or both, weirdly). The main crux of their thesis is that once we achieve an AGI that can rewrite its own code, it will quickly turn into a superintelligence, and then it will either work to serve humanity or else work to eliminate humanity as a threat, either by outright exterminating us, or putting us into some kind of zoo.

This is all very science fictional stuff—but now more than ever, we are living in a science fictional world. So what is actually going to happen? Do I believe we going to enter the singularity, and give birth to a new species of superintelligent AI that will ultimately replace us? Or, in the lingo of Silicon Valley, what is my P(doom)?

TL;DR: I have two P(doom) values, one of which is 0%, the other of which is 90%. My P(doom) for basically all of the scenarios that involve a runaway superintelligence is 0%, but my P(doom) for massive catastrophic social upheaval due to the disruptive nature of AI technology is 90%.

For the last century or so (basically ever since Turing’s work during WWII), the field of artificial intelligence has followed a cyclical pattern. First, researchers make some sort of breakthrough, which leads to rapid technological advancements and a brief AI boom. During this boom, futurists and technologists rave about how this technology will keep scaling up forever until it ushers in a sci-fi utopia/dystopia and utterly changes what it means to be human. Then, the technological development stalls as researchers run up against a hard barrier that makes further scaling impossible, at which point most of the investors sours on the technology and we fall into an “AI winter” for a decade or two.

The problem with the futurists and technologists who promote AI technology is that the vast majority of them are transhumanists who believe that intelligence is purely an emergent phenomenon that is 100% materialistic in nature. In other words, they believe that the human mind is little more than an organic machine created through the process of evolution, and that 100% of our intelligence, emotions, spirituality, and experience can be explained and understood through purely material processes. Therefore, if they can build a machine that replicates the same biological processes as the human brain, and subject it to similar conditions that evolution subjected us to, intelligence will naturally emerge from such processes and conditions.

But what if they’re wrong? What if there are more things in heaven and in earth than are dreamed up in our modern philosophies? I’m not saying that evolution didn’t play a role in the creation/emergence of intelligence—only that it’s insufficient. And why wouldn’t it be? Science, by definition, can only explain what it can measure. And what about the questions that we can’t ask? The things about this universe that are as foreign to our own understanding as quantum physics is to a German Shepherd?

For these reasons, I do not think that these generative AI models are going to keep scaling upward until we achieve a general superintelligence. At some point in the next 0-18 months, I think that the researchers and developers are going to start hitting hard limits that we don’t understand, because of the limitations of our understanding of the human brain and how our own intelligence emerged or was created.

I am extremely skeptical of all of the doom porn floating around out there, that we are months away from achieving AGI, and that a superintelligence will shortly thereafter replace us as the dominant species on this planet. For one thing, the goalposts for AGI are constantly moving—by the standards two or three decades ago, we have already achieved it—and for another, the transhumanists have turned this concept of AGI into a sort of Messianic savior / world-ending destroyer. And I just don’t buy into that religion.

So if I’m right, all of this doom porn about a world-ending superintelligence is utterly misguided. Which, on a certain level, is somewhat comforting. But on the other hand, that also means that we shouldn’t expect AI to save us—and that anyone who tries to tell us otherwise is ultimately trying to sell us something.

The big AI developers like OpenAI, Anthropic, etc. have every incentive to hype up the doom porn. It makes them look powerful, which in turn attracts investment capital. At the same time, they also have every incentive to promote this idea that a superintelligent AI can be our savior, since if AGI is inevitable, shouldn’t we put everything we have into making sure that our AI overlords are benevolent and have humanity’s interest at heart? But again, if we take that view, we also end up pumping lots of investment capital into these AI companies, turning them into massive cultural behemoths without really questioning their ultimate aims.

What if instead of building a superintelligent AI savior, we ultimately end up with a new form of techno-feudalism, powered by AI? What if a true superintelligence never emerges, and all of the energy and resources we’re pumping into AI is really just going to create a new class of elites, with the rest of us dependent on some sort of universal basic income and totally at the mercy of the owners, controllers, and operators of AI?

To me, this seems like a much more likely scenario—and from what I can tell, we are already in the opening phases of it. Generative AI has already become so powerful that it will likely replace a large number of jobs or render them obsolete—which may or may not be a problem in the medium- to long-term, but will certainly be a problem in the short-term. As increasing numbers of people find themselves unemployed, it will put a tremendous strain on our welfare safety nets, and drive calls for increased government spending on social problems. But our governments are already so deep in debt that these pressures can only lead to some combination of (hyper)inflation, soveriegn debt crisis, and austerity-driven political instability.

Some people think that the solution to all of this is a universal basic income (UBI). But every time a UBI has been introduced, it has always led to negative outcomes, including wealth outcomes. Unfortunately, if AI is truly going to be a huge driver of unemployment (which doesn’t require AGI or a superintelligence—our current models are already powerful enough to drive massive disruption in the labor market), then I don’t see how we can avoid a massive push toward UBI. Certainly not with how our current investments in AI are so centralized—but again, all of the AGI doom-porn is driving us to centralize things even more. So while all of the benefits of this new technology accrue to Sam Altman, Elon Musk, Dario Amodei, etc, and they keep holding out the promise of a messianic superintelligent AI that never truly emerges, the rest of us end up in a world where we have very little agency or control over our lives, with or without a UBI.

It doesn’t have to be this way. But if we all keep buying into the doom porn without looking critically at these AI companies and their transhumanist messianic promises, I think that this is the future we’re most likely going to get.

Restored footage from just after WWI

I don’t know how the YouTube algorithm decides what to show me, but every once and a while something really fascinating shows up in my recommends. This was one of those times. Really excellent job restoring this old footage.

“Every age seems to spawn a leader…”

Every age fraught with discord and danger seems to spawn a leader meant only for that age, a political giant whose absence, in retrospect, seems inconceivable when the history of that age is written. —Dan Simmons, The Fall of Hyperion.

How I Hacked My ADHD to Read 5-15 Books Every Month

For a long time, I wanted to read more books. Year after year, I would set a reading goal, only to fail miserably.

I have a mild case of ADHD, which makes it very difficult to focus exclusively on a monotonous task for longer than about fifteen minutes (or alternately, makes it difficult to notice anything else when I’m in a state of hyperfocus). Because most novels take around 8-10 hours to read, it was usually my ADHD that made it difficult to finish any of them.

Then on year, I set a resolution to read or DNF (“did not finish”) every novel that has ever won a Hugo or a Nebula award. I knew that it would be difficult, so I went in with a plan. Long story short, the plan worked out amazingly well, and by the end of the year I had read (or DNFed) nearly 150 books.

But I didn’t stop there. After accomplishing the reading goal, I kept up with the same plan, tweaking it here and there until it became the main process by which I read books. And it still works amazingly well, as you can see from my current stats for this year:

  • January: 11 books read, 6 DNFed
  • February: 8 books read, 2 DNFed
  • March: 15 books read, 6 DNFed
  • April: 6 books read, 6 DNFed
  • May: 10 books read, 5 DNFed

So how does it work? Basically, by hacking my ADHD to turn it into an asset instead of a liability. Here is what I do:

1. Read lots of books simultaneously

This is the main principle that drives my reading process. Instead of trying to work against my ADHD and force myself to focus on the same book all the time, I keep a pile of books that I’m currently reading, and cycle through them. Whenever I get bored of my current book, I put it down and allow myself to become distracted with the next book. In this way, even though I’m constantly getting distracted, I’m also constantly reading, since the distractions are just other books.

If you don’t have ADHD, this might sound like it’s a little maddening—and for normal people, it probably is. But one of the nice things I’ve found about ADHD is that it really expands my capacity to hold multiple thoughts or ideas in my head at the same time. Yes, my mind is constantly bouncing around between all of them, but because I have enough room to hold them all, it’s actually not that hard to read, say, a dozen books simultaneously and remember what’s going on in each of them. I just have to make sure that I don’t let too much time slip by between the last time I pick it up.

Which leads to…

2. Keep a spreadsheet to measure daily progress

Because ADHD can really hamper my executive function, I try to simplify and automate as much as I can. For reading, that means tracking my progress on a spreadsheet, so that I don’t need to keep any of that in my head. Instead, when the time comes to restack my currently-reading pile, I just check the spreadsheet and stack them in the order that it tells me.

As an added bonus, seeing the numbers on the spreadsheet go up over time gives me a lot of motivation to keep reading. And when I’m in a place of low motivation, the spreadsheet helps me to pull back and reorder things, putting the short, easy books at the top and pulling the hard, longer books out of what I’m currently reading, to pick up later. Because it’s all tracked, when the time comes to pick up a book again after setting it aside for a few months, the spreadsheet helps me to do that quickly.

3. Have dedicated reading time

Another huge thing that helps with the executive function issue is keeping to a routine that includes some reading time, so that I don’t have to think about reading—I just do it. Personally, I’ve found that the best time for this is at night, shortly after putting the kids to bed. I’ll usually read through about half a dozen books before I become sleepy enough that it’s time to turn off the light.

I’ll admit, I’m not always great about keeping to this routine, but thankfully it’s the sort of thing that you can pick up easily after missing for a couple of days. It also helps that my wife usually likes to read in bed with me at the same time.

4. Start a new book almost every day

The thing I’ve found with ADHD is that it really makes me crave novelty. So whenever I feel like my reading habits are flagging, one thing that usually helps is to pick up a new book and put it on the top of the pile. After reading the first ten or so pages of a new book, that’s usually enough to put me into reading mode, and then I’ll devour the rest of the pile.

5. DNF early and often

Of course, if I’m starting a new book almost every day, that’s a recipe for getting buried in books really quickly! So to counteract that, and give myself room to experience more novelty in my reading life, I don’t force myself to finish every book that I start. In fact, there are some months where it seems I DNF more books than I finish! But that’s okay, because it makes room for the really good books. And honestly, there are so many books in the world that it just doesn’t make sense to spend a lot of time on the mediocre or terrible ones.

So that’s how I do it. How about you? What are some hacks that you’ve found that help to read more books?

Fantasy from A to Z: G is for Gemmell

I love Robert E. Howard and J.R.R. Tolkien, but my favorite fantasy writer of all time is David Gemmell.

David Gemmell had a rough life. He was born and raised in a lower-class region of the UK to a single mother and an absent father. According to his bio, he was kicked out of school for setting up a “gambling syndicate” on the playground, whatever that’s supposed to mean, and eventually came to support himself by becoming a bouncer. Looking for something a little more stable, he eventually became a writer for the local newspaper, though from what I understand he was ultimately fired from that job as well.

His fiction writing career began when he went to get some nagging ache checked out by a doctor, and learned that he had terminal cancer. They estimated that he had only months to live, and scheduled a follow-up visit about two weeks later to find out exactly how much time he had left. Stunned, he decided to chase his childhood dream of writing a fantasy novel, and Legend was the result.

Legend is an absolute masterpiece, not because the prose is perfect or the story is totally original or the worldbuilding is super deep, but because it has so much heart. It poses a question that Gemmell found extremely urgent: what is it that makes life worth living? And then, unlike many more flowery and pretentious novels, it answers the question with raw, direct honesty: the thing that makes life worth living is to give it up for a worthy cause.

The story follows a cast of unlikely characters who are all drawn to a hopeless siege that everyone knows will ultimately result in defeat. The Khan has united the tribes and amassed an army of half a million soldiers, but to invade the southern kingdom he must lead them through a narrow pass that is held by an ancient fortress. However, the fortress is woefully understaffed, with only ten thousand defenders, most of whom are untrained farmers. Everyone who goes to fight there knows that they will die.

The story is about why they decide to fight anyway. There’s the proud daughter of the ailing duke commanding the fortress, who fights to preserve her family’s honor. There’s the cowardly berserker who never really wanted to get drawn into the siege, but stays to protect the girl. There’s the Temple of the Thirty, an ancient order of warrior monks who train relentlessly in the martial arts so that when the time comes to fight in the defining conflict of the age, they are ready to fight for the good, the true, and the beautiful. And there is Druss the Legend, an aging warrior who has wandered the land and single-handedly turned the tides of battles, but now fears growing old and senile, and desires more than anything else to go out of this life on his own two feet with his battle-ax in hand.

David Gemmell wrote this novel in the two weeks between his first appointment with the doctor and the second. But when he went back in, the oncologist informed him that the first test was actually a false positive, and that he had no cancer at all. Stunned for a second time, Gemmell took a critical look at the novel he’d written and stuffed it in his trunk, convinced that it wasn’t very good. After all, who was he to think that he could write a novel?

The story would have ended there, except that a friend of his found out about it, asked to read it, and was so impressed by it that he urged David Gemmell to send it off to a publisher. After a lot of nagging, Gemmell eventually decided to humor his friend, and the book became a massive bestseller over in the UK. David Gemmell went on to write some two dozen fantasy novels, all of them in the same vein as Legend, and they are absolutely fantastic. 

I still remember the sinking feeling in my heart when I read the last book in the Drenai Saga, and realized that there wouldn’t be any more. And I also remember the way my mind was blown when I realized that all of Gemmell’s novels are interconnected in an interdimensional “cosmere” of sorts, with a handful of recurring characters who travel across worlds. Yes, he was doing the Sanderson Cosmere thing before Brandon Sanderson published his first book (and unlike Sanderson, he kept it as a genuine easter egg and never advertised it). 

I’ve collected nearly all of Gemmell’s books, including the crime thrillers he originally published under a secret pen name, and both of the graphic novels that go for a couple hundred bucks. Most of his books I own in mass-market paperback, though I still hope to acquire a signed first-edition hardback copy of Legend. But I’m still dragging my feet to read them all, because it really is an awful feeling when you get to the end of your favorite author’s ouvre and realize that’s all there will ever be.

David Gemmell died in the 00s, in the same way that I hope to die: sitting at his keyboard, writing the last book in his final book series. His wife later finished it. From what I understand, he actually did die of cancer, so maybe there was something to that original mis-diagnosis that kick-started his writing career. Can you imagine how different things would be if that had never happened? I don’t even want to think about it. David Gemmell’s books are amazing, and the world is so much richer because of them. If David Gemmell’s books are new to you, you’re in for a real treat!

Fantasy from A to Z: All Posts

“What is a year?”

A year? What is a year? All time is relative. One day may be a lifetime, a year can be forever. It is not the number of days, but what goes into those days. —Louis L’Amour, The Warrior’s Path.