In truth, automated systems already rule the world, because there’s simply too much data for humans to do it on their own. Instant online loan applications necessarily employ software which embodies the policies of the lender. In 2002 there were estimated to be 500,000 surveillance cameras on private property in London, and those screens aren’t watched by humans. It is thought that in the next few years machine-to-machine internet traffic (such as automatic inventory control systems placing orders) will exceed all human generated traffic, and nobody crawls the web by hand. This is not even counting the current generation of “botnets“, dumb but virulent programs that attempt to silently steal CPU cycles on as many computers as possible, in order to do things like send spam or take specific machines off the internet via Distributed Denial of Service attacks. Such criminal behavior is worrying, but the situation would be intractably complex even if everyone on the internet was trying to play nice.

If you take a step back and apply the right metaphors, what you get is an ecosystem of self-directed software agents, which Suarez generically and broadly terms “bots”. These software agents are evolving, not because we’re anywhere near the type of self-replicating self-modifying programs that might be subject to bona-fide survival selection pressure — that’s a lot to ask from a little Perl script written by a day-job programmer at a bank — but because there’s often a strong incentive for us to make them just a little more efficient. Every fraction of a percent pattern that Google can tease out of our click-through habits translates into millions of dollars of revenue; thus, the bots are getting smarter, and there are already far too many of them out there for any one person to understand all their possible interactions.

The point isn’t that software runs the world. Post Y2K-bug, that should be obvious. Rather, Suarez wants us to think seriously about all the traditional machine-intelligence doomsday scenarios, because all of them are entirely possible without that cinematic “I’m sorry Dave” moment. It may be the concept of an alien mind that scares us, but Very Bad Things can happen within an enormous system of stupid but autonomous agents, most of them privately run and completely unknown and uninvestigated. “Consciousness” really isn’t the issue. Rather, we need to recognize that the digital environment has already become every bit as complex, fragile, and necessary as the biological and physical environment.

Suarez suggests the creation of a parallel “dark intenet” which is encrypted, authenticated, and humans only. The idea is that all bots allowed onto this network would have to be open-source and properly vetted. I like the idea, but I’m not sure it will work: individual programs can be vetted, but nobody can predict the behavior of the entire interacting system. I do support better security protocols for the internet, but I think humans are doomed to give up control to software on any new network for exactly the same reason we’ve already given it up on the internet: millions of tiny automated efficiencies end up enormously amplifying our power. What began as merely convenient must inevitably end as world-changing.

For what makes Futurology (capitalization mine) really, actually different than a belief that something momentous will happen in 2012, when the Mayan calendar wraps around? Not a lot, as far as I can tell. And now it turns out that two religious scholars have concluded exactly the same thing, in a 2008 paper in the Journal of Contemporary Religion:

Futurology-as-religion has charismatic leaders, authoritative texts, mystique, and a fairly complete vision of salvation. Futurology is, in effect, a new religious movement (NRM).

Let’s break this down a little further. How will we recognize when the “singularity” occurs? Some accounts speak of a period of “unprecedented technological progress” or an exponential growth in computing power, but we’ve been seeing that for 50 years. Or it is described as a point beyond which change is so rapid that prediction is impossible, but prediction more than a few years into the future is impossible anyway, if for no other reason than the butterfly effect. It’s not that I dispute the core argument that technology will continue to alter humanity in nearly unrecognizable ways. In fact, I find many of the future technologies discussed by the Singularists to be quite plausible, including nanotechnology, better AI, and greatly extended human life — there’s lots of serious research going on in all these fields. And I also suspect that we will continue to live through a period of accelerating technological capability, because it is the nature of technology to build upon itself. But why must there necessarily exist some special point? And why must all of these technological transformation necessarily be, well, good?

Even quantitative exponential growth in computing power (or other measurable human capacities) doesn’t imply a singularity. Exponential growth accelerates endlessly, but not infinitely fast; it has no special points or infinite asymptotes.

The only claim that seems at all concrete, the only thing that might give a definite date to the singularity, is the moment when a machine becomes smarter than a human. Such a machine, it is claimed, could improve on itself in a recursive and accelerating fashion, rapidly exploding up to incomprehensible intelligence levels and coming to rule the universe. Surely, this would change history in Godlike ways.

Except that nobody knows what machine intelligence is. Or how we’d recognize one if we met it. The word “intelligence” suggests that one day the computer would wake up and talk to us (presumably, through IM) but this is mere metaphor. (Kurzweil et al. also speak of replacing neurons with hardware or software to produce a synthetic human brain, but that would be a re-implemented human intelligence.) The phrase “machine consciousness” is even less useful, because we can’t even define the word “consciousness” for humans.

Nobody knows what the words use to describe the singularity actually mean.

If no one can specify criteria for noticing when this singularity has actually occurred, I argue that it doesn’t exist even in a theoretical, conceptual sense. In a practical sense it’s therefore no better than Nostradamus, or 2012, or tea leaves. What’s left in the concept is merely belief: belief that somehow, somewhen, something big and important is going to happen. The End of The World (as we know it.) The Ascent to Paradise. Living forever in the consciousness of the machine. Apocalypse. Salvation.

All hail the prophet Kurzweil.