It is now possible to see what a person is looking at by scanning their brain. The technique, published last November by a team of Japanese neuroscientists, uses FMRI to reconstruct a digital image of the picture entering the eye, albeit at very low resolution and only after hundreds of training runs. Still, it’s an awesome development, and many articles covering this research have called it “mind reading” (1, 2, 3, 4, 5). But it really isn’t, and it’s fun to explore what real “mind reading” would imply.

When I hear “mind reading” I want psychic abilities. I want to be able to know what number you’re thinking of, where you were on the night of March 4th, and what you actually think of my souffle. This is the sort of technology that could be badly misused, as the comments on one blog note:

Am I the only one finding this DEEPLY disturbing? It opens the doors to some of the scariest 1984-style total-control future predictions. Imagine you can’t hide your f#&%!ng MIND!

Fortunately, we’re not there yet. Morover, if we did have the technology to read minds, we’d have much bigger societal issues than privacy to deal with. The existence of “mind reading machines” would imply that we possessed good formal models of the human mind, and that is a can of worms.

But back to today. The paper by Yoichi Miyawaki and colleagues describes a technique for exploiting retinotopy, the fact that certain areas of the visual cortex are direct “maps” of the retina. First, a series of 10×10 black and white test images are shown to a someone while their neural activation is recorded by FMRI. The responses to these test images are used to ascertain which areas of the visual cortex correspond to which areas of the subject’s field of vision. When the neural map is complete, it can be read “backwards,” going from neural scanner results to a low resolution representation of whatever the subject is currently looking at.

This is a long way from a tool for the thought-police. First, the algorithm requires training on each new person. Also, an MRI machine is a huge, expensive, complicated piece of machinery which requires the subject to stay very still over a period of minutes — widespread brain scanning is, for the moment, completely out of the question. But most fundamentally, the information recovered is nothing more than what the eye is currently looking at. You might as well just tape a digital camera to the subject’s head. The pictures would be a lot better.

What is it that we imagine for a mind reading machine? Perhaps a printout, in words, of every thought that goes through someone’s mind. But do people really think exclusively in words? What about their emotions, or their unconscious responses, or even the complete set of minor joint aches and temperature sensations all over their body? Or how about a video playback of the events of yesterday evening? Impossible, because that’s not how human memory works. When we think about it carefully, we realize that we have an extremely poor conception of what is actually “in someone’s head.”

Compounding this problem is the fact we can’t even say what’s in our own heads. We think we can, but we can’t. Decades of psychological experiments show that access to the contents of our own minds and the working of our own thought processes is very limited. Consequently, we cannot answer the question “what would a mind-reader read?” through introspection.

This is why, before we could build a mind-reading machine, we would first need formal models of a “mind.” We need the sort of mathematical models that one can manipulate with a computer, because computers will surely be intensely involved in any mind reading technology. If recent developments in linguistics and artificial intelligence research are any guide, these models will be huge, associative, and statistical in nature, nothing like the structured logic we think we possess. For example, Google translates web pages between different languages without using anything like formal grammar models.

In other words, we cannot “read minds” because we have very little idea of how minds might be stored on a computer. This problem is known in AI as “knowledge representation,” and we still know very little about it.

Good formal models of the mind, if possible, are the technological precursor to entire fields of information engineering, and this is why I’m not worried about mind-reading technology per se. We’ll get beneficial things like accurate machine translation and computers that respond to voice queries — no more fighting with software that just doesn’t understand what you want. (Think also of the possibilities for art and expression.) We’ll also get uncomfortable technologies like sickeningly effective advertisements that exploit behavioral quirks we didn’t know we had, and NSA-funded conversation snooping programs that make existing keyword scanners look like the toys that they are. Finally, it would be possible to use accurate human mind models for pure evil: imagine a computer virus that was designed to read your personal files and figure out how best to convince you that the Dictator was beneficent. All of this may sound very far-fetched, but we’re going to build these things if we possibly can: think of how much money Google makes from each percentage point of improvement in ad clickthroughs.

If the Japanese FMRI technique seems positively simplistic in this light, that’s because it is. They have read retinas, not minds. They are extracting a representation we already have abundant experience with: images. Saying that we’ve made a step towards reading minds is ridiculous; Thomas Edison might just as well have claimed to “record thoughts” when he announced the phonograph.

I bother with all of this both because I think science journalism is often done badly, and because I believe that it’s important to get hysterical about the right thiings. One comment posted to a video of the research reads, “this is the beginning of the end of free thought.” Perhaps the continuation of this type of FMRI research really will one day lead to the ability to determine what someone is thinking without invoking their consent, but torture already does that. To me, the ability to represent someone’s thoughts in electronic form has far greater implications than mind-reading per se, and this sort of FMRI research — as impressive as it is — contributes little to that enterprise.

In truth, automated systems already rule the world, because there’s simply too much data for humans to do it on their own. Instant online loan applications necessarily employ software which embodies the policies of the lender. In 2002 there were estimated to be 500,000 surveillance cameras on private property in London, and those screens aren’t watched by humans. It is thought that in the next few years machine-to-machine internet traffic (such as automatic inventory control systems placing orders) will exceed all human generated traffic, and nobody crawls the web by hand. This is not even counting the current generation of “botnets“, dumb but virulent programs that attempt to silently steal CPU cycles on as many computers as possible, in order to do things like send spam or take specific machines off the internet via Distributed Denial of Service attacks. Such criminal behavior is worrying, but the situation would be intractably complex even if everyone on the internet was trying to play nice.

If you take a step back and apply the right metaphors, what you get is an ecosystem of self-directed software agents, which Suarez generically and broadly terms “bots”. These software agents are evolving, not because we’re anywhere near the type of self-replicating self-modifying programs that might be subject to bona-fide survival selection pressure — that’s a lot to ask from a little Perl script written by a day-job programmer at a bank — but because there’s often a strong incentive for us to make them just a little more efficient. Every fraction of a percent pattern that Google can tease out of our click-through habits translates into millions of dollars of revenue; thus, the bots are getting smarter, and there are already far too many of them out there for any one person to understand all their possible interactions.

The point isn’t that software runs the world. Post Y2K-bug, that should be obvious. Rather, Suarez wants us to think seriously about all the traditional machine-intelligence doomsday scenarios, because all of them are entirely possible without that cinematic “I’m sorry Dave” moment. It may be the concept of an alien mind that scares us, but Very Bad Things can happen within an enormous system of stupid but autonomous agents, most of them privately run and completely unknown and uninvestigated. “Consciousness” really isn’t the issue. Rather, we need to recognize that the digital environment has already become every bit as complex, fragile, and necessary as the biological and physical environment.

Suarez suggests the creation of a parallel “dark intenet” which is encrypted, authenticated, and humans only. The idea is that all bots allowed onto this network would have to be open-source and properly vetted. I like the idea, but I’m not sure it will work: individual programs can be vetted, but nobody can predict the behavior of the entire interacting system. I do support better security protocols for the internet, but I think humans are doomed to give up control to software on any new network for exactly the same reason we’ve already given it up on the internet: millions of tiny automated efficiencies end up enormously amplifying our power. What began as merely convenient must inevitably end as world-changing.

For what makes Futurology (capitalization mine) really, actually different than a belief that something momentous will happen in 2012, when the Mayan calendar wraps around? Not a lot, as far as I can tell. And now it turns out that two religious scholars have concluded exactly the same thing, in a 2008 paper in the Journal of Contemporary Religion:

Futurology-as-religion has charismatic leaders, authoritative texts, mystique, and a fairly complete vision of salvation. Futurology is, in effect, a new religious movement (NRM).

Let’s break this down a little further. How will we recognize when the “singularity” occurs? Some accounts speak of a period of “unprecedented technological progress” or an exponential growth in computing power, but we’ve been seeing that for 50 years. Or it is described as a point beyond which change is so rapid that prediction is impossible, but prediction more than a few years into the future is impossible anyway, if for no other reason than the butterfly effect. It’s not that I dispute the core argument that technology will continue to alter humanity in nearly unrecognizable ways. In fact, I find many of the future technologies discussed by the Singularists to be quite plausible, including nanotechnology, better AI, and greatly extended human life — there’s lots of serious research going on in all these fields. And I also suspect that we will continue to live through a period of accelerating technological capability, because it is the nature of technology to build upon itself. But why must there necessarily exist some special point? And why must all of these technological transformation necessarily be, well, good?

Even quantitative exponential growth in computing power (or other measurable human capacities) doesn’t imply a singularity. Exponential growth accelerates endlessly, but not infinitely fast; it has no special points or infinite asymptotes.

The only claim that seems at all concrete, the only thing that might give a definite date to the singularity, is the moment when a machine becomes smarter than a human. Such a machine, it is claimed, could improve on itself in a recursive and accelerating fashion, rapidly exploding up to incomprehensible intelligence levels and coming to rule the universe. Surely, this would change history in Godlike ways.

Except that nobody knows what machine intelligence is. Or how we’d recognize one if we met it. The word “intelligence” suggests that one day the computer would wake up and talk to us (presumably, through IM) but this is mere metaphor. (Kurzweil et al. also speak of replacing neurons with hardware or software to produce a synthetic human brain, but that would be a re-implemented human intelligence.) The phrase “machine consciousness” is even less useful, because we can’t even define the word “consciousness” for humans.

Nobody knows what the words use to describe the singularity actually mean.

If no one can specify criteria for noticing when this singularity has actually occurred, I argue that it doesn’t exist even in a theoretical, conceptual sense. In a practical sense it’s therefore no better than Nostradamus, or 2012, or tea leaves. What’s left in the concept is merely belief: belief that somehow, somewhen, something big and important is going to happen. The End of The World (as we know it.) The Ascent to Paradise. Living forever in the consciousness of the machine. Apocalypse. Salvation.

All hail the prophet Kurzweil.