On Combinatorics and Zero to One

I've started reading Peter Thiel and Blake Masters' book Zero to One. I can't give a comprehensive review yet, but I can say that I agree completely on one of the book's most contentious allegations: that technology more or less stalled around 1970.

Debate is hot on this one. Here's a decent thread over at Hacker News. In a nutshell, Thiel argues that information technology has been one of the only exceptions to technological stagnation, and that the explosion in IT has masked a more general lack of progress in other areas.

Critical to understanding this debate is answering the question of what exactly Mr. Thiel is talking about.

Since 1970, cars and planes have gotten safer and more efficient. Virtually every manufactured good has grown more affordable, sometimes radically so. Medicine is better at treating severe conditions like cancer and heart disease. We have new vaccines for things like chicken pox and even some strains of influenza. We've been landing larger and progressively more capable space probes on Mars, and are about to try to touch one down on a comet. We've got a space station the size of a football field and it's been continuously staffed for over a decade.

So is Peter off his rocker? I don't think so. All the things listed above are innovations, but they're incremental ones. Linear improvement has continued post-1970 in almost every area. It's the "breakthrough" that has become rare.

What Is a Breakthrough?

So what is this mythical beast? Can it be defined? If you read the debates that have sprung up across the Internet quite a few people -- the ones who think Peter is de-rockered and without hinge -- seem to more or less deny its existence.

I think they're wrong. I think it can be defined, and I'm going to take a shot at it.

That there is called a fitness landscape. Click the image to visit its page of origin, and do a Google image search for more examples of what they can look like.

Imagine an "organism" whose genetic code contains only two integer values that can vary from zero to ten. To generate a fitness landscape for that organism you'd enumerate all combinations of those values and for each measure the average Darwinian fitness of the resulting phenotype. Those fitness values would be the Z axis in a 3d surface plot that would look much like the one above.

Things that simple can be found in toy genetic programming examples, but real world fitness landscapes are absolutely huge and probably non-computable over the age of the universe. (If they were computable, evolution as a long drawn out process might not exist.) The human genome has about 3,234,000,000 base pairs that can vary among four values, yielding a three-billion-dimensional hyper-landscape of width four. That can't be visualized as anything other than a blob of fuzz and surely can't be computed. Real world fitness landscapes also tend to be dynamic if the environment is changing. They can even be recursively dynamic if they result in organisms that themselves change the environment.

But conceptually they all -- at any given point in time -- "look like" the surface plots you'll find on Google Images. Like those plots, real world fitness landscapes have varying and complex structures. The nature of that struture is what's important here.

Take a look at the plot on the right above and what do you see? There are many peaks. Some are connected to one another by slopes that go up. Others are connected by slopes that go down, then up. For many there's more than one path from peak to peak. Some are completely isolated.

Now imagine you're going to go hiking, and think about the implications of all these different landscape features. In some cases you reach a peak and can proceed directly to the next higher peak. In other cases you must go down -- you must cross a fitness valley -- to reach a higher elevation. Finally there are those peaks that aren't reachable at all without a descent to the "floor."

If our hiker is an evolving genetic lineage, this presents a problem: some peaks can't be reached without descending (which is obviously risky to one's survival), and in some cases the descent might be so far that it constitutes non-viability (death). When a fetus is stillborn due to a genetic abnormality, that's what happened -- the recombination of gametes produced a vector off in deep blue territory.

Luckily evolution has a few tricks up its sleeve, like recombination. By recombining alleles from different parents, it becomes possible for evolution to make "leaps" that cross fitness valleys.

There's a whole sub-section of evolutionary theory that studies the evolution of evolvability, which basically means the production by evolution of new algorithmic hacks for efficiently traversing real-world fitness landscapes. Evolution "learns how to learn." This gets into a lot of interesting areas that are out of scope here, but if you're on the hunt for an intellectual rabbit hole with a lot of this you can't go wrong with evolutionary dynamics.

But back to breakthroughs.

Some biologists refer to our technology as our "extended phenotype." Its blueprints aren't stored in DNA but are instead carried by cultural information storage and transfer mediums like books, direct imitation, or the one you're reading now. But like DNA, technology is encoded in information and these sequences of information could be thought of as points in a very large hyperdimensional state space. Like organisms, individual artifacts of technology can be said to possess differing fitness. This might be defined by reference to human biological success, subjective perception of quality of life, or market success, but there's certainly a better and a worse. Technological state space has structure.

What do cars with higher MPG ratings, bigger and cheaper flat panel LCDs, and faster Internet connections have in common? The answer is that they're all examples of simple linear ascent.

If some combination of Shannon's theorem and quantum mechanics gives us a theoretical maximum for how fast information can be accurately transmitted over a twisted pair of copper wires, that represents a fitness peak. Going from Morse code telegraph to ten gigabit Ethernet represents a linear climb from a lower fitness to a higher fitness with each step along the way being incrementally better than the last.

Peter calls this "one to N" innovation. I call it hill climbing. It's low risk, and it's easy to see why. Start where you are. Evaluate nearest neighbors. Proceed upward. No valleys are crossed, steps are often of similar difficulty, and the outcome is predictable.

Yet if technological state space is anything like biological state space, it likely contains a variety of structural motifs.

It's almost certainly full of peaks that cannot be reached by mere gradient ascent. A breakthrough is when one lands near one of these via some other exploratory strategy.

The late nineteenth and early twentieth century is full of innovations that -- at least as far as I can see -- are clearly examples of this. The first transmission of information by radio had no clear technological antecedant, nor did the alternating current transformer or the solid state laser.

To argue that breakthroughs don't exist or are qualitatively identical to large incremental changes is to argue that the fitness landscapes inhabited by our technological artifacts all look like this:

I consider that rather unlikely.

So What Happened Around 1970?

By definining a breakthrough as a non-incremental transition through cultural-evolutionary state space, I can also put forward a very high level conceptual hypothesis for what went wrong around the time Nixon cancelled Apollo.

Recall our brief digression into evolutionary learning theory. Evolution isn't a single process. Instead, it's an overlapping composition of multiple learning techniques: simple mutation and selection, sexual recombination, dynamic regulation of mutation rates in response to stress, and possibly weirder things that you'll learn about if you Google "evolution of evolvability" and start reading the literature on the subject.

The point is that if all evolution did were mutation, asexual reprodution, and selection, any adaptation that didn't involve a simple linear hill climb would be fantastically unlikely.

Maybe somewhere around the time the Beatles broke up, our set of search algorithms underwent a diversity collapse.

For some reason we were able to make numerous leaps through techno-evolutionary space from roughly the renaissance to half-way through the presidency of Richard Nixon, with what looks like a peak from roughly 1945 until 1969. Then something happened to us culturally, economically, intellectually, or maybe all of the above, and our collection of methods for exploring the space of the technically possible collapsed down to a couple variations on the idea that you get higher by going up.

What About IT?

Why has IT been an apparent exception?

The answer might be something very pragmatic: it's an area where experimentation is cheap and where there's little regulation to limit tinkering. Computer science is one of the only major areas of science where a single individual can make an important contribution in their spare time using tools that cost less than $1000.

If this speculative collapse in learning algorithm diversity was a social and political phenomenon, you'd expect it to hit large organizations and bureaucracies pretty hard. Humans in large numbers can get very herd-like and tribal. Yet here and there in the cracks and crevices there'd be plenty of individuals and small groups who didn't get the memo.

Other bigger areas of science and technology are too expensive or too dangerous for free-wheeling permission-free innovation. To raise a lot of money you've got to work within some big political frameworks, and as technophilic and future-crazed as I am I'm not sure I want my neighbors playing with nuclear fission in their basements.

Yet I'd also question whether IT has been as much of an exception as some people think. There seems to have been more fundamental "zero to one" innovation in IT than other areas, but if you know history you know a lot of what we're doing now was invented in the 60s. A whole lot of the IT mega-boom has been incremental improvements to that stuff. Some of the gains have been very large, even exponential, but it's still gradient ascent from a learning strategy point of view.

The One True Way

So what happened... in more concrete terms?

I don't know, but I've got some ideas. My leading hypothesis for the cause of this collapse in meta-technique revolves around what I see as a dramatic rise in fundamentalist styles of thought and discourse over roughly the same time frame that Peter (and I) see a decline in zero-to-oneness.

By fundamentalism I don't just mean religion, though fundamentalist Christianity and Islam are probably the biggest manifestations. I also see highly dogmatic forms of secular capital-S Skepticism and some varieties of evangelical Atheism as essentially fundamentalist, not to mention all our fist-pounding political ideologies and social movements.

What I'm really getting at here is a shift in the way people think -- not a change in belief but in cognitive style. What I observe (at least in the West) since 1970 is a huge shift toward belief in a One True Way of Thinking that is absolutely infalliable and applicable to every situation.

By following the One True Way one arrives at the One Truth, and then stops. Sounds a bit like a greedy hill climber reaching a local maximum to me.

Might the fundamentalist cognitive style not imply fixation upon a single learning strategy?

I'm less confident in this than I am in the more abstract hypothesis above, but if I were a betting man it's where I'd place my wager.

Getting our Mojo Back

We should go back to the Moon.

No, seriously. That's my solution. We humans are complex beings whose motivations have as much to do with myth and symbolism as with fact and material desire, and we love a good plot twist.

So how about this one. We go back to the Moon, and this event marks the return of broad-spectrum multi-paradigm thought and bold inventive leaps. The cancellation of Apollo marks the beginning of a minor dark age, and the return to the Lunar surface marks its end.

We should go to Mars too, and eventually to stay, but somehow I think completing the Lunar circle is the most metaphorically appropriate way to break the spell.

Show Comments