The Simple Dilemma
“Keep it simple, stupid” is often repeated and invariably good advice. Nevertheless, it’s easier said than done. The truth is that simplicity is anything but simple.
Despite our best efforts, things seem to get complicated all by themselves. That’s not because we want it that way, but it’s the way the world works and it’s not going to get any easier.
Digital technology is undoubtedly making our world more complex. More powerful computers are ushering in an era of Big Data, while increased connectivity means that everything interacts with everything else, adding further complication. If we want clarity and simplicity, we need more than just platitudes.
The Information Problem
In the late 1940’s, as the post-war era had just begun, the digital world we know today was created at Bell Labs through two landmark breakthroughs. The first and more famous was the invention of the transistor in 1947, which made possible the small, cheap binary circuits that are embedded in every electronic device we know today.
The lessor known, but in many ways no less important, revelation was Claude Shannon’s 1948 paper A Mathematical Theory of Complexity which spawned the field of information theory. It established, among other things, a measurement unit for information – the bit – a binary piece of information.
In other words, information would be defined by a single choice – a coin flip if you will. We now regularly measure information by the amount of bits it takes to describe things ranging from one character (8 bits or a byte), a one page document (about 2 kilobytes,) a full length movie (1-2 gigabytes) or the Library of Congress (235 terabytes).
Not only are we measuring information now, but we are generating and storing massive amounts of it. According to a recent McKinsey report, medium sized companies today commonly store more data than the 235 terabytes that the Library of Congress has.
Information is also getting cheaper. You can store all the music in the world (literally) for about $600, so we’re only going to see more of it.
The Simple and the Complex
Shannon’s insight opened the door to understanding what makes things simple and complex: Simple things are easy to communicate while complex things are difficult.
For instance, a googol is a very large number but simple to describe (i.e. 10100). while 479,001,599 is much smaller, but a prime number that can’t be reduced to anything simpler, so it’s very complex.
That is the essence of Kolmogorov-Chaitin complexity, which defines complexity (and therefore simplicity) by the number of bits (or coin flips) that it takes to describe an object without losing information.
Sometimes, for security, we would rather things be complex, which is why web sites will reject passwords that can be broken down to just a few bits and credit card companies multiply large prime numbers to encrypt transactions. University professors use lots of big words so only other academics can understand them and therefore justify their tenure.
Usually though, we want things to simple, even if that means we lose some information, which is why web videos can be grainy and jump around. We compress information (i.e. simplify it) to make it easier to transmit. That, in large part, is what drives the communications industry today.
Interactivity, Fractals and Chaos
So far so good. However, our simple story remains incomplete. Kolmogorov- Chaitin complexity only refers to single entities. What happens when you have a bunch of them interacting?
That’s the question that Benoit Mandelbrot sought to answer in the early 1960’s. He noticed that the noise created in communication lines followed it’s own kind of order and inferred two types of effects that created it.
Joseph Effects: These are persistent trends. Just like in the biblical story, where Joseph predicted seven fat years and seven lean years, events in a time series are highly dependent on what precedes them.
Noah Effects: These are sudden events that create discontinuity. A storm comes and blows everything away, creating a new fact pattern that will be propagated through further Joseph effects.
It was these insights that led him to pioneer the field of fractal geometry, which can create shapes of infinite intricacy and complexity merely by repeating simple rules. The most famous example, of course, is named after the man himself: the Mandelbrot set pictured below. As you zoom in and out (see link above) new details seem to emerge without end.
Mandelbrot’s fractals were more than just a fancy mathematical parlour trick. In The Blind Watchmaker, biologist Richard Dawkins describes a how interaction between relatively simple things can create the greatest complexity imaginable – life itself. This amazing phenomenon is the essence of the new science of emergence.
In 1980, Loren Carpenter applied Mandelbrot’s ideas to computer graphics. He realized that by repeating simple fractals, he could generate amazingly lifelike scenes, synthetically. His method became so successful that the company he helped found, Pixar Animation Studios, became a Hollywood hit machine.
Carpenter then took the idea a step further and applied the principle to social interactions. If order could emerge without direction, then maybe social action could as well. In the video below, he shows how hundreds of people can coordinate their actions well enough to play the computer game “pong” collectively without leadership or hierarchy.
Later, Duncan Watts and Steven Strogatz discovered that not only that the same forces drive ordinary social networks, but that they follow the same power law rule that Mandelbrot uncovered in his original research into communication lines and that Chris Anderson found to continually pop up in his book The Long Tail.
It doesn’t stop there either. Even in staid management circles, the idea is catching on with Henry Mintzberg leading the charge to challenge decades of corporate planning orthodoxy by advocating greater emphasis on emergent strategy.
Simple Rules For A Complex World
The evidence is clear: There’s nothing simple about simplicity. Our world is getting ever more complex. Computational power is increasing exponentially. Barriers are breaking down. Outcomes are extremely sensitive to initial conditions because events materialize out of unplanned interactions.
Therefore, our goal should not be to seek out ultimate simplicity, but maximum manageability and there are some basic principles that can help us do that.
Factor Down One (or possibly two) Levels: As I wrote in an earlier post, technologies are made up of combinations of other technologies. That applies to not only physical objects, but cultural objects as well, such as democracy, a body of law or even a corporation. So we can always break down a concept to its constituent parts.
However, our sense of purpose can quickly disappear down the rabbit hole if we allow every discussion to devolve into an examination of the components of components’ components.
Factoring a problem down one or (possibly two) levels will explain the vast majority of variables and will allow you to isolate and focus on one or two specific areas of concern. Any more than that and the amount of information you will have to deal with will be unmanageable.
Limit Variables: Centuries ago, medieval scholars gave us Occam’s razor, which urged us to use as few variables as possible.
It’s good advice. As this HBR article points out, less is more. If you have 5 steps in a process, see if you can narrow it down to three. If you have fifty slides in a presentation, try to cut it down to thirty-five or even thirty. Small teams generally perform better than large ones.
They key here is to use the absolute minimum number of entities that you can without losing essential information. When in doubt, leave it out.
Look For Patterns: Although individual events might be unpredictable, the fates usually consort in recognizable patterns. As mentioned above, systems with interaction and feedback tend to result in power law distributions, while independent systems generally follow a bell curve. Making those types of distinctions accurately is incredibly important.
In a similar way, industries tend to have their own rhythms and cycles. If you are an ad agency, for instance, and clients drop surprise holiday campaigns on you every year, then they shouldn’t be such a surprise.
A salient example is Dunbar’s number, which suggests that relationships in groups start to break down once the total number of people goes over 150. In a company, once you grow past that, management practices have to change drastically, just as you need to speak differently when presenting to a group than you do in a one-on-one conversation.
As I noted in the beginning, while things may start out simple, they will inevitably become more complicated over time. So while we might yearn for a simpler era and a simpler life, modern life demands more of us.
The best that we can do is keep entities simple, accept that systems become complex and seek to manage that complexity while understanding that it can not be controlled.