Why Math Matters
When media first started going digital, journalists feared that they would be judged by the numbers, rather than on the mastery of their craft. To a large extent, that actually happened, but it wasn’t so bad after all. Most were able to adjust.
So there was no small amount of irony surrounding the recent high profile negotiations between ESPN and The New York Times over the services of Nate Silver. The stakes were enormous, as Mr. Silver accounted for as much as 20% of the Grey Lady’s web traffic.
Math, it seems, has become more than a means to evaluate, but journalism itself. Whereas before, a reporter’s main tools were sources and shoe leather, now algorithms are on the job too. Most of all, Nate Silver is a sign of the zeitgeist. In this new age of big data, math is no longer an ancillary activity, but plays a central role in helping organizations compete.
A 21st Century Approach To Talent
I’ve worked with a lot of very capable journalists, but creative people are a different breed. While world-class journalists are intelligent and determined, artistic talent has an intangible element, something you can’t describe with any precision.
A lot of professionals make their living by being able to spot creative genius amongst the thousands of journeyman hopefuls. In the recording industry these people work in A&R, in Hollywood, they are often development executives and in other places they are merely called talent scouts.
Today, however, machines are evaluating creativity as well. Record labels use software called Music X-Ray to judge the whether a new track is likely to be a hit or a flop. Mike McCready, its creator, has even been the subject of a HBR case study. Another company, Epagogix, uses a similar process to evaluate screenplays for Hollywood studios.
So while legendary talent spotters like Clive Davis continue to thrive, creative businesses are increasingly using math to improve their chances. An expert is no match for an expert armed with an algorithm.
Ria Persad is one of those people born with extraordinary gifts. She started her research at the Jet Propulsion Laboratory when she was 12, began taking courses at Harvard at 14 and worked at the legendary Institute for Advanced Study, before the age of 20. She also performs as a concert pianist under her married name, Ria Carlo.
In an earlier age, she would most likely be relegated to academia, where she would spend her days writing arcane formulas on a blackboard that only a handful of people in the world could understand. Today, however, she runs a thriving business, Statweather, which consults for the energy trading industry.
Persad compares her work to how algorithmic trading has replaced intuition on Wall Street and says, “The program—not a human—does the pattern recognition, the bias and error adjustments of models, the tracking, and makes the “judgment call” based upon the most dynamically optimal combination of models.”
In other words, she’s not like your friendly neighborhood weatherman trying to predict if you should take your umbrella with you today or not, but uses over a century of data to determine weather patterns months in advance and her accuracy is 74%, nearly double what conventional methods achieve.
She also notes that, much like journalists at the dawn of the digital age, traditional meteorologists see her automated methods as a threat. No surprise there.
Modeling Human Behavior
Near hysteria broke out following Edward Snowden’s revelations about the NSA’s collection of telephone metadata. It wasn’t that we should have been surprised that records of our calls are being kept (after all, the information is on our phone bill), but that those records can be used to track important facets of our lives.
In fact, there is probably no more effective tool for dismantling terrorist networks because, when a mathematical technique called social network analysis is applied, we can quantify connections and connect dots that would baffle a human observer working alone.
Valdis Krebs of Orgnet applies similar techniques to business. His company is regularly called upon by Fortune 500 companies to discover where subject matter experts lie in an organization, advise on post-merger integration and how to foster more innovation. His software has even been deployed to evaluate steroid use in baseball.
Icosystem is another company that applies math to business using a technique called agent based modeling. Pepsi used its service to help them understand how consumers move through a supermarket, Humana has deployed it to design benefit plans and US military has contracted the company to help maintain its fighting force.
Whereas before, human behavior was a domain managed by liberal arts majors, its study is now being revolutionized by mathematicians.
The Diminishing Value Of Experts
News services have traditionally used experts to provide analysis in order to give the audience insight into how events will play out. Unfortunately, research shows that experts are very likely to be wrong.
From Peggy Noonan’s almost comical narrative about “vibrations” that ensured a Romney victory to Meredith Whitney’s bold (and entirely wrong) claim that municipal bond markets were set to crash. Historically, nobody had standing to question them because they were, well, the experts.
That’s what Nate Silver and his brand of mathematical analysis have changed. There is no claim to mystical powers of insight, simply a rigorous analysis of known facts. Perhaps even more importantly, the nature of the analysis is not to conjure the “right” answer from the ether, but to become less wrong over time as more data comes in.
And that’s also what companies like Statweather, Orgnet and Icosystem are doing more broadly for business as a whole. More than ever before, math matters and not just to keep score, but to help us play the game.