Last month, Duncan Watts, a professor of sociology at Columbia University, wrote a New York Times article about some work he and two colleagues did on the predictability of hit songs.Â The work had much broader applicability than just music, of course, and offers an explanation of why predicting hits (in music, movies, fashion, etc.) is so difficult.
Â After noting that “Conventional marketing wisdom holds that predicting success in cultural markets is mostly a matter of anticipating the preferences of the millions of individual people who participate in them,” he goes on to say that
The common-sense view, however, makes a big assumption: that when people make decisions about what they like, they do so independently of one another. But people almost never make decisions independently â€” in part because the world abounds with so many choices that we have little hope of ever finding what we want on our own; in part because we are never really sure what we want anyway; and in part because what we often want is not so much to experience the â€œbestâ€ of everything as it is to experience the same things as other people and thereby also experience the benefits of sharing.
This idea that what is slightly more popular can become even more popular as a result of that initial popularity, is called “cumulative advantage.”Â In the experiment Watts and his colleagues conducted, subjects chose favorite songs in situations in which they knew what songs others had chosen, and in which they did not ( the actual experiment was a bit more complicated, of course).Â
In all the social-influence worlds, the most popular songs were much more popular (and the least popular songs were less popular) than in the independent condition. At the same time, however, the particular songs that became hits were different in different worlds, just as cumulative-advantage theory would predict. Introducing social influence into human decision making, in other words, didnâ€™t just make the hits bigger; it also made them more unpredictable.
The results suggested that predicting the future is not just difficult, but impossible, no matter how much knowledge we have or analysis we do.Â Because we can always create a story after the fact about why something was successful, or a hit, we rarely lose our belief in the predictability of the world.Â As Watts says, we can’t stop trying to predict the future, but we do need to be more skeptical about both predictions and explanations.Â That skepticism holds for predictions of failure too.Â Perhaps the best way to look for hits is to launch as many new products as possible in ways that allow you to hold back substantial investment until it’s likelihood of success becomes clearer.Â A little early humility may go a long way.