The Wisdom of Crowds

The Wisdom of Crowds is a 2004 book written by American journalist James Surowiecki about the aggregation of information in groups, resulting in decisions that, he argues, are often better than could have been made by any single member of the group. The book presents numerous case studies and anecdotes to illustrate its argument, and touches on several fields, primarily economics and psychology.

The opening anecdote relates Francis Galton’s surprise that the crowd at a county fair accurately guessed the weight of an ox when their individual guesses were averaged (the average was closer to the ox’s true butchered weight than the estimates of most crowd members, and also closer than any of the separate estimates made by cattle experts). The book relates to diverse collections of independently-deciding individuals, rather than crowd psychology as traditionally understood, however its title is an allusion to Charles Mackay’s ‘Extraordinary Popular Delusions and the Madness of Crowds,’ published in 1841 (which chronicled economic bubbles, witch-hunts, crusades, and similar phenomena).

Not all crowds (groups) are wise. Consider, for example, mobs or crazed investors in a stock market bubble. According to Surowiecki, these key criteria separate wise crowds from irrational ones: Diversity (each person should have private information even if it’s just an eccentric interpretation of the known facts); Independence (people’s opinions should not be determined by the opinions of those around them); and Decentralization (people are able to specialize and draw on local knowledge). Rational bubbles occur when members of the crowd become too conscious of the opinions of others; individuals attempt to emulate each other and conform rather than think differently.

Surowiecki is a very strong advocate of the benefits of decision markets and regrets the failure of DARPA’s controversial Policy Analysis Market to get off the ground. He points to the success of public and internal corporate markets as evidence that a collection of people with varying points of view but the same motivation (to make a good guess) can produce an accurate aggregate prediction. According to Surowiecki, the aggregate predictions have been shown to be more reliable than the output of any think tank. He advocates extensions of the existing futures markets even into areas such as terrorist activity and prediction markets within companies. Applications of the wisdom-of-crowds effect exist in three general categories: Prediction markets; Delphi methods; and extensions of the traditional opinion poll.

The most common application is the prediction market, a speculative or betting market created to make verifiable predictions. Similar to Delphi methods but unlike opinion polls, prediction (information) markets ask questions like, ‘Who do you think will win the election?’ and predict outcomes rather well. Answers to the question, ‘Who will you vote for?’ are not as predictive.

Assets are cash values tied to specific outcomes (e.g., Candidate X will win the election) or parameters (e.g., Next quarter’s revenue). The current market prices are interpreted as predictions of the probability of the event or the expected value of the parameter. Betfair is the world’s biggest prediction exchange, with around $28 billion traded in 2007. NewsFutures is an international prediction market that generates consensus probabilities for news events. Several companies now offer enterprise class prediction marketplaces to predict project completion dates, sales, or the market potential for new ideas. A number of Web-based quasi-prediction marketplace companies have sprung up to offer predictions primarily on sporting events and stock markets but also on other topics. Those companies include Piqqem, Cake Financial, and Covestor.

The Delphi method is a systematic, interactive forecasting method which relies on a panel of independent experts. The carefully selected experts answer questionnaires in two or more rounds. After each round, a facilitator provides an anonymous summary of the experts’ forecasts from the previous round as well as the reasons they provided for their judgments. Thus, participants are encouraged to revise their earlier answers in light of the replies of other members of the group. It is believed that during this process the range of the answers will decrease and the group will converge towards the ‘correct’ answer. Many of the consensus forecasts have proven to be more accurate than forecasts made by individuals.

In his book ‘Embracing the Wide Sky,’ British savant Daniel Tammet points out the potential for problems in systems which don’t have a well defined means of pooling knowledge: Subject matter experts can be overruled and even wrongly punished by less knowledgeable persons in systems like Wikipedia. Tammet also cites Kasparov versus the World, an online competition that pitted the brainpower of tens of thousands of online chess players choosing moves in a match against Gary Kasparov, which was won by Kasparov. In his book ‘You Are Not a Gadget,’ futurist Jaron Lanier argues that crowd wisdom is best suited for problems that involve optimization, but ill-suited for problems that require creativity or innovation. In the online article ‘Digital Maoism,’ Lanier argues that the collective is more likely to be smart only when 1. it isn’t defining its own questions, 2. when the goodness of an answer can be evaluated by a simple result (such as a single numeric value,) and 3. when the information system which informs the collective is filtered by a quality control mechanism that relies on individuals to a high degree. Only under those circumstances can a collective be smarter than a person. If any of these conditions are broken, the collective becomes unreliable or worse.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.