Wisdom of the Crowd

Wikipedia

Dub the Dew

The wisdom of the crowd is the process of taking into account the collective opinion of a group of individuals rather than a single expert to answer a question. A large group’s aggregated answers to questions involving quantity estimation, general world knowledge, and spatial reasoning has generally been found to be as good as, and often better than, the answer given by any of the individuals within the group.

An intuitive and often-cited explanation for this phenomenon is that there is idiosyncratic noise associated with each individual judgment, and taking the average over a large number of responses will go some way toward canceling the effect of this noise. This process, while not new to the information age, has been pushed into the mainstream spotlight by social information sites such as Wikipedia and Yahoo! Answers, and other web resources that rely on human opinion. In the realm of justice, trial by jury can be understood as wisdom of the crowd, especially when compared to the alternative, trial by a judge, the single expert.

The classic wisdom-of-the-crowds finding involves point estimation of a continuous quantity. At a 1906 country fair in Plymouth, eight hundred people participated in a contest to estimate the weight of a slaughtered and dressed ox. Statistician Francis Galton observed that the mean of all eight hundred guesses, at 1197 pounds, was closer than any of the individual guesses to the true weight of 1198 pounds. This has contributed to the insight in cognitive science that a crowd’s individual judgments can be modeled as a probability distribution of responses with the mean centered near the true mean of the quantity to be estimated.

The term crowd, in this usage, refers to any group of people, such as a corporation, a group of researchers, or simply the entire general public. The group itself does not have to be cohesive; for example, a group of people answering questions on Yahoo! Answers may not know each other outside of that forum, or a group of people betting on a horse race may not know each others’ bets, but they nevertheless form a crowd under this definition. The wisdom of the crowd applies to democratic journalism in that a group of non-experts determine what news is important, and then people outside the group can view the news based on those rankings. The social news sites Reddit and Newsvine both fall into this category and rely heavily upon the wisdom of the crowd in creating their content.

Wisdom-of-the-crowds research routinely attributes the superiority of crowd averages over individual judgments to the elimination of individual noise, an explanation that assumes independence of the individual judgments from each other. Thus the crowd tends to make its best decisions if it is made up of diverse opinions and ideologies. One study reduced the independence of individual responses in a wisdom-of-the-crowds experiment by allowing limited communication between participants. Participants were asked to answer ordering questions for general knowledge questions such as the order of U.S. presidents. For half of the questions, each participant started with the ordering submitted by another participant (and alerted to this fact), and for the other half, they started with a random ordering, and in both cases were asked to rearrange them (if necessary) to the correct order. Answers where participants started with another participant’s ranking were on average more accurate than those from the random starting condition.

Crowds tend to work best when there is a correct answer to the question being posed, such as a question about geography or mathematics. The wisdom of the crowd effect is easily undermined. Social influence can cause the average of the crowd answers to be wildly inaccurate, while the geometric mean and the median are far more robust.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.