Sunday, August 10, 2008

Silas experimentally tests "wisdom of crowds" effect

In case you hadn't heard of it, James Surowiecki has gathered a substantial amount of evidence supporting the "Wisdom of Crowds" Hypothesis, which says, basically, that if you can aggregate the independent guesses of large numbers of people, they will average out to be the correct answer, even though the individual guesses will tend to be far off. Surowiecki found confirmation of this hypothesis in such varied cases as guessing the beans in a jar, the weight of a bull, and the location of a missing submarine.

Well, just for kicks, I went out and did my own experiment to see if could replicate his results. When I went out for a night on the town last Friday (8/08/08), I asked the people I talked to how much they thought I weighed. Here's what I looked like, so you can make a guess and tell how far off you would have been:

To help them make their best guess, I allowed them to see me without the blazer and from different angles, and to, ahem, touch me. To adhere to the protocols necessary to find wisdom in crowds, I made sure they didn't discuss guesses with each other until they submitted them, and that they made the guesses anonymously (by writing it town and putting it in a plastic bag out of my view). I excluded non-numerical entries such as "Fuck U".

The results? After collecting 16 entries at two venues, here is what I got (and this will be expressed in terms that conceal my weight and their guesses, which will be revealed in a later post):

The average (mean) of their guesses was off by 3.6%.
The average error (mean error) of all the guesses was 10.3%.
The median guess was off by 6.3%

This is pretty much consistent with the Wisdom of Crowds phenomenon: the average of the guesses was much more accurate than most of the individual guesses, and the median guess. It probably would have been even better if I had collected a bigger sample.

Pretty neat, huh?


Bob Murphy said...

That's a new one to meet people at the bar (and get them to touch you).

I don't trust crowds since going to Battery Park on July 4 one year, and everybody had set up their lawn chairs facing the wrong way to wait for the fireworks. It was a riot when they started going off behind us, and everyone who had gotten there 7 hours earlier for the front row realized their seats s*cked.

Serves them right for misleading 5,000 people.

uke said...

Interesting stuff. You look nothing like I would have imagined from all that time on the War on Terror boards.

Alex S said...

Very cool...I tried this with M&M's in my class, but they all talked with each other anyway. You should check out my book review: