The Great Thumbtack Experiment

I recently heard about the concept of collective wisdom on a back episode of the Best of the Left podcast (sorry, I don't remember which episode, but I think it was from earlier this summer).  In short, Jay Tomlinson (the podcast host) told us about a study in which the collective wisdom concept was proven to work in a specific, concrete type of situation. 

According the the study (which I can't find), if you present a random sampling of people a concrete item upon which to guess, something to which they could not possibly know the actual answer . . . Their answers will be wildly divergent but, as a group, they will provide the correct answer.

We all know that if I present a sealed jar of jellybeans to a group and ask them to guess how many jellybeans are in the jar, the guesses will be wildly different.  The study indicates that if all of those wildly divergent answers are averaged, the average will be spot-on. 

I was dubious and tempted to blow it off, but I realized I could replicate the experiment.  So I did, on a small scale.

I placed three random handfuls of thumbtacks into a clear box and sealed the box.  I made sure that I had no idea how many thumbtacks were in the box so that I could not bias the sample.  I then walked around for two days handing the box to everyone I ran into, asking them to guess how many thumbtacks were in the box.  I recorded the answers, but I didn't let anyone see those answers until after their guess was made and recorded.  I had the fortune of working in two different offices those two days, so I in actuality had two small samples.

First Sample Guesses:

Second Sample Guesses:

Combined Sample Guesses:

First Sample Average: 120
Second Sample Average: 90
Combined Sample Average: 104
Actual Total Thumbtacks: 110

So, the results of my independent experiment correlate with the study.  The margin of error is actually smaller for the larger group than for the two smaller groups, and, as you can see, is much smaller than the high guess of 500 and the low guess of 40.  When the high guesser told me his guess, I kicked him and told him to give me a real guess.  He insisted that 500 was his real guess.

My lesson?  Collective wisdom works best when there's a jackass in the crowd.  :-)

~Riot.Jane