Wisdom of the JSM Crowd?

Why so different from Galton's Ox Contest?  Need your input!

 

Over 300 attendees stopped by our booth at JSM in Chicago and submitted their guesses to how much money was in the bowl. Erika Menius of Chiltern won the prize with a guess of $137. Was her guess exact? No, but it was the closest - the dollar value was $138 -comprised of 118 $1 bills, 2 $5 bills and 1 $10 bill.

 

Click to See the Data for Yourself

 

Adults Skew Right, Kids Symmetric

We were struck by the extremely long right tail of this skewed distribution. The maximum guess ($2996) was 22 times the actual sum ($138). Moreover, the distribution is regularly shaped and looks like a typical member of a long-tailed family of distributions, e.g. exponential; the high guesses are not outliers. The mean of all 308 guesses was $253 and the median was $178. This histogram confirms the skewed distribution - the distribution of the guesses was so strongly skewed to the right that not much detail emerges of the area around the mean:

 

 

 

 

 

 

 

 

A different picture (right) is shown in the distribution of guesses from a particular non-statistican group - children 16 and under.  It is only a small sample (N=7), but the guesses are more symmetric, tightly clustered, and closer to the actual mean.

 

What's your take?  

Have you seen this distribution in crowd estimation tasks?  Let us know - perhaps we can collaborate on a paper.


Sir Galton and the Ox Contest

This distribution is certainly in no way similar to the distribution of guesses in the Ox Contest reported by Galton that we featured at our booth exhibit. In case you've forgotten, Francis Galton observed a contest at a livestock show in 1907, in which attendees were invited to guess the dressed weight of an ox.  Both the mean and the median of the guesses were spot on - within 1%.  In our contest, the mean was off by 83%, and the median by 29%.

Speaking of the Ox contest, a closer read of Galton's 1907 data shows that distribution of ox estimates was skewed as well, though not nearly as strongly, and his skew was to the left (the under-estimation errors were greater than the over-estimation errors). The magnitude of the errors was also much smaller than we experienced in our contest - the farmers' guess at the first quartile was short by only 4% (ours was short by 13%), and the farmers' guess at the 75th percentile was over by only 3% (ours was over by 98%).

Obviously the farmers were well familiar with oxen and experts at estimating their weights. Statisticians seem to be less familiar with money, at least paper money. (The few children who participated did better.)

How to Estimate?

Many people struggled with the question of how to estimate, and, since guesses were supposed to be independent of one another, they did not share whatever methodology was in their heads, since there was usually more than one person at the bowl.  The only clue we have is this scribbled formula for the volume of the bowl.  The trick is in the units - it is not inches or centimeters (the result would be much too small).  Perhaps it is the dimension of a crumpled dollar bill - i.e. the bowl has a radius of "3 crumpled dollar bills?"

One feature of the money contest was the presence in the jar of one ten dollar bill, and two $5 bills. We were surprised by the number of participants who speculated that we had hidden a number of $100 bills in the center of the jar. Since our goal in the contest was to generate excitement and participation, it would not have been in our interest to spend more money than necessary in order to convey a given impression of riches to be won, a realization that would flow from the application of business understanding and common sense (or "bayesian priors and domain knowledge" as statisticians would term it).

It seems clear, though, that the inclusion of a few larger denominations contributed to the marked skew in the distribution.

The Irony

One irony of the Galton review of the Ox Contest was the fact that he was a committed eugenicist, and believed in the destiny of the "higher orders" in society to rule over the untutored populace. In the results of the contest (in which the median guess turned out to be within 1% of the true weight) Galton found reason to reconsider the virtues of democracy, and titled his note in Nature "Vox Populi:"


"This result is, I think, more creditable to the trustworthiness of a democratic judgment than might have been expected."

Want to hear more?

Peter Bruce sends out periodic newsletters about our upcoming courses with interesting insights.  You can choose which type(s) of courses you'd like to hear about (Introductory, Bayesian Statistics, Biostatistics, Clinical Trial Statistics, Data/Text Analytics, Engineering, Environment, Predictive Modeling, Social Sciences, Survey Statistics, or Using R, Python, SQL) by clicking the "Yes" button beneath "Want to be notified of future courses?" above. 

Want to be notified of future courses?

Yes
Student comments