A Well-Designed Poll on a Difficult Subject

Dec 17, 2014

Since my last post was so negative, and so critical of the stuff given to us to read, it’s a pleasure to encounter a good example of quantation, even on an unpleasant subject. I’m referring to a Washington Post-ABC News poll of public opinion related to the recent Senate report on torture and the CIA. Here’s what I liked about the survey design:

It was short – seven questions, with one additional sub-question. (The survey had a total of 28 questions, but the rest addressed other subjects.)
The questions were easy to understand and (in my opinion) neutrally worded.
All but one of the questions had only two possible answers (other than “No opinion”).
The numerical results had meaningful interpretations, and lent themselves to cross-correlation.

I found the results surprising, and for that reason alone I encourage you to read it. But regardless of your personal opinions on the matter, I commend it as a good example of quantation.

I see far too many surveys, in both business and personal life, where the questions are hard to understand and hard to answer, and where it’s clear that tabulating the results won’t provide meaningful answers. That’s understandable when the intention of a survey is to support a particular point of view, but it’s a shame when expensive decisions rest on the results.

“Painting with Numbers” is my effort to get people to focus on making numbers understandable. I welcome your feedback and your favorite examples. Follow me on twitter at @RandallBolten.