The Lost Art of Knowing What We Don't Know
Nov 5, 2013
This week I attended an alumni function at my old high school, St. Albans in Washington, DC, where one of the events was a presentation by Michael Hansen, a St. Albans math teacher, on “metaknowledge,” the awareness of what we know and what we don’t know. (Trust me on this definition – the definitions you’ll find in Wikipedia and other places are totally incomprehensible.) Understanding this concept is central to why we’re so terribly surprised and unprepared when things go wrong with business plans, or rollouts of complex projects like the Obamacare website.
Hansen gave the attendees a test containing ten questions that had specific quantitative answers. Almost no one would know the answer to more than a few of the ten, but most of us could make intelligent guesses. Examples:
Now, if participants are accurately assessing their level of certainly, a typical score on the test should be 9 “right” answers, or possibly 8 or 10. But no! Hansen has administered this test to thousands of people, and the average number of “right” answers is about 3! The reason for this: people grossly underestimate their level of uncertainty. That is, they don’t know what they don’t know.
So you CFOs and aspiring CFOs out there – or to anyone else who has to plan in an uncertain environment – how good do you really feel about the uncertainties you face?
“Painting with Numbers” is my effort to get people talking about financial statements and other numbers in ways that we can all understand. I welcome your interest and your feedback.
Hansen gave the attendees a test containing ten questions that had specific quantitative answers. Almost no one would know the answer to more than a few of the ten, but most of us could make intelligent guesses. Examples:
- In what year was John D. Rockefeller born?
- What is the distance from Earth to the moon, in miles?
- How many novels did F. Scott Fitzgerald write?
The participants were asked not to give single numerical answers to each question, but a range within which they felt 90% certain the correct answer fell. If they actually knew the answer to a question the range might be very narrow (e.g., “1835-1842” for John D. Rockefeller’s year of birth, which was 1839), and very broad if the participant had no idea (e.g., “1780-1900” for the same question). In scoring the test, a “right” answer is one in which the correct answer falls into the range provided by the participant.
Now, if participants are accurately assessing their level of certainly, a typical score on the test should be 9 “right” answers, or possibly 8 or 10. But no! Hansen has administered this test to thousands of people, and the average number of “right” answers is about 3! The reason for this: people grossly underestimate their level of uncertainty. That is, they don’t know what they don’t know.
So you CFOs and aspiring CFOs out there – or to anyone else who has to plan in an uncertain environment – how good do you really feel about the uncertainties you face?
“Painting with Numbers” is my effort to get people talking about financial statements and other numbers in ways that we can all understand. I welcome your interest and your feedback.
Related Blogs
Other Topics
Other Topics