Watching the news is a great mental exercise in critical thinking. For example, watch any cable news station and notice how often studies with statistics are reported. That sounds like a good thing, right? We would much rather hear that the news being shared is backed up by scientific studies, surveys, and analysis rather than opinion, right?
Wrong.
The reality is that most of the statistics you hear reported in the news already have an inherent bias. First of all, if the study found a result that differed with the opinion of the news anchor, you probably wouldn’t hear about the study at all. Second, you rarely hear exactly HOW the question was asked, WHO the survey respondents were, and how many people were in involved in the study.
For example, when you hear someone report “62% of the public believes we should pull out of the Iraq war” do you ask yourself:
1) How was the question framed? For instance, did they ask “Should we pull out all troops immediately?” or “Should we gradually withdraw troops over the next several months?” or “Should we pull out of Iraq even if that leads to greater instability and terrorism in the Middle East?” Framing is key.
2) Who did they ask? For instance, was this an online poll on a site that favors one side of the political spectrum? Were 10 people polled or 10,000?
3) Finally, how were the statistics interpreted? If you’ve ever had to quantify your success in an annual performance review, you know how easy it is to highlight successes with statistics while burying the negative ones.
The problem is that numbers and statistics are so compelling. They sound official and as though they’ve been vetted by unbiased third parties. And therein lies the ultimate assumption. It is easy to assume that because the information is presented in a seemingly unbiased way, that it is actually unbiased.
So, next time you hear someone present a compelling argument full of statistics, dig deeper and ask yourself which statistics are not being reported.
Have you been fooled by an argument or pitch that was flooded with misleading statistics?
Did you know critical thinking was rated the NEW #1 workplace skill? Download our paper: 
In the immortal words of Mark Twain, “There are three kinds of lies: Lies, damned lies, and statistics.”
This is a very clear explanation of the key assumption that looking unbiased = being unbiased. A good summary also of some of the other sins which lie behind surveys. After all, when was the last time any charity or company published a survey demonstrating that its services or products were not needed….?
Two days ago on the Fox news report with Shepherd Smith, they played a clip that used this phrase to refute an allegation of a poor job market:
“Increases in uneployment are down 5% from what they were two years ago,”
Phrases like this one clearly demonstrate the need for critical thinking in individuals– especially when considering the current economic context and the apperant motive behind the statement, namely, to imply that unemployment is decreasing, and to use that implication to win political favor. Apply logic to the statement and its deceptive nature becomes evident.
If the number of unemployed individuals two years ago were 1/100 and 50 people per day were loosing jobs, and today the number of unemployed people were 99/100 and only one person per day were loosing a job, the above statement would still be true. Yet, that would clearly not be a reduction in unemployment. The rate of job loss and the percentage of unemployed individuals are two completely separate matters. This basic and factual logical falacy was not pointed out by either the reporter, or by any of the sound clips played as opposing views.
I really wish that formal logic were taught beginning in grammar school. There are too many sharlitans in our time with too much ability to persuade. And the snake oil of the day is statistics.