The Course of Reason

How to Be a Better Skeptic This Year, According to the CIA and Socrates

January 3, 2013

So your New Year's Resolution is to be a better skeptic? Mine too! But what should we do? Good question. 

Teaches philosophy memeThe Wisest Man in Athens knew nothing. Sure, everyone else in Athens knew nothing, too. The difference was that the Wisest Man knew that he knew nothing—he was a good skeptic, and while everyone else thought they knew something, they weren't as good at skepticism. Of course, this is according to the Oracle at Delphi, so take it with a grain of salt.

The point is, the key to wisdom is knowing how little you know, and daily practice of active skepticism can help achieve this. In my humble opinion, it is more important to be skeptical of one's own beliefs than of others' beliefs. It is easy to doubt others' positions, and it comes naturally to many of us. But what is difficult, and crucial, is to constantly question one's own positions. It's a good sort of thing to have as a New Year's Resolution.

The CIA (Central Intelligence Agency) has a different name for skepticism. They call it "Intelligence Analysis." They are woefully terrible at it, of course, but so are we all. They have a little book, called A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis. It teaches young analysts specific methods for figuring out what they know, and more importantly, what they don't know.

According to the primer, it "highlights how structured analytic techniques can help one challenge judgments, identify mental mind-sets, stimulate creativity, and manage uncertainty. In short, incorporating regular use of techniques such as these can enable one to structure thinking for wrestling with difficult questions." Well heck, skeptics manage uncertainty and wrestle with difficult questions all the live long day! Let's take a peek at these structured techniques and see if they can help us in our New Year's Resolution.

1. Grow an epic beard.

bearded woman
The Wisest Woman in Athens

All the best skeptics were Greeks, and they all had epic beards, so, according to the CIA, all epically bearded people are good skeptics.

Okay, that one was a test to see if you are paying attention, and to whet your skeptical appetite. Is the above reasoning valid? If not, what is the deductively valid conclusion that can be drawn? Track down the major and minor premises!

On with the show!

Key Assumptions Check.

The primer tells us, "identifying key assumptions can be one of the most difficult challenges an analyst faces, as they are ideas held-often unconsciously-to be true, and, therefore, are seldom examined and almost never challenged."

I imagine this difficulty holds not just for analysts, but for all humans, including us skeptics. The primer provides a four-step method for checking key assumptions.

1. Pick a belief of yours to scrutinize, and write it down.
2. Write down all the premisses that must be true in order for the above belief to be validly supported.
3. Question each premise, asking what reasons you have for thinking it is true.
4. Keep only the premisses with really good reasons backing them up, and toss out the rest.

Now, obviously there is a bit of informality creeping around the edges of this method. For instance, whether or not there are "really good reasons" backing up a premise might come down to a judgment call. It is important to be brutally honest with yourself, and err on the side of doubt.

If you only had to toss a few premises, it might be worth your time to do some research and see if you can scrounge up some good reasons in support of them. Otherwise, if you find yourself slashing premises left and right, you've got to let the belief go.

snooopy heart
Letting go of cherished beliefs can be hard.

Quality of Information Check.

The first method focuses on a specific belief and evaluates it. This method focuses on a specific source of information, and evaluates it. It's really important to do this, because often we have many beliefs tied to a single source, so identifying a weak source can efficiently dispose of a large number of unsupported beliefs at once.

This process can be a little research intensive. It's best to focus on sources of information that play central roles in supporting a large number of beliefs, like your favorite news organization.
Get as much background information on the source as you can find. Specifically look for underlying interests of the source that might conflict with its presentation of accurate information. Identify the circumstances surrounding the source, and think about how those circumstances affect its presentation of information. Think about bias.

Check to see if its information is corroborated by any independent sources. Corroboration is crucial, but so is independence. If a bunch of sources corroborate each other's information based on a single shared source, then you've really only got the one source, which may be very weak.

Based on these considerations, assign the source a level of confidence. Its level of confidence may be higher or lower, depending on different types of information. For instance, a book of Bronze Age scribbling may be a poor source for astrophysics, chemistry, medicine and psychiatry, but it is probably pretty good for figuring out what people were like in that time period and geographic location.

I'm just sayin'...

Analysis of Competing Hypotheses

Do you have any friends? If so, you can use this method and the next to great effect! Suppose you've got a big pile of evidence, or a few peculiar observations. What's the best explanation of the evidence or observations? We've all seen this method in action by our favorite diagnosticians at Princeton-Plainsboro Teaching Hospital. (Hint: it's not Lupus.)

Begin by listing all reasonable hypotheses/explanations at once. Then, go through and think about what evidence would disprove each hypothesis, and then look for that evidence. The method harkens back to Sherlock Holmes, who was both the literary inspiration for Gregory House and an avid proponent of analyzing competing hypotheses. Holmes wisely told us to eliminate all the impossible explanations, because whatever remains, however improbable, must be the truth.

House apple
Some sort of symbolism about differential diagnosis being responsible for the Fall of Man.

Devil's Advocacy

Yes, I've saved the best for last. Devil's Advocacy is a perennial favorite for skeptics, as we so often play the role when challenging others' beliefs. However, it is important to make sure you have a good friend who will Devil's Advocate against your own beliefs. Make sure you are good friends with the person, though, and that there's a great deal of trust between you. Historically, this is the role Socrates played for the good people of Athens. And they killed him for it! He should have played only with friends.

Here's how you play:

1. Pick a strongly-held belief of yours.
2. Ask a friend to present the best case against that belief.
3. Honestly assess how well you are able to defend your belief against the onslaught.
4. If you struggle to defend it, you should be less confident in it, or abandon it altogether.

This is a great tool to be used in conjunction with the other methods. For instance, when evaluating a premise in a Key Assumption Check, it is helpful to run a Devil's Advocacy against it.

You, after your friend demolishes your belief in free will.

Be sure to use these methods throughout the year, thereby improving your skepticism skills. There are a wide range of topics ripe for the picking: morality, health, the law, literature, even religion! Just because you're an atheist doesn't mean you are done skeptically assessing your own position. If you're not an atheist, well, there is plenty of room for skeptical improvement this year!


About the Author: Seth Kurtenbach

Seth Kurtenbach's photo
Seth Kurtenbach is pursuing his PhD in computer science at the University of Missouri. His current research focuses on the application of formal logic to questions about knowledge and rationality. He has his Master's degree in philosophy from the University of Missouri, and is growing an epic beard in order to maintain his philosophical powers. You can email Seth at or follow him on Twitter: @SJKur.


#1 jmaii on Friday January 04, 2013 at 7:51am

I was thinking of growing a beard... Anyway, "A Tradecraft Primer" is also available online free from the GPO:




Guests may not post URLs. Registration is free and easy.

Remember my personal information

Notify me of follow-up comments?

The Committee for Skeptical Inquiry's magazine (available at is called...

Creative Commons License

The Course of Reason is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.

CFI blog entries can be copied or distributed freely, provided:

  • Credit is given to the Center for Inquiry and the individual blogger
  • Either the entire entry is reproduced or an excerpt that is considered fair use
  • The copying/distribution is for noncommercial purposes