Anecdotal evidence
Posted: 19 September 2007 04:37 AM   [ Ignore ]
Sr. Member
RankRankRankRankRankRankRankRankRank
Total Posts:  252
Joined  2007-07-12

Hi,

I don’t really understand what is the problem with anecdotal evidence. It is evidence after all, not fairy-tales. Can anyone explain?

Profile
 
 
Posted: 19 September 2007 07:33 AM   [ Ignore ]   [ # 1 ]
Administrator
Avatar
RankRankRankRankRankRankRankRankRankRank
Total Posts:  15305
Joined  2006-02-14
wandering - 19 September 2007 04:37 AM

I don’t really understand what is the problem with anecdotal evidence. It is evidence after all, not fairy-tales. Can anyone explain?

It depends on the context. If I tell you that I saw Mr. Jones rob the liquor store, that is evidence I can use in a court of law. Now, there are plenty of studies that show eyewitness evidence to be globally quite unreliable, so while this does constitute evidence, the problem with it is simply that people take it too seriously. Memory is very fallible, for instance.

HERE is a short wiki page on the subject, HERE is another from the Skeptic’s Dictionary.

The problem with anecdotal evidence in the life sciences for example, or from medical studies, is that one claims to have (“anecdotally”) experienced the efficacy of some procedure like magnet therapy. In fact, that is not really a correct description of what you experienced. It’s a perhaps unjustified interpretation of what you experienced. What you really experienced was the illness, putting on the magnet, and getting better. But it could just as easily have been that you would have gotten better without the magnet. The body is naturally a good fighter of disease. So you didn’t actually experience the causal effects that you describe.

The only way to determine the efficacy of a certain procedure, where the causal force of that procedure is sometimes subtle and hard to distinguish, is to set up a statistically valid, large, randomized trial with a number of patients in the same general state, give them all the procedure, and then do a statistical calculation to see if wearing the magnet correlates with getting better, or if there is instead a statistically insignificant correlation.

 Signature 

Doug

-:- -:—:- -:—:- -:—:- -:—:- -:—:-

El sueño de la razón produce monstruos

Profile
 
 
Posted: 19 September 2007 12:49 PM   [ Ignore ]   [ # 2 ]
Moderator
Avatar
RankRankRankRankRankRankRankRankRankRank
Total Posts:  4052
Joined  2006-11-28

The real problem with anecdotal evidence is twofold:

1) When examined on a large scale, it turns out to be unreliable in predicting actual relationships.
2) People are far more inclined to believe anecdotes that statistics or large scale studies.

As an example. You do all the research on a new car you want to buy. It is far and away the safest, least trouble prone car on the market. There are some problems with it, since nothing is perfect, but far fewer people have trouble than with any other car. Then, you talk to a friend who bought it. His broke down twice in the first month, and he’s convinced the thing is a lemon. If you’re like most people, you will be so moved by this anecdote, you won’t buy the car. Instead you’ll buy one someone else you know had a great experience with. Then, if you have trouble you’ll say, “Well, at least it’s not as bad as that other car would have been!” And if you don’t have trouble, you’ll say, “Whew, I’m sure glad I didn’t buy the pieces of crap! This car is great, so I was right!”

The problem is that even if I can show thorugh unchallengeable statistics and data that a certain relationship doesn’t exist or is spurious, people will “follow their gut” and believe the story instead of the data. I see the pain this causes all the time in my practice. See the example IO posted [url=http://www.centerforinquiry.net/forums/viewthread/2987/P15/]HERE[/url} for how this thiking works.

It’s true not all anecdotes represent incorrect information. Often, if one event follows another the first does actually cause the second. Such reasoning is heuristically useful. But there are situations, usually involving very complex fphenomena such as physiology and medicine, where our anecdotes are far less likely to lead us to the correct answer, and yet we stick with our mistaken understanding because the stories seem so straightforward.

 Signature 

The SkeptVet
The SkeptVet Blog
Militant Agnostic: I don’t know, and neither do you!

Profile
 
 
Posted: 19 September 2007 01:21 PM   [ Ignore ]   [ # 3 ]
Administrator
Avatar
RankRankRankRankRankRankRankRankRankRank
Total Posts:  15305
Joined  2006-02-14
mckenzievmd - 19 September 2007 12:49 PM

As an example. You do all the research on a new car you want to buy. It is far and away the safest, least trouble prone car on the market. There are some problems with it, since nothing is perfect, but far fewer people have trouble than with any other car. Then, you talk to a friend who bought it. His broke down twice in the first month, and he’s convinced the thing is a lemon. If you’re like most people, you will be so moved by this anecdote, you won’t buy the car. Instead you’ll buy one someone else you know had a great experience with. Then, if you have trouble you’ll say, “Well, at least it’s not as bad as that other car would have been!” And if you don’t have trouble, you’ll say, “Whew, I’m sure glad I didn’t buy the pieces of crap! This car is great, so I was right!”

This is an extremely common and irrational way to take anecdotal evidence too seriously. In fact it has been studied in cognitive science; it goes by the name the Availability Heuristic.

People gauge the frequency of an event by how easy it is to bring the event to mind. Since your friend had trouble with his car, it’s very easy for you to bring that sort of event to mind, and you overestimate its probability. Similarly with someone who puts a magnet on his throat and believes it’s cured him of the mumps.

Another common one that is quoted on the Wiki page:

Someone argues that cigarette smoking is not unhealthy because his grandfather smoked three packs of cigarettes a day and lived to be 100. The grandfather’s health could simply be an unusual case that does not speak to the health of smokers in general.

This example of recourse to anecdotal evidence is exactly the same misuse as people who believe that some quack drug cured them of an illness.

 Signature 

Doug

-:- -:—:- -:—:- -:—:- -:—:- -:—:-

El sueño de la razón produce monstruos

Profile
 
 
Posted: 19 September 2007 02:09 PM   [ Ignore ]   [ # 4 ]
Sr. Member
RankRankRankRankRankRankRankRankRankRank
Total Posts:  459
Joined  2007-06-19
wandering - 19 September 2007 04:37 AM

Hi, I don’t really understand what is the problem with anecdotal evidence. It is evidence after all, not fairy-tales. Can anyone explain?

You (we) are prone to misinterpretation of facts. So we need some controlled condition to call something a ‘strong evidence’, where such condition tend to minimize our tendency to misinterpret the facts. Anecdotes ussually don’t happen in such conditions. For instance, we tend to attribute causality to facts which happens one after the other, when we have a lot of others facts that we didn’t take into account.

Example: when I was a child (I was around 9 or 10) we were playing with a friend. I claimed at this age I was atheistic, and we were arguing about god existance. The other guy said out loud ‘if god exists, there be a power failure’. Three minutes after this, the lights turned off in all the neighborhood (the power interruption were very common here in these days).

 

Another trouble is probability. When we are talking about generalization, it doens’t mean that every single observation should follow the rule, so it is possible to find examples which contradict the rule, but when we collect a significative amount of observation following a good randomized procedure, we can observe that the generalization applies.

Example: I can claim that the most probable value when two dices are thrown is 7. You throw the dices 2 times and don’t get any ‘7’, so you say that the rule is untrue… but as far as you continue and throw the dices 20 times, you can observe that there is a tendency to 7.

Profile
 
 
Posted: 25 February 2008 07:05 PM   [ Ignore ]   [ # 5 ]
Sr. Member
RankRankRankRankRankRankRankRankRank
Total Posts:  212
Joined  2008-02-25

Awesome notes! thank you so much.

Profile
 
 
Posted: 03 March 2008 07:30 AM   [ Ignore ]   [ # 6 ]
Sr. Member
Avatar
RankRankRankRankRankRankRankRankRankRank
Total Posts:  9284
Joined  2006-08-29

I am proud to report that for the first time in my life I experienced hallucination. I have been extremely busy at work and I during the last week I got only about tree hours of sleep per night. On Friday evening I started to hear voices in my head. It was very real and quite scary. I can see now how people might believe these things to be real. I know I was deprived of sleep, but were the cause of hearing the voices unknown to me, I am not sure how I would have reacted.

Profile
 
 
Posted: 03 March 2008 05:16 PM   [ Ignore ]   [ # 7 ]
Jr. Member
RankRankRank
Total Posts:  52
Joined  2008-02-22
Daisy - 25 February 2008 07:05 PM

Awesome notes! thank you so much.

I agree… good stuff. A few things popped into my head reading down the thread, so I thought I would share one that I think touches on this. It’s really a fascinating discovery.

It comes from Richard Wiseman’s book, Quirkology. I’ll have to summarize this, but I’ll try my best.

It starts out with a story about, Vic, who worked in a laboratory, a haunted laboratory. While working alone in the lab one night, Vic started to feel very strange and suddenly saw a grey figure emerge out of nowhere. Well, this terrified him.  Vic, being a scientific guy, sought out a reasonable explanation. First, he thought possibly hallucination, nope, and unsatisfied with attributing this to poltergeist activity he ran an experiment. By running a clamped foil (for fencing) along the floor of the lab he noticed it began to vibrate, most noticeably at the center of the room. He then traced this strange wave phenomenon back to a newly fitted fan in the air extraction system. The ghost in the machine was what is referred to as “infrasound”, which can’t be heard but can carry a rather large amount of energy.

Spooky stuff… In the 60’s, NASA scientist ran some test to study the effects of infrasound that is produced by rocket engines. They found that it posses the potential to vibrate the chest, affect respiration and produce gagging, headaches and coughing. It can also be produced naturally from ocean waves, earthquakes, tornadoes, and volcanoes for example.

Wise guy Wiseman, and a team of demented colleagues performed an experiment of their own. They set up an infrasonic generator built from an extra-long stroke sub-woofer placed in a plastic sewer pipe back stage at a concert by the acclaimed pianist, Genia. At certain times during the concert they sent out blast of infrasound onto the unsuspecting concert going guinea pigs. They had asked the concert goers to fill out questionnaires during the show to measure their emotional responses and to note any unusual experiences. Things went off without a hitch. Later the data from the responses given by the concert goers was analyzed. What they found were some interesting reports from the crowd, such as, “shivering on my wrist, odd feeling in stomach”, “increased heart rate, ears fluttering, anxious”, “felt like being in a jet before it takes off”, and my favorite, a women reported feeling, “a pre-orgasmic tension in body and arms”. Must have been a good tune… or more likely, the infrasound! We can tell this by the timing of when the crowd was to fill out the questionnaires and the injection of infrafun.

So, are some “ghostly” experiences due to infrasound? The fun doesn’t stop there, what about the voice of God….

Here I’ll quote from Richard’s book:

“Aeron Watson and David Keating from Reading University have constructed a computer model of a Scottish Neolithic passage grave. Using this model, the researchers have argued that the site has an infrasonic resonant frequency, such that a person beating a 30cm drum could produce powerful low frequency sounds. Others have suggested that some large organ pipes found in certain churches and cathedrals are also capable of producing similar effects.

As part of the preparation for the concert, the team visited several churches and cathedrals that contained especially large organ pipes, and discovered that some were indeed creating significant levels of infrasound. This suggests that people who experience a sense of spirituality in church may be reacting to the extreme bass sound produced by the pipes. Further support for the idea came from one pipe manufacturer who informally told the team that, given that the sounds from these pipes are inaudible, they can either be viewed as a very expensive way of creating a small draft, or a cost effective way of helping the congregation find God.”

[ Edited: 03 March 2008 08:03 PM by MANO ]
Profile
 
 
Posted: 03 March 2008 07:36 PM   [ Ignore ]   [ # 8 ]
Moderator
Avatar
RankRankRankRankRankRankRankRankRankRank
Total Posts:  4052
Joined  2006-11-28

Good on you, George! grin

 Signature 

The SkeptVet
The SkeptVet Blog
Militant Agnostic: I don’t know, and neither do you!

Profile
 
 
Posted: 04 March 2008 07:53 PM   [ Ignore ]   [ # 9 ]
Sr. Member
RankRankRankRankRankRankRankRankRank
Total Posts:  212
Joined  2008-02-25
MANO - 03 March 2008 05:16 PM
Daisy - 25 February 2008 07:05 PM

Awesome notes! thank you so much.

I agree… good stuff. A few things popped into my head reading down the thread, so I thought I would share one that I think touches on this. It’s really a fascinating discovery.

It comes from Richard Wiseman’s book, Quirkology. I’ll have to summarize this, but I’ll try my best.

It starts out with a story about, Vic, who worked in a laboratory, a haunted laboratory. While working alone in the lab one night, Vic started to feel very strange and suddenly saw a grey figure emerge out of nowhere. Well, this terrified him.  Vic, being a scientific guy, sought out a reasonable explanation. First, he thought possibly hallucination, nope, and unsatisfied with attributing this to poltergeist activity he ran an experiment. By running a clamped foil (for fencing) along the floor of the lab he noticed it began to vibrate, most noticeably at the center of the room. He then traced this strange wave phenomenon back to a newly fitted fan in the air extraction system. The ghost in the machine was what is referred to as “infrasound”, which can’t be heard but can carry a rather large amount of energy.

Spooky stuff… In the 60’s, NASA scientist ran some test to study the effects of infrasound that is produced by rocket engines. They found that it posses the potential to vibrate the chest, affect respiration and produce gagging, headaches and coughing. It can also be produced naturally from ocean waves, earthquakes, tornadoes, and volcanoes for example.

Wise guy Wiseman, and a team of demented colleagues performed an experiment of their own. They set up an infrasonic generator built from an extra-long stroke sub-woofer placed in a plastic sewer pipe back stage at a concert by the acclaimed pianist, Genia. At certain times during the concert they sent out blast of infrasound onto the unsuspecting concert going guinea pigs. They had asked the concert goers to fill out questionnaires during the show to measure their emotional responses and to note any unusual experiences. Things went off without a hitch. Later the data from the responses given by the concert goers was analyzed. What they found were some interesting reports from the crowd, such as, “shivering on my wrist, odd feeling in stomach”, “increased heart rate, ears fluttering, anxious”, “felt like being in a jet before it takes off”, and my favorite, a women reported feeling, “a pre-orgasmic tension in body and arms”. Must have been a good tune… or more likely, the infrasound! We can tell this by the timing of when the crowd was to fill out the questionnaires and the injection of infrafun.

So, are some “ghostly” experiences due to infrasound? The fun doesn’t stop there, what about the voice of God….

Here I’ll quote from Richard’s book:

“Aeron Watson and David Keating from Reading University have constructed a computer model of a Scottish Neolithic passage grave. Using this model, the researchers have argued that the site has an infrasonic resonant frequency, such that a person beating a 30cm drum could produce powerful low frequency sounds. Others have suggested that some large organ pipes found in certain churches and cathedrals are also capable of producing similar effects.

As part of the preparation for the concert, the team visited several churches and cathedrals that contained especially large organ pipes, and discovered that some were indeed creating significant levels of infrasound. This suggests that people who experience a sense of spirituality in church may be reacting to the extreme bass sound produced by the pipes. Further support for the idea came from one pipe manufacturer who informally told the team that, given that the sounds from these pipes are inaudible, they can either be viewed as a very expensive way of creating a small draft, or a cost effective way of helping the congregation find God.”

I don’t get your point here, my “awesome notes” comment was mainly addressed to Doug, Mackenzie and Barto’s gorgeous pointers.

Profile
 
 
Posted: 28 April 2008 11:16 AM   [ Ignore ]   [ # 10 ]
Sr. Member
Avatar
RankRankRankRankRankRankRankRankRankRank
Total Posts:  7641
Joined  2008-04-11
mckenzievmd - 19 September 2007 12:49 PM

The real problem with anecdotal evidence is twofold:

1) When examined on a large scale, it turns out to be unreliable in predicting actual relationships.
2) People are far more inclined to believe anecdotes that statistics or large scale studies.

As an example. You do all the research on a new car you want to buy. It is far and away the safest, least trouble prone car on the market. There are some problems with it, since nothing is perfect, but far fewer people have trouble than with any other car. Then, you talk to a friend who bought it. His broke down twice in the first month, and he’s convinced the thing is a lemon. If you’re like most people, you will be so moved by this anecdote, you won’t buy the car. Instead you’ll buy one someone else you know had a great experience with. Then, if you have trouble you’ll say, “Well, at least it’s not as bad as that other car would have been!” And if you don’t have trouble, you’ll say, “Whew, I’m sure glad I didn’t buy the pieces of crap! This car is great, so I was right!”

 

Great thread, very eye opening….
Great example, I could see myself in this! Even though I am a very skeptical person, and tend to do the research myself (in buying my last car), I found myself in just the situation you described (I wanted a Prius)!
geo

[ Edited: 28 April 2008 09:55 PM by asanta ]
 Signature 

Church; where sheep congregate to worship a zombie on a stick that turns into a cracker on Sundays…

Profile
 
 
Posted: 28 April 2008 11:51 AM   [ Ignore ]   [ # 11 ]
Administrator
Avatar
RankRankRankRankRankRankRankRankRankRank
Total Posts:  15305
Joined  2006-02-14

Asanta, I think you should try to get the formatting right. Put your own comment OUTSIDE of the “[ / q u o t e ]” tag, for instance. You can edit your past posts, and “preview” them before posting to see if they show up properly. But the way they are formatted now, they are very difficult to read. It’s impossible to know what is your own words and what’s the post you’re commenting about without going back to the original post to see the difference.

 Signature 

Doug

-:- -:—:- -:—:- -:—:- -:—:- -:—:-

El sueño de la razón produce monstruos

Profile
 
 
Posted: 28 April 2008 01:58 PM   [ Ignore ]   [ # 12 ]
Sr. Member
Avatar
RankRankRankRankRankRankRankRankRankRank
Total Posts:  7641
Joined  2008-04-11

Thanks for the advice,I will go back and edit. red face  OOPs! I cannot figure out how to move my words outside of the blue ‘quote’ area. Please advise!  confused
geo

[ Edited: 28 April 2008 02:03 PM by asanta ]
 Signature 

Church; where sheep congregate to worship a zombie on a stick that turns into a cracker on Sundays…

Profile
 
 
Posted: 28 April 2008 02:46 PM   [ Ignore ]   [ # 13 ]
Administrator
Avatar
RankRankRankRankRankRankRankRankRankRank
Total Posts:  15305
Joined  2006-02-14
asanta - 28 April 2008 01:58 PM

Thanks for the advice,I will go back and edit. red face  OOPs! I cannot figure out how to move my words outside of the blue ‘quote’ area. Please advise!  confused
geo

A quote is opened with [ q u o t e a u t h o r = ” XXXX ” ] and closed with [ / q u o t e ] (without the spaces). You need to close your quoted material before starting to write your own stuff.

 Signature 

Doug

-:- -:—:- -:—:- -:—:- -:—:- -:—:-

El sueño de la razón produce monstruos

Profile
 
 
Posted: 28 April 2008 02:49 PM   [ Ignore ]   [ # 14 ]
Moderator
Avatar
RankRankRankRankRankRankRankRankRankRank
Total Posts:  4052
Joined  2006-11-28

Asanta,

I hope you don’t miind, but I did the edit for you so you can see what it looks like. If you go in to edit the post now, you should see the formatting code as Doug described.

 Signature 

The SkeptVet
The SkeptVet Blog
Militant Agnostic: I don’t know, and neither do you!

Profile
 
 
Posted: 28 April 2008 09:56 PM   [ Ignore ]   [ # 15 ]
Sr. Member
Avatar
RankRankRankRankRankRankRankRankRankRank
Total Posts:  7641
Joined  2008-04-11

THANKS!!! now I get it!!!!!
geo

 Signature 

Church; where sheep congregate to worship a zombie on a stick that turns into a cracker on Sundays…

Profile