Certain words push my buttons. One of them is "anecdotal," at least when applied in blanket fashion to evidence for the paranormal.
We see this all the time. A skeptic dismisses every bit of evidence for a certain category of paranormal phenomena with the sweeping assertion, "It's all anecdotal." Most of the time, the skeptic has never even looked at the evidence, and is simply assuming it's anecdotal, since, after all, it can't possibly be valid.
In other words, the skeptic is caught in an endless loop of circular reasoning: Paranormal phenomena can't be real. Therefore all evidence for such phenomena is faulty. How we do we know all the evidence is faulty? Because paranormal phenomena can't be real.
The analysis seldom goes deeper than this. Rare is the skeptic who has actually looked at the evidence in detail, including the strongest cases, and still retained his skepticism.
Often, even highly specific evidence is simply assumed to be anecdotal. Years ago I came across a skeptical essay on the Web that said something like this (paraphrased from memory): "It's sometimes claimed that a hospital patient reported a near-death experience during which she left her body, floated outside the building, and saw a tennis shoe on a ledge. A social worker, hearing this story, went to the specified location and was able to find and retrieve the shoe, which exactly matched the patient's description. Convincing? Of course. The problem is, no such patient or social worker has ever been identified, nor has the hospital ever been found. The story is purely anecdotal."
Actually, the identity of the social worker is well known. She is Kimberly Clark, and she wrote a book about this experience. The hospital was Harborview Medical Center in Seattle. (I covered the case in a five-part series of posts, starting here. It is also briefly summarized here in PDF form.) The case may be controversial, but it is not anecdotal. No case that has been carefully documented and researched, including interviews with the participants (Clark and other hospital staffers), can be considered a mere "anecdote."
But the Clark case does rely to a large extent on Clark's own testimony. Maybe that's not good enough. Of course, such testimony is widely considered good enough to stand up in a court of law, and good enough to constitute the bulk of the data in most social sciences, such as field anthropology. But maybe the controversial nature of paranormal claims, as well as the undeniable history of fraudulent psychics and mediums, requires us to find a more objective basis for analysis.
Okay. Then let's say we were going to design a truly rigorous scientific test for a particular paranormal phenomenon. How would we go about it? For the sake of argument, let's take mediumship. Suppose we want to test a particular medium's purported abilities. How would we do it in the most objective way possible?
First, we would realize that a medium's statements may be misremembered by the sitter; hits may be recalled while misses are forgotten. To guard against this, we hire a stenographer to record every utterance word for word.
Next, we are aware that fake mediums can use cold reading (intuitive guesses based on the sitter's responses) to elicit information from the sitter. To prevent this, we hit on the idea of using a proxy sitter. That is, the person sitting with the medium will be only a stand-in. He will know nothing about the deceased persons we are hoping to contact. Since he knows nothing, he can give nothing away.
After the séance, the stenographer's transcript will be given to the person whose dead friends or relatives were purportedly contacted, and this person (who was not present at the sitting) will rate every single statement in terms of its accuracy and relevance. Misses will be noted as well as hits. And we won't accept vague statements that could apply to almost anyone. We are only interested in specific, verifiable information of obvious relevance.
What else can we do? Well, we can ensure that the medium has no way of knowing whom the proxy is sitting in for. The identity of the "man behind the curtain," the person seeking information on his dead loved ones, will remain unknown to anyone but the experimenter. In fact, we will take pains to ensure that this person has no known connection with the medium, with the proxy, or with us. It will be someone out of left field, someone who doesn't even live in the same geographical area as the medium.
But what if the medium has hired people to collect information – even to spy on us – and pass it on to her? We take the precaution of hiring detectives to tail her without her knowledge whenever she goes out, to watch her house when she is home, and even to intercept and open her mail.
Now, suppose we do have a successful reading. Couldn't a skeptic argue that it was a fluke, a series of lucky guesses? Maybe the medium just happened to hit a home run this one time. So we can't be satisfied with a single proxy sitting, no matter how good it may be. We will do many of them – dozens, even hundreds of sittings over a period of many years. All results, whether positive or negative, will be reported. In fact, we'll publish them in a peer-reviewed journal, we'll keep all the transcripts on file where interested parties can study them, and we'll invite our colleagues to test the medium also, either in concert with us or on their own.
Now, would the resulting body of data constitute anecdotal evidence? Only if the meaning of the word "anecdotal" is twisted past recognition.
It will probably come as no surprise when I report that exactly this protocol was carried out by Richard Hodgson in his years-long investigation of the Boston medium Leonora Piper. All of it – the proxy sittings, the stenographer, the multiple tests, the peer-reviewed publications, even the private detectives opening poor Mrs. Piper's mail! (Mrs. Piper didn't have a telephone, but if she had, Hodgson would undoubtedly have found a way to monitor that, too.)
Despite all these precautions, Mrs. Piper produced such strong and repeated evidence of communication with the dead that even the ultra-cautious Hodgson (who had previously debunked other mediums, including the famous Madame Blavatsky) was persuaded, as were many other researchers who participated in the tests.
Skeptics who encounter this case sometimes say vaguely that Hodgson's protocols must have been inadequate, but they are unable to say how. (See, for instance, Martin Gardner's pitiful debunking attempt.)
Or they say that it all happened a long time ago, back around 1900, and therefore can be safely ignored – as if the facts, collected and reported at the time and carefully preserved, could be altered simply by the passage of time. (If so, then shouldn't all scientific observations from more than a few years ago be disregarded? For that matter, how about all the historical records that we depend on to reconstruct the past?)
Or they say that the Hodgson studies are "anecdotal," thus showing that they don't know what the word "anecdotal" means.
Now, let me be clear. There is no doubt that some of the alleged evidence for the paranormal is anecdotal. When your Uncle Harvey tells you he saw a ghost in 1969, you can choose to believe him or not, but this claim is not testable or verifiable. So if skeptics want to say that some of the evidence is anecdotal, I have no quarrel with them.
When they say all the evidence is anecdotal, however, they are simply mistaken. Most of them are innocently mistaken; a few, I think, are intentionally misleading their audience. As an example of the latter, I would point to Paul Kurtz, the founder of CSICOP, who said on a TV show some years ago (I'm quoting from memory): "It would be nice to believe in life after death, but there is absolutely no evidence for it." Absolutely no evidence? Not even a tiny smidgen of it? Kurtz knows better than this. But he is counting on his audience – and the interviewer – not to know.
Maybe the best lesson from all of this is to beware of blanket statements. It is as incorrect to say that all the evidence is anecdotal or nonexistent as it is to say that all the evidence is valid and trustworthy. The job of diligent students in this field is to separate the documented facts from the rumors, the rigorous studies from the merely anecdotal reports.