About the Museum
The Museum of Hoaxes is dedicated to promoting knowledge about hoaxes. (Click here for opening hours, etc.) On our blog we post about dubious- sounding claims, and whatever else strikes our fancy. The site is also home to the Hoaxipedia (the museum's online encyclopedia of hoaxes), the Hoax Forum, and the Top 100 April Fools' Day Hoaxes.

The museum was created in 1997 by Alex Boese. He's assisted by a staff of deputy curators and docents. Alex is the author of three books, most recently Elephants on Acid: And Other Bizarre Experiments (which has nothing to do with hoaxes). Check out the list of the Top 20 Most Bizarre Experiments of All Time for a preview.

Web Hoax Museum

Marlon,

PS - You should check into DNA perfection... I bet you'd be real big on that as well. Get that done and you might then be SUPER-ENLIGHTENED! She missed the metal plate in the head after making the claims that were made??? Just that one miss and NO MORE information is required to show it's not real. I've got a $15 metal detector that could probably do just as well as she did... Come on Marlon, they're the ones who claimed small metal objects, past fractures, etc etc could be detected... not us. SHE / IT failed... get over it!

Quote: (PS - You should check into DNA perfection... I bet you'd be real big on that as well. Get that done and you might then be SUPER-ENLIGHTENED! )

I think that if I would be super enlightened I wouldn't be a bit irritated now because some people don't agree with me. Haha... But If I annoy people with writing things as (have a good day, ore smile) I suppose I'd better not do that again.

Why do you find my discription of reality is nonsense, cause that's what it comes down on, isn't it?

In the dictionary (Dutch) stands:The discription of reality is: "The real existence of something" / " the here and know".

Know we can only inteprit reality with our senses. And our experiences has much to do with our senses. Why should the one thing we sense be reality and the other not. Is only the physical reality real? Why? That isn't the meaning of the word reality.

About Natasha...I can imagine that you don't believe in her gift because she missed the metal plate...but then your looking at a part and not at the whole.

Marion,

We find your description of reality a nonsense because we have a tool called "science" which is the best tool we have for distinguishing sense from nonsense. It's by no means perfect, but it certainly beats other tools such as "superstition", "jumping to conclusions" and similar.

If I was unaware of magnetism or similar phenomena, then I would be inclined to disbelieve that a small metallic object could attract another metallic object through a barrier such as a thin sheet of glass. However, skeptical as I would be, if someone gave me a magnet, and allowed me to experiment with it, then I'd soon discover that there was some attractive force. I'd try different sheets of glass, other barriers, see that the magnet attracts other metallic objects, and eventually I'd be convinced that the magnet was able to attract some metals. Therefore I'd conclude that the claim for the existence of magnetism was sense, not nonsense. If I gave the magnet to another "magnetism skeptic" then s/he could then convince themselves of the reality of the effect. And, if magnetism was a new discovery, there would be huge numbers of cases of people demonstrating that the effect is real.

However, when I see claims for such phenomena such as the efficiacy of homeopathic remedies, all I see is that when people try to verify these claims, and do so using proper methods and experiments, the results suggest that the claim is incorrect. Hence I classify such claims as nonsense.

There is a problem in society in general that the vast majority of people simply do not have the education and "tools" required to adequately separate sense from nonsense. Arrogant as this may sound, once someone studies science properly, and undestands how it really works (including an in-depth knowledge of statistics), there is no other conclusion that can be made.

It only takes a quick read of one of any number of books, such as _Methodological Errors in Medical Research_ by Bjorn Anderson (ISBN: 0-632-02137-3) to see that even among trained medical researchers publishing in medical journals, a large proportion don't have sufficient training to design experiments adequate to separate sense from nonsense. And that explains a lot of what we see in the real world.

Cheers,

Ross-c

PS: Marion, I'd like to make a challenge to you.

Some people in this thread have spoken of auras. I don't know whether you believe in auras or not, but please hear me out. Would you consider the following an adequate experiment to see whether or not people can truly see auras.

First, we'd need a number of people (lets call them 'seers') who claim to be able to see auras. Since this experiment is going to be based on consistency, the seers would have to trust each other's ability, so we'd allow them to talk to each other, do demonstrations, and establish that all of them (say 3 of them) do have this "power".

There would then be a session where the three 'seers' would agree on a number of properties of aura that differ between people, and how these are named. The three seers would be placed in separate rooms with no means of communication. New people ('subjects') whom the seers have not seen before would be shown to each of the seers in turn. The seers would describe the auras of these people in the terms previously agreed.

If the descriptions of the auras agreed to a degree such that random chance could be ruled out with 95% certainty, then it would be concluded that the seers can truly see the aura of the subjects, and hence that the aura exists.

Marion, do you think that this would be both a fair, and an accurate experiment to test for the ability of seers to see the auras, and therefore that the aura exists? Rather than just a yes/no answer, may I ask you to explain the reasoning behind your answer?

Cheers,

Ross-c

Natasha isn't lying. And CSICOP were highly unfair. I have so much to say on this matter but I'm not going to say it all. However, the scientists who tested Natasha called her "deluded" that is highly disrespectful, distasteful and in all honestly I find it repulsive. That is just pure discrimination. Just because she has a gift that they cannot see does not mean she is delusional. That is all. Thankyou.

A gift they cannot see because it didn't work?
I mean, if it had, they'd have seen the impressive rate of diagnoses, rather than a weak 4/7 after four hours.

Marlon, first you cite the supposed degrees your "Doctor" has, then when you are faced with irrefutable evidence that the schools which granted them are phonies, suddenly you say "Barbara Brennan holds a Master Degree in atmospheric physics. That is one thing. I do not know much about her other grades."

It's painfully obvious that you have drawn your conclusion without any actual EVIDENCE and that you are not going to let any EVIDENCE change your mind.

The facts are what the facts are and, unfortunately for your belief system, they don't support what you think. There comes a time when a reasonable person throws in their hand. I think you have long since passed that point. You're trying to bluff with a pair of deuces. Throw in.

Tali,

Natasha claimed to have a "gift", and claimed that this gift enabled her to look inside people. Her ability or inability to look inside people is something that can be objectively tested, and the evidence is that she can't. So, if she still believes that she can see inside people, then "deluded" is a fair judgement. Given the result of the experiment, the only ways that we can describe her is "deluded" or "a fraud". Which would you prefer?

Cheers,

Ross-c

I agree with Puck T Benson upthread, who calculated the proabilities randomly using a computer program. I did the maths on this problem another way, by writing a computer program to generate all 5040 (7*6*5*4*3*2*1) possible permutations of patients and symptoms, and getting the number of correct diagnoses - ranging from 0 to 7 - in each permutation, and totalling them up. These are the results (same as Puck's)

0 - 1854
1 - 1855
2 - 924
3 - 315
4 - 70
5 - 21
6 - 0
7 - 1

The probabilities associated with each score are these numbers divided by 5040. The probability of Natasha getting a score of 4 was thus 1.389%. Or 1 chance in 72 - not 1 in 50 or 1 in 55, as other people have said.

Ross-c said:

"Natasha claimed to have a "gift", and claimed that this gift enabled her to look inside people. Her ability or inability to look inside people is something that can be objectively tested, and the evidence is that she can't. So, if she still believes that she can see inside people, then "deluded" is a fair judgement. Given the result of the experiment, the only ways that we can describe her is "deluded" or "a fraud". Which would you prefer?"

Very well said, Ross. The bottom line here is that she claimed to have a "gift," she agreed to be tested, she was tested based on what she claimed she could do and she failed. End of story.

As Ross said, either she's a conscious fake or she's self-deluded. Pick one.

I commend you, Fomalhaut (a well-known star in the Southern night sky - I take it that you are into astronomy) on your computer programming skills, but your statistical reasoning needs to go one step further. In Statistical testing, we begin with the presumption that there is nothing of interest in the data. Here that refers to Natasha guessing. We then ask, what is the chance of getting a result as good as the one actually found or better by coincidence? Here that is the probability of getting 4 or more right, i.e. (70+21+1)/5040, which is 1 in 55. The reason we include the 'or better' is that by guessing Natasha could conceivably have gotten 5 or more right, though this would be far less likely.

Fomalhaut, the easy way to see the fault in your reasoning is this: Imagine that she had looked at 1000 people, not 7. Then the probability of any one single result (correct out of 1000) would be small. Hence a probability less than 5% would prove nothing, as we'd expect results better than random guessing about half the time, and all of these would individually have a probability of < 5%. Hence claiming 95% confidence would be plain silly. Looking at the probability of results as good or better than she achieves solves this problem.

Cheers,

Ross-c

Thanks for the comments. I think see the point of the 4-or-more bit now.

But I'm puzzled that Andrew Skolnick, posting here on 9 Dec 04, quotes Professor Ray Hyman responses to Puck's first (wrong) estimate of the probability:

"The second mistake this critic makes is to use the probability for getting exactly four correct matches. The number that is relevant for our test is the probability of getting four or more correct matches. Contrary to this persons assertion, the probability of getting exactly four matches in our test is .01533 and not 1/840 (.0012) as he claims. The relevant probability is the probability of getting four or more correct matches which is .01899 (rounded to .02 or 1 in 50)."

Now, by my calculations, the probability of getting exactly four matches is 70/5040 or 0.01389, not .01533. And the probability of getting four or more is 0.01818, not .01899. These are slight differences, but differences all the same. So how did they get their figures? By using a formula out of some book, it seems. What is this formula, and how was it derived?

Fomalhaut, I wrote a quick program to calculate the probabilities, and I get exactly the numbers you do. 40/5040 for exactly 4 right out of 7, and 92/5040 for 4 or more right out of 7.

I wondered if the original calculations were done using the binomial distribution (which would have been a silly thing to do, but I gave it a go). Those numbers are much further off. So, I have no anwer as to where their numbers came from.
I'd guess that they made a calculation error, but could be wrong.

Cheers,

Ross-c

Ooops! That 40/5040 was a typo. It was 70/5040.

Cheers,

Ross-c

Ross, I'm glad we got the same answers. Perhaps the explanation for the discrepancy between our figures and CSMMH's lies in Andrew Skolnick's 9 Dec 04 post, from which I've already quoted, but which includes these remarks:


"The problem we are dealing with is known as the matching problem. The mathematics for calculating the correct odds is not self evident. Indeed, it is very complicated. I painstakingly worked out the correct probabilities using the formulae in Frederick Mosteller's Fifty Challenging Problems in Probability With Solutions. I believe this is still available from Dover Books. The critic might find it useful to carefully follow the argument in this book. My other source was Hoel, P.G., Port, S.C., and Stone, C.J. (1971). Introduction to Probability Theory. This latter source provides some useful approximations for those who do not have the patience to calculate the exact probabilities. Richard Wiseman was able to check my probability calculations using tables provided by the Journal of the Society for Psychical Research. Our probabilities agreed."

So perhaps the figures they produced were "useful approximations" rather than "exact probabilities" for a "very complicated" problem. And the problem is indeed very complicated - or at least extremely tedious - if attempted manually. But it is quite easy to solve using a simple computer program (as you found out). This rather suggests that the probabilities were calculated using methods appropriate to a pre-computer era, and which gave approximate rather than exact answers. The 1971 book on probability cited is some 35 years old, and thus dates from an era when computers were monstrous beasts that lived in university mathematics departments, and so it was most likely written for use by people who did not have access to such computers.

I'm neither a mathematician nor a statistician, but it seems to me that people who are working with probability and statistics really ought to be able to calculate their figures with an accuracy appropriate to the era in which they live. I find myself slightly raising an eyebrow that CSMMH apparently lacks the computing skill to correctly calculate probabilities (even if in this case it is relatively inconsequential). And I find myself wondering what else they can't do.

Hmmm.... Interesting. I didn't use any formulae to calculate the probability, my program just enumerated all possiblities, and counted the matches for each one. It would be possible for both of us to agree on incorrect results if we both made the same incorrect assumption.

On the page:

http://www.ds.unifi.it/VL/VL_EN/urn/urn6.html

the matching problem is discussed, and the formulae given for calculating the probabilities of matches. I wrote another program to calculate the probabilities of various matches for the 7 people on the tv show. I get ...

ross@home$ ./matching
For k=0 b is 1854
For k=1 b is 1855
For k=2 b is 924
For k=3 b is 315
For k=4 b is 70
For k=5 b is 21
For k=6 b is 0
For k=7 b is 1
ross@home$

Or, 70/5040 for k=4, and 92/5040 for k >= 4. I.e. the same as both of us got with our previous programs.

The page mentions a "Poisson approximation", but this is for very large matching problems, not 7. Also, even if the problem had been solved by hand, it wouldn't be that difficult. So, there's probably more to this than it appears.

Though, in the end, the differences in the probabilities are very small.

Cheers,

Ross-c

Hi Cranky Media Guy

You said: It's painfully obvious that you have drawn your conclusion without any actual EVIDENCE and that you are not going to let any EVIDENCE change your mind.

I don't agree with what you say. If you look at her credentials you can see that she followed several studies and trainings. With some of them you can "earn" a official grade and with some of them not. Some of them are universal studies and some of them are not. When I read the credentials I understood that the grades in Atmospheric Physics, Theology and Philosophy are official.

Now, someone here told me that the universities where she got the grades in Theology and Philosophy are not regocnized in some states. He showed me some sites. I didn't really understand those sites but I think he told the truth by saying that these universities aren't regocnized by some states.

My opinion about this case is that only because a school is not regocnized by some states doesn't mean that you don't learn anything during your studie at that university. As far as I know, after reading his post, there are only a few states that don't recognize the universities. So most states DO recognize that universities. Because of that I think she has the right to say that she holds a doctorate in Philosopy an Theology. As far as I know from the most states do.

Hi Ross-c

You seem quite reasonable to me. I also find it very fair that you say science isn't perfect...

About the experiment.I don't thinkt this is a fair, and an accurate experiment to test for the ability of seers to see the auras, and therefore that the aura exists.

The seers could make appointments with eachother.

For example thay can say...If the person has blond hair and wears glasses, we say that the person his aura is torned on the left side. If the person is a man and is bold, his aura is red, with a lot of grey and green. Ore they could say, if the person has brown eyes his aura is very large and bright...

And so they could make many appointments.
And if no one nows this, it seems to be that they ARE able to see aura's.

So, this test would be worhtless if you want to prove that some people can see aura's and that the aura exists.

I hope I have made my point claar about this

Marlon

Marlon,

First, can I apologise for getting your name wrong in one of my answers. The font I was reading this in at work wasn't that clear.

OK: About your answer concerning the experimental design. Yes, the experiment is flawed as it would permit the seers to cheat. However, it's actually much worse than that.

In the initial stages where the seers are agreeing on everything and checking out each other's abilities, there is the possibility that they could agree to cheat. But, a far more subtle problem is that the seers might adjust their decisions to be consistent without realising it. Hence, there would be no concious cheating, but an unconcious cheating. I think it's important to consider the possibility that this unconcious agreement is a potential cause for consistency.

I believe that it's problems like these that cause a lot of the problems between "scientific" and "alternative" types. Imagine this scenario: the three seers spend quite some time together, and without realising it, start to synchronise their predictions. "hmmm... both A and B say the aura is detached here, is there something that I'm missing ...". They perform the experiment, and consistency is found. Then, someone accusses them of coming to an agreement ahead of time, i.e. cheating. The seers know that they haven't (conciously) cheated, and hence their beliefs in their ability to see the aura are strengthened even further, as they're being told that they "must have cheated", but they know that they didn't. Hence they are now *really sure* that they have the power. That such implicit effects can occur seems reasonable given the research into the necessity for double blind (not just single blind) experiments.

In terms of science being imperfect. I work at a uni myself both teaching and researching. About two years ago I decided to "upgrade" my knowledge of stats and experimental design. Hence reading books such as "Methodological errors in medical research". What I'm slowing coming to understand is how poor people's abilities are in designing and interpreting experiments in quite a number of fields. Problems in medical research are better known than most, because of the importance of medical research being correct. Even then I've just been reading how hospitals have been using medicines to prevent heart attacks which the research suggests will increase the survival rate. Except that it isn't, and poor research is quoted as the most likely cause. In other sciences, there is much less pressure to ensure that research is actually accurate, and I personally believe that there are "fields" of science where much, if not most, published research is unreliable.

I'm not claiming that I'm good at stats (yet). But, in my field, I seem to be in the situation where I know that I'm not yet really good enough, but I'm better than (it seems) many other people. This is scary.

Cheers,

Ross-c