|
Tests
of Astrology and $1Million
In his article
on this site, The Researchers Researched, Dennis
Elwell criticised the James Randi Educational Foundation's 'One Million
Dollar Paranormal Challenge'. (For the homepage of the challenge, go to:
www.randi.org/research/index.html
).
James Randi
got in touch about this, and Dennis Elwell chose to defend his ground.
The following discussion ensued. Some interesting points were also made
in a discussion on the Skyscript
forum (in the 'Philosophy and Science' area) and at the foot of this
page I've included a contribution from Deborah Houlding, originally posted
on that forum.
_________________________________________________
From
James Randi:
A lie on
your page....
Dennis Elwell
states, re my million-dollar challenge, that "So, only eyewitness
evidence is allowed." That is a lie, not even suggested by the terms.
The fact that the test is designed so that no "judging" is required,
means that the results can be seen without any judging process. For example,
if persons are asked to match or identify their horoscopes, no judging
is required; they are either right, or wrong.
This is
the sort of canard that the astrologers have been circulating to avoid
accepting the challenge. If that is untrue, then I expect astrologers
to now accept the challenge. That will result in another long silence
from them....
[Posted 2 January 2004]
_________________________________________________
From
Dennis Elwell:
James Randi,
typically forthright, uses the word lie. A lie is an intentionally false
statement, which implies I am trying to mislead, and that is not the case.
I was characterising his approach to evidence and validation across the
range of phenomena, not particularly astrology.
When it
comes to proving anything there is a spectrum, ranging from absolute certainty
to the probable. There is laboratory proof and there is courtroom proof.
Faint hearts and bigots take refuge in the simplistic, the reductionist.
Plenty of people would subscribe to the view that "If you can't bite
it, it don't exist." I put Randi in this category.
On his website
he says: "All tests must be designed in such a way that the results
are self-evident, and no judging process is required." He further
says the challenger must demonstrate the claimed ability under satisfactory
"observing" conditions. And again: "We are only interested
in an actual demonstration." In his current complaint he once more
waxes ocular, insisting that "the results can be seen".
To appreciate
how abysmally naive this is, consider extending the same yardstick to
respectable phenomena. Forget the supernatural, how does the natural fare
in Randi's world? Perhaps the most pervasive knowledge of our time is
Darwinian evolution, and yet if Randi offered his million to anybody who
could demonstrate it under the conditions he seeks to impose I guarantee
he would keep his money. There is much in science that cannot be eyeballed,
but which depends on judgment, inference, deduction, evaluation. Even
Nobel laureates do not have to measure up to his impossible standards.
It is always
interesting when confirmed sceptics like Randi propose tests they imagine
might prove or disprove astrology. Helpfully he suggests that if persons
are asked to match or identify their horoscopes no judging would be required,
because they are either right or wrong. Really?
Let us unpick
this superficially plausible proposal. What does he mean by "horoscope"?
Webster gives two meanings. There is "the configuration of the planets,
esp. at the time of a person's birth, from which astrologers predict the
future." And the other meaning is "a diagram representing the
configuration of the stars and planets at any given time." So do
we present test subjects with a dozen different charts, all circles and
squiggles, and ask them to pick their own?
What Randi
means of course is that they are given an interpretation of the charts.
Question is, whose interpretation? The answer depends on making a judgment,
and we all know what a tricky business that is. However, suppose assorted
interpretations, from whatever source and of whatever merit, are handed
to the test subjects. Here comes another judgment -- they are tasked with
identifying their own. You might think this is as easy as recognising
your face in the mirror. After all, you know what you are like. But is
it beyond doubt that individuals actually possess the self-insight required?
How good are people at recognising personality profiles of themselves,
prepared by non-astrological means? Does our self-image infallibly correspond
to the reality? The answer is no.
So this
test superimposes two layers of uncertainty, and between them the truth
could easily slip through. If judgment is to be excluded from results,
as Randi insists, then it must surely be excluded from the test design.
On this
matter of judgment, anyone who studies the terms and conditions of his
million-dollar challenge will realise that Randi presumes to be judge
and jury in his own court. If you are tempted to take the challenge seriously,
remind yourself that Randi is not a scientist but an illusionist, a smoke-and-mirrors
man. Conjurers raise lying to an art form. He is a showman, and his posture
as the guardian of rationality is as entertaining as anything he has ever
done on stage. Like his hero Houdini his speciality was escapology, and
it is fair to speculate how an escape artist would react to being put
in a tight corner by some importunate challenger.
[2 Jan 2004]
_________________________________________________
From
James Randi:
Rather than
plough through all the spurious "reasoning" given here, I'll
briefly describe a test we did of astrology, and how we have tested some
dowsers.
First, in
accord with our standard protocol, we asked the astrologer what he could
do, under what conditions, and with what accuracy. He claimed that if
we supplied him with 12 subjects, with each one born under a different
"sign," he could identify their individual signs -- with 100
percent accuracy, by simply questioning them about their personal preferences
and tastes. He only required that proof of their birthdate would be available
to him in case of any conflict arising.
We obtained
the 12 subjects for him, and went to extra pains to be sure that each
one was born well "centered" in their sign. The astrologer spent
about 10 minutes questioning each person, with a witness present to be
sure that they were not asked questions that might point to their birth
dates. The questions consisted of inquiries regarding their taste in movies,
in clothing, and politics -- among other such subjects. The astrologer
-- following our instructions -- gave each one of the subjects a sealed
piece of paper upon which he had written the sign to which he had assigned
them.
When the
experiment was concluded, we asked the astrologer if he was satisfied
with the conditions and the conduct of the tests. He said that he was,
and repeated that he was very sure of his accuracy.
He was correct
on one of the 12 persons.
Note: this
was a test designed by the subject himself. We did not impose this task
on him. It was his choice, and his design, in all respects. Before and
after the test, he agreed that conditions were ideal, and that he had
no objections and was secure in his conviction that he had been successful.
I'll add that this particular applicant, for the first year following
the test, made no objection nor did he offer any excuses for his failure.
Shortly after that, however, he began claiming that we had deceived him
in regard to the driver's licenses he'd been shown -- but following our
invitation to repeat the test, satisfying this objection of his, he declined
to be tested again.
We've tested
many dowsers, since dowsing is a major claim made on the JREF prize. In
every single instance, we have allowed -- in fact, encouraged -- the applicant
to design his/her own test. Usually, they propose to be tested for locating
a substance or an object concealed in one of 10 paper cups, and most of
them claim that they will have 100 percent success, with a few claiming
only nine out of 10. In every single one of these tests, the dowser has
obtained either one or two successes out of 10.
Again, note:
the applicants themselves designed the tests. Always, they supplied the
materials -- the object or the substance -- that they prefer to use, and
in many cases they even supplied the containers that would be used. The
results -- in common with all other tests we conducted -- are perfectly
evident, and require no judgment or decision to be made. I cannot understand
how Mr. Elwell has such a difficult time understanding this aspect of
the JREF challenge. It's not at all difficult to design such a test, and
it should not be difficult to understand why it is designed in that fashion,
nor how. There are no decisions to be made, no evaluations, and no uncertainty
about whether or not the applicant has been successful. We very much prefer
what's known as a "forced-choice" design, which allows no uncertainty.
And remember, the participants MUST AGREE ON THE PROTOCOL, IN ADVANCE!
Mr. Elwell,
in his response, has invented all sorts of circumstances and hypothetical
situations in which applicants might find themselves, but these are not
conditions that have ever applied. I agree with him that the subject of
a horoscope interpretation might not be the ideal person to match up actuality
with that interpretation. This would be a condition that I would not accept;
for one thing, it provides the applicant was too many excuses for a failure.
However, since this has never occurred -- despite Mr. Elwell's hope that
it had occurred -- his objection is entirely hypothetical.
I thank
you for the opportunity to appropriately respond to Mr. Elwell. If this
opportunity were presented to me more frequently, I believe there would
be less misunderstanding of the true nature of the JREF challenge -- which
has been seriously misrepresented so many times over the years. Our challenge
is a genuine one, it is serious, and those who we are tested are treated
fairly, properly, and in a strictly rational and reasonable manner.
[3 Jan 2004]
_________________________________________________
From Dennis Elwell:
It
is perhaps a misfortune for James Randi's credibility that he is a professional
illusionist. Someone more suspicious than myself would demand to be present
at all stages of the tests he describes, studying each in forensic detail.
Uppermost in his mind would be the possibility that Mr Randi has a preference
for not parting with his million and, more important, wishes not to see
his rationalist prejudices come unstuck. He would also be aware that the
Amazing Randi (his proud boast) might have amazing ways of preventing
such an unwelcome outcome.
As someone with an amateur interest in magic I am unimpressed by the continual
harping on applicants designing their own test. I dare say in his own
performances Mr Randi made great play of giving the audience a free choice
of handcuffs, locks, ropes, and suchlike. The challenge to the magician
(assuming that Mr Randi were to be wearing that hat at the time) would
be to take the proposed test and see where it might be tweaked. With most
experimental designs there is some weakness to be exploited.
Consider the test of the astrologer, which is wrongly described as a test
of "astrology". One astrologer of unknown ability and experience,
but apparently not clever enough to first run a pilot experiment on himself,
is not a test of astrology, only of his claims. People in all walks of
life have inflated ideas of their talents.
If I were of malevolent intent, I could load this test against the possibility
of success. Knowing the attributes of each sign I would make sure I chose
subjects who were atypical. I do not say this was done of course, only
that tests are not automatically infallible simply because they have been
chosen by the claimant, and the terms agreed in advance. A critical study
of these JREF claims of failure, by an investigative journalist for instance,
would include the claimant's account of what did and did not happen. (But
the puzzled claimant might not always realise why the test failed.)
To have Mr Randi's version of what happened, while undoubtedly useful,
may not be conclusive. There is a mysterious reference here to driving
licences the astrologer had been shown. The plot thickens.
Reading the account of the dowsing experiments I am astonished that so
many sincere exponents should have volunteered to make themselves look
foolish. We are asked to believe that every one of them, before making
their confident claim, had failed to self-test in advance. Here again
Mr Randi's assertions call for a proper investigation, getting both sides
of the story, and independent witnesses. This is unlikely to happen because
Mr Randi, with admirable misdirection, has convinced everybody at the
outset that it is the claimants on whom the suspicion falls.
Finally, Mr Randi says my objection is entirely hypothetical. I do not
see why a hypothetical test of astrology, as proposed by himself, should
not meet with a hypothetical response.
[5 Jan 04]
_________________________________________________
From
James Randi:
(The first
paragraph is excerpted from Dennis Elwell's comments, immediately above)
"Consider
the test of the astrologer, which is wrongly described as a test of "astrology".
One astrologer of unknown ability and experience, but apparently not clever
enough to first run a pilot experiment on himself, is not a test of astrology,
only of his claims."
Exactly!
And that's precisely what we were testing -- his claims -- which he defined
exactly, and for which we tested! Had we been testing "astrology,"
it would have been necessary for us to assemble a group of reputable astrologers
and obtain from them, first, an authoritative definition of astrology,
second, a decision on what astrology should be able to do, and third,
how that could be tested. Mr. Elwell chooses not to understand this, though
I'm sure that he does. He has a great deal invested in this fruitless
argument, since if our challenge is genuine -- and it most certainly is
-- he doesn't have a leg to stand on. Furthermore, the man we tested was
a full-time professional astrologer of some repute, and he assured us
that in line with the "caveat" printed at the end of our challenge,
he had tested this particular procedure several times before making his
application. Mr. Elwell is simply wrong in his comments.
Since I
wasn't able to spend several hours describing to Mr. Elwell exactly how
the astrology test took place (NOT -- please note -- a "test of astrology,")
I'll just say here that the subjects were chosen from a group of approximately
120 people in a studio audience. We merely asked individuals who were
"in" each individual sign, in turn, to hold up their hands,
and the nearest one was chosen to be a test subject. The only provisions
we had were that they had to be adults, not children, and if their birthdate
was unsatisfactory due to being too far toward the beginning or end of
the "sign," we took the next-nearest person. I'll add that I
had nothing to do with the choice of the subjects, since not only was
I busy with other matters, but it was obviously not fair for me to have
been part of that process.
The fact
that Mr. Elwell refuses to grasp is that we really do know what we're
doing, and that threatens his world-view rather seriously, it seems.
Again, I
don’t have the time to go through every detail of this man's delusions
concerning the JREF challenge. We've been perfectly forthright about this
matter, and if some choose to willingly misunderstand and/or misinterpret
it, we can't go about trying to educate them, continually. I’ve run out
of patience with this man, and I assure you that it would be a big mistake
to assume that I could not handle each and every one of his picayune objections;
it should be obvious by now that he is being willfully ignorant of the
reality represented by the JREF challenge. I'll enter into no more discussions
with him. I'm simply too busy handling responsible questions, applications,
and comments from responsible people.
Mind you,
Mr. Elwell, if you wish to apply for the JREF prize, that's an entirely
different matter. That may be the only way that you'll get educated. And
the million dollars is not my million dollars, sir, it belongs to the
foundation I represent, and it cannot be used for any purpose other than
as prize money in the challenge. Ah, but you knew that.
[7 Jan 04]
_________________________________________________
From
Dennis Elwell:
I
had to smile at Mr Randi's assertion that I have a great deal invested
in this argument, because it threatens my world view. Any visitor to www.randi.org
will be able to judge which of us has the stronger vested interest. He
is running a debunking factory, inviting not only moral but financial
support. After a lifetime spent in this endeavour he would look pretty
foolish if he were now to be proved wrong on some major point of contention.
Indeed, if I were in his position I confess I should have long ago resolved
that there was no way I could allow the prize to be won. A climb down
of that magnitude would not only be a betrayal of the confidence in me
of my supporters, but a propaganda boost for every loony coterie in the
land. If some claimant was shaping up to be successful I should even feel
justified in a little chicanery, exploiting the wriggle room my rules
allowed. After all, if something can't reasonably happen, it obviously
hasn't, therefore it follows I must have overlooked some vital flaw in
the test.
Mr Randi is nothing if not forthright, but what I miss is transparency.
He demands that test results shall be transparent, but I am not sure this
is reciprocated on his part. For example, he states in his literature
that the million dollars is held by an investment firm, and that validation
of this account and its current status can be obtained on request. I had
heard that the bulk of the million was in fact in the form of pledges
from well-wishers, and anxious to correct or confirm this important point
I emailed him thus: "In your million dollar challenge application
you offer to validate the account with a New York investment firm. I should
be grateful for this information, which I trust contains contact details."
His reply simply read: "Follow the instructions. What's so difficult
about that....?"
Transparency?
The astrologer he tested seems to have had belated second thoughts about
the transparency of what actually happened. He raised the business of
the driving licenses. We are not told what the objection was, but Mr Randi
thought it sufficiently substantial that he offered to satisfy it in another
test. Something else puzzles me. The most natural procedure, at the end
of the interview with each subject would be to say to them "I think
you're a Virgo" (or a Pisces, or an Aries, or whatever). They would
then say yes or no, and the witness would keep score. But instead we hear
about sealed pieces of paper, a seemingly unnecessary complication. I
should like to ask this astrologer for his version of events. Perhaps
Mr Randi should be prepared to open his files, with the applicants' permission
of course, and a guarantee of anonymity.
There has to be an ever-present awareness that presiding over these tests
are not scientists, but a conjurer. I am as solicitous of Mr Randi's reputation
as he is of my education, and to protect himself from unfounded suspicions
I would suggest that applicants are invited to bring their own witness
(ideally a member of the Magic Circle) and that video cameras are present
so that the scene can be revisited. I wonder whether the dowsers, surprised
that their object or substance was not under the cup they thought it was,
vaguely recalled that they had been similarly mystified by conjurers with
cups and strangely migrating balls, or when they were challenged to "find
the lady".
No doubt the tests were fair, but they have to be seen to be fair, and
the applicants themselves may not always realise that the tests they have
suggested are other than watertight. I myself have set public tests for
psychics, but by proposing my own experimental design to see if it would
be acceptable to them. One experiment was double-blind, a nicety that
might not have occurred to psychics but which added weight to the outcome.
It is a matter of huge importance that investigations of the paranormal
are properly conducted, because what is at stake here is public confidence
in science itself. People have had peculiar experiences at first-hand,
or have heard of them from others, and there is a widespread conviction
that all the answers are not in yet. Misgivings have been expressed that
the exploration of this area should have been in the hands of magicians
rather than scientists. In an article in New Scientist sociologist Henry
Collins warned against giving nonscientists control over scientific procedures.
He was referring specifically to conjurers. Interestingly there is a high
proportion of conjurers in CSICOP, the sceptics organisation, of which
Mr Randi was a founding father.
Insights into CSICOP and Mr Randi are to be found in an article by George
P Hansen (CSICOP and the Skeptics) at www.tricksterbook.com.
Mr Randi suggests I should apply for the million dollar prize. Since I
have long believed the prize was unwinnable, and have said so, this would
be a strange hypocrisy. But I might just challenge Mr Randi, using my
own rules and independently audited results.
[14 Jan 04]
_________________________________________________
From Deborah Houlding:
(for more on Deborah, see her website www.skyscript.co.uk)
Garry
raised this question in the Skyscript
forum:
“Is
it in the nature of astrology for it to 'perform' on command in order
to win $1M for an astrologer? Personally I'm very much inclined to think
not.”
I
would never expect astrology, as I understand it, to do this. Astrology
gives me evidence of connection to a spiritually animated environment
that responds to our needs. From what I have experienced, the greater
the need -- the more earnest we are in our attempts to find a way to manipulate
difficult situations with consideration for others, as well as ourselves
-- the more directly it responds. I am not against the view that natural
astrological factors are capable of proof, but I don’t believe that anyone
has yet found an efficient method to isolate those factors, in a way that
responds to independent testing. Rather than being disheartened by that,
I’m comforted by the knowledge that there is intelligence inherent in
astrology that expects us to raise ourselves up to it.
The
exchange between James Randi and Dennis Elwell is lively and interesting,
and I lie somewhere in the middle where I recognise both viewpoints as
being equally sincere. I first read Elwell’s detailed response to ‘The
Researchers’ and think that, as ever, Elwell deserves to be congratulated
for being prepared to challenge the critics on their own terms. He does
an admirable defence job in demonstrating that if their interest is to
deride astrology’s claims, then he can just as easily deride theirs. And
nobody does that better than Dennis Ewell.
But
what we seem to end up with is a never-ending stalemate of disproof. These
are really academic mind games that circulate around the shallow perimeters
of astrology. The problem is that those shallow perimeters attract all
the attention. A great deal of the argument revolved around the issue
of whether it is possible to identify our Sun-signs through knowledge
of personality traits. How futile can we get? Yet look how many people
are out there trying to make a quick buck on the back of this simplistic
nonsense. James Randi gave the example of one astrologer who tried to
claim the prize this way. If I had a million dollars to spare I would
love to be the one saying “what rubbish – let’s see if you can prove astrology
that way – put up or shut up”.
I
don’t have a problem with the criteria of James Randi’s challenge at all.
Astrologers wouldn’t even feel the need to defend themselves against the
fact that the prize has never been claimed were it not for the way we
allow astrology to be presented to the public, as if it is the simple
solution to all of life’s mysteries: just pay enough money and astrology
can tell you anything you want to know. It is because we have allowed
astrology to be marketed as a black and white subject that our critics
respond with cynicism, ridicule and challenges comprising of black and
white tests.
Unlike
Elwell, I don’t feel the need to defend astrology. I suppose I feel that
the astrology I most admire and respect is above and beyond the academic
attacks. But for the rest of it, there is much that I am happy to have
criticised and in fact I believe that a certain criticism plays a useful
part in deflating the sometimes irrational ego of the astrological community
at large. We are too prone to present astrology in an exaggerated and
over-simplified light. Dennis Elwell is quite right when he states that
one of the problems of astrology is that it is client-driven and panders
to the self-absorbed. There is an element within the accepted image of
astrology today that I am embarrassed to be a part of, and I am sure I
am not the only one. Rather than rushing to the defence of astrology every
time it is attacked, why don’t astrologers act collectively to enforce
a more realistic and sophisticated understanding – that astrology has
great potential for benefit, but is also riddled with great potential
for misuse, misunderstanding and oversimplification?
Our
main organisations and associations only seem interested in ways to make
astrology more popular, but to whose benefit? (As an example I am thinking
of the recent suggestion, promoted at the last AA conference, to have
astrology redefined as a religion in order that it receives less stringent
controls over its presentation on TV. Well, no thank you - in my opinion
the public have enough open doors to find their way through to astrology
if they are interested without having to contort astrology into something
it’s not just to get a few celeb astrologers a bit more air time).
Randi’s
challenge seems to have been accepted as a criticism against astrology
whereas in fact I don’t see it that way. It is a simple statement that
if anyone can prove that astrology is unquestionably verifiable through
a simplistic application of some of its established laws, that person
will be the first to have actually achieved this. It does all astrologers
some good to remember that. We shouldn’t treat astrology as infallible
and independent of the need for great consideration and judgement, or
be unaware of the great mystery that remains in this subject. Those that
take astrology too literally will sooner or later face a massive crisis
when rules they thought were simple, constant and reliable prove not to
be. Such occasions call us to stretch our consciousness up to the next
level of understanding. As Elwell has commented on many occasions, if
the case for a reliable astrology had ever been proven we would all be
driving around in limos and drinking champagne in our tea breaks.
Ultimately
I think it is a healthy state to have astrology surrounded by critics,
because at the moment they seem to be the only ones that are acting to
stop some of the outrageous claims that are being made in the name of
‘commercial astrology’ becoming ever more popular. So I don’t mind if
they are reminding us of scientific limitations, the difficulties of proving
astrology statistically or observably, or taking a stance on moral obligations.
(I found myself agreeing with Pope John Paul II’s New Year warning two
years ago when he warned against rushing out to diviners to foresee what
the future holds, instead of developing an attitude that seeks to use
well the time that each one of us has available, motivated by understanding
and a commitment to being responsible on a daily basis.)
Astrology
has its place and within that place it deserves – and gets – respect.
Whenever it becomes over-exposed or simplified to make it ‘easier’ and
more popular, the claims of what astrology is capable of naturally become
more extreme, creating an ever-increasing scope for critics to (justifiably)
come in with their attacks.
Most of them are only doing what I’d probably do myself if they weren’t
there doing it for me.
[16 Jan 04]
_________________________________________________
From
Suitbert Ertel:
(for more on Prof. Ertel, see his homepage)
My
concern is not whether astrology would ever be able to win Randi's prize.
I only doubt whether Elwell's concluding remark is true that this prize
is unwinnable at all. It all depends on whether Randi accepts statistical
evidence for anomalistic claims. Small effects interspersed with much
error variance may require a great number of observations until grains
of gold are retrieved from dust.
A decade or so ago, I corresponded with Randi about whether he would accept
statistical evidence for Gauquelin-type planetary effects from a new sample
of famous people, and he seeemed to consider that seriously. The matter
got stuck because too many technical problems are involved due to necessary
controls at every stage of such research. The point is that Randi accepts,
or at least accepted at that time, statistical evidence in principle.
More straightforward ideas might be submitted to him for consideration.
Psi effcts such as excess hit numbers in symbol card guessing tests cannot
be safely demonstrated on command by individuals. Even psychics like Uri
Geller might fail when tested under harsh control, which is what will
probably occur at the James Randi Educational Foundation in Florida. But
I would dare predict that, say, a group of 20 psi-gifted subjects selected
by valid procedures, if tested simultaneously under Randi's supervision
for 2 hours, would come up with highly significant deviations from chance.
Individual hit scores may fluctuate, but larger groups should display
sufficiently stable effects, as far as my own experimental observations
go. Intricate procedural demands must be met with, sure, but they would
appear manageable.
Would such group test procedure also work with astrological judgments?
Would a larger selection of psi-gifted astrologers interpreting natal
charts make hits above chance? I wouldn't start off with stars, but with
more terrestrial phenomena which have already proved to be replicable
under statistically appropriate conditions.
[20 Jan 04]
_________________________________________________
Update
- March 04
Following
his last post above, Prof Ertel has been in discussion with James Randi
about a test which, he is convinced, might enable him to win the $1M Paranormal
Challenge.
The test would involve PSI phenomena being demonstrated by a group of
psi-gifted children that Prof. Ertel would select from a larger population
using a test that he developed (in which, incidentally, children's psi
performance was twice as strong as the adults' performance).
If James Randi agrees with his project in principle, Prof. Ertel would
try to get a German TV channel interested in televising the tests as part
of a series of programmes with a betting theme. So although things may
seem to have gone quiet, there are some very interesting developments
behind the scenes. More news will be posted when available.
[21 Mar 04]
_________________________________________________
|
