In the judging procedure, the receiver is taken out of the Ganzfeld state and given a set of possible targets, from which they select one which most resembled the images they witnessed. Most commonly there are three decoys along with the target, giving an expected rate of 25%, by chance, over several dozens of trials.
Analysis of results
The most suspicious pattern was the fact that the hit rate for a given target increased with the frequency of occurrence of that target in the experiment. The hit rate for the targets that occurred only once was right at the chance expectation of 25%. For targets that appeared twice the hit rate crept up to 28%. For those that occurred three times it was 38%, and for those targets that occurred six or more times, the hit rate was 52%. Each time a videotape is played its quality can degrade. It is plausible then, that when a frequently used clip is the target for a given session, it may be physically distinguishable from the other three decoy clips that are presented to the subject for judging.
Hyman wrote these studies were an improvement over their older counterparts, but were not a successful replication of the ganzfeld experiments, nor a confirmation of psi. He concluded the autoganzfeld experiments were flawed because they did not preclude the possibility of sensory leakage.Richard Wiseman published a paper discussing a non-psi hypothesis based on possible sender to experimenter acoustic leakage in the autoganzfeld to account for the results. David Marks has written "Wiseman and his colleagues identified various different ways in which knowledge of the target could have been leaked to the experimenter. These included cues from the videocassette recorder and sounds from the sender who, of course, knew the target's identity... their conclusions provide little reassurance that sensory cueing of the experimenter was in any way substantially blocked."Milton and Wiseman (1999) carried out a meta-analysis of ganzfeld experiments in other laboratories. They found no psi effect; the results showed no effect greater than chance from a database of 30 experiments and a non-significant Stouffer Z of 0.70.Lance Storm and Suitbert Ertel (2001) published a meta-analysis of 79 studies published between 1974 and 1996 and concluded the positive statistically significant overall outcome indicates a psi effect. In response, Milton and Wiseman (2001) wrote the meta-analysis of Storm and Ertel was not an accurate quantitative summary of ganzfeld research as they had included early studies which had been widely recognized as having methodological problems which make it impossible to interpret the results as evidence of a psi effect.Another meta-analysis was conducted by Daryl Bem, John Palmer, and Richard Broughton in which the experiments were sorted according to how closely they adhered to a pre-existing description of the ganzfeld procedure. Additionally, ten experiments that had been published in the time since Milton and Wiseman's deadline were introduced. Now the results were significant again with a Stouffer Z of 2.59.
Positive belief in psi; ESP
Prior psi experiences
Practicing a mental discipline such as meditation
Emotional closeness with the senderWhile there are a number of reasons that researchers avoid special participants and sample only normal populations, these factors are important considerations in future replications of ganzfeld experiment, and may be useful in predicting the outcome of these studies.
Isolation – Richard Wiseman and others argue that not all of the studies used soundproof rooms, so it is possible that when videos were playing, the experimenter could have heard it, and later given involuntary cues to the receiver during the selection process. It could even have been possible that the receiver themselves could hear the video.
Randomization – When subjects are asked to choose from a variety of selections, there is an inherent bias to choose the first selection they are shown. If the order in which they are shown the selections is randomized each time, this bias will be averaged out. The randomization procedures used in the experiment have been criticized for not randomizing satisfactorily.
The psi assumption – The assumption that any statistical deviation from chance is evidence for telepathy is highly controversial. Strictly speaking, a deviation from chance is only evidence that either this was a rare, statistically unlikely occurrence that happened by chance, or something was causing a deviation from chance. Flaws in the experimental design are a common cause of this, and so the assumption that it must be telepathy is fallacious.Writing in 1985, C. E. M. Hansel discovered weaknesses in the design and possibilities of sensory leakage in the ganzfeld experiments reported by Carl Sargent and other parapsychologists. Hansel concluded the ganzfeld studies had not been independently replicated and that "ESP is no nearer to being established than it was a hundred years ago."David Marks in his book The Psychology of the Psychic (2000) has noted that during the autoganzfeld experiments the experimenter sat only fourteen feet from the sender's room. Soundproofing tiles were eventually added but they were designed to "absorb sound not to prevent transmission." According to Marks this was inadequate and no different than using any standard internal wall. The door and door frame were also a possible source of sensory leakage and none of these problems were ever eliminated.Terence Hines wrote in 2003 that the ganzfeld studies could not be said to provide evidence for psi as the alleged evidence disappears as the tightness of experimental controls is increased. As research progresses variables in science become clearer as more studies are published that describe under what specific condition the particular effect can be demonstrated. This is in opposition to the ganzfeld studies. According to Hines, there was "no clear way to obtain results showing any psychic phenomenon reliably" and that "the most reasonable conclusion" was that the effect did not exist and had never existed.In a 2007 review, Ray Hyman wrote that parapsychologists agree they have no positive theory of psi as it is negatively defined as any effect that cannot be currently explained in terms of chance or normal causes. Hyman saw this as a fallacy, as it encouraged parapsychologists to use any peculiarity in the data as a characteristic of psi. Hyman also wrote that parapsychologists have admitted it is impossible to eliminate the possibility of non-paranormal causes in the ganzfeld experiment. There is no independent method to indicate the presence or absence of psi.
Until parapsychologists can provide a positive way to indicate the presence of psi, the different effect sizes that occur in experiments are just as likely to result from many different things rather than one thing called psi. Indeed given the obvious instability and elusiveness of the findings, the best guess might very well be that we are dealing with a variety of Murphy's Law rather than a revolutionary anomaly called psi.
In their book 50 Great Myths of Popular Psychology (2011), Scott O. Lilienfeld and colleagues have written that the ganzfeld being a reliable technique is far from being resolved. They concluded that ESP has not been successfully demonstrated in experiments for over 150 years so this is hardly encouraging.In a 2013 podcast, Brian Dunning reviewed the flaws of the ganzfeld studies and came to the conclusion the technique had failed as evidence for psi and interest in ganzfeld has declined.
It now appeared that in one session – number 9 – the following events had taken place.
Sargent did the randomization when he should not have.
A 'B' went missing from the drawer during the session, instead of afterwards.
Sargent came into the judging and "pushed" the subject towards 'B'.
An error of addition was made in favour of 'B' and 'B' was chosen.
'B' was the target and the session a direct hit.
This article, along with further criticisms of Sargent's work from Adrian Parker and Nils Wiklund remained unpublished until 1987 but all were well known in parapsychological circles. Sargent wrote a rebuttal to these criticisms (also not published until 1987) in which he did not deny what Blackmore had observed, but argued that her conclusions based on those observations were wrong and prejudiced. His co-workers also responded, saying that any deviation from protocol was the result of “random errors” rather than any concerted attempt at fraud. Carl Sargent stopped working in parapsychology after this and did not respond "in a timely fashion" when the Council of the Parapsychological Association asked for his data, and so his membership of that organization was allowed to lapse.Writing for Skeptical Inquirer in 2018, Blackmore states that Sargent "deliberately violated his own protocols and in one trial had almost certainly cheated." Psychologists reading Daryl Bem's review in Psychological Bulletin would "not have a clue that serious doubt had been cast on more than a quarter of the studies involved" Sargent and Chuck Honortons. When Blackmore confronted Sargent, he told her "it wouldn't matter if some experiments were unreliable because, after all, we know that psi exists". Blackmore also recounts having a discussion with Bem at a consciousness conference where she challenged him on his support of Sargent and Honorton's research, he replied "it did not matter". Blackmore writes, "But it does matter. ... It matters because Bem's continued claims mislead a willing public into believing that there is reputable scientific evidence for ESP in the Ganzfeld when there is not".