In evaluating the flaws in this experiment, it’s important to consider potential issues related to causation, correlation, and causal fallacies. Below are some key points to consider regarding the methodology and possible flaws in the study.
1. Lack of Control Over Confounding Variables
Potential Flaw: One key issue in this experiment is the lack of control over confounding variables. Schwartz acknowledges that a few of the sitters were acquaintances of the mediums. This could significantly affect the results. If a medium recognizes the sitter or has prior knowledge of them, this could introduce bias or allow the medium to make educated guesses based on subtle cues.
Analysis: This could introduce a causal fallacy known as "the fallacy of insufficient evidence." If the mediums were indeed picking up on subtle cues or had prior knowledge of the sitters, any apparent "hits" could be due to ordinary psychological processes, such as cold reading, rather than actual communication with the deceased. The study did not sufficiently control for this factor, making it difficult to conclude that the medium's impressions were caused by supernatural forces.
2. Correlation vs. Causation
Potential Flaw: The fact that the mediums were able to provide names and personal information of the deceased doesn’t necessarily imply that they were truly communicating with the dead. A correlation between a medium’s statements and the information about the deceased does not establish causation. The mediums could have arrived at the correct answers through psychological techniques, random chance, or vague statements that could apply to many people.
Analysis: The experiment doesn’t clearly rule out these alternative explanations. The correlation between what the mediums said and the information the sitters provided could simply be a result of psychological strategies such as cold reading or guessing. In these cases, the mediums could seem accurate, but no true causal relationship with the afterlife has been established. Without a way to separate coincidental success from true psychic ability, the experiment’s findings do not necessarily support the claim of supernatural communication.
3. Questionable Methodology (Yes/No Format)
Potential Flaw: The experiment required the sitter to answer only "yes" or "no" to the medium’s statements. This limited format can introduce bias into the results. If a medium makes a broad, vague statement ("I sense a male figure with the letter J"), the sitter might answer "yes" if they think it fits their situation, even if it’s a common name. This can lead to confirmation bias, where the sitter’s own expectations influence the results.
Analysis: This binary answer format can skew the results and may cause both the medium and the sitter to focus on the few instances where the medium's impression happens to be correct, while ignoring failures. This could be an example of confirmation bias, where the sitter and the researcher emphasize the instances that fit the hypothesis while downplaying those that don't. It makes it hard to evaluate whether the medium’s performance is actually extraordinary or simply based on chance.
4. Lack of Blinding or Double-Blinding
Potential Flaw: While the medium couldn’t see the sitter, there’s no mention of whether the researchers (who were observing the experiment) were blinded to the identity or circumstances of the sitters. If the researchers knew details about the deceased (even if they didn’t share this information with the medium), this could inadvertently influence the way they recorded or interpreted the results.
Analysis: This introduces a potential experimenter bias or observer effect, where the researchers' expectations influence the outcome. For example, if a researcher is convinced that mediums are truly communicating with the dead, they may be more likely to interpret vague or non-specific information as "accurate."
5. No Clear Criteria for Success
Potential Flaw: The report does not mention clear criteria for how the success of the mediums was measured. While Schwartz claims he was impressed with the mediums’ performance, it’s unclear what specific metrics were used to assess the mediums' accuracy.
Analysis: Without clear criteria, there’s no way to objectively determine whether the medium's claims were accurate or just coincidental. Without clearly defined standards, it’s easy for the experiment to fall victim to subjective interpretation and confirmation bias, where what appears to be success is just random chance.
6. Sample Size and Statistical Significance
Potential Flaw: The study only involved a small number of sitters (10) and mediums (4). This limited sample size makes it difficult to draw reliable conclusions. Even if the mediums had some correct "hits," it could simply be due to chance or the limitations of a small sample.
Analysis: The small sample size raises concerns about the statistical significance of the results. Without a larger sample or a control group, the results could easily be due to random chance rather than evidence of paranormal phenomena.
Conclusion:
The experiment has several potential flaws that weaken its ability to establish causation between the mediums' performance and actual communication with the deceased. These include issues with confounding variables, lack of control over biases, the correlation vs. causation problem, and methodological issues like limited answer formats and small sample sizes. While the study might show some correlation between the mediums' statements and the information from the sitters, the conclusions about afterlife communication are not well-supported based on the provided methodology.