Ego depletion, cognitive dissonance, confirmation bias, intrinsic motivation, loss aversion, etc.—do psychological phenomena like these even exist? If so, in what sense are they real and how can we objectively tell?
I do not want to make this a philosophical discussion, but a scientific-methodological one, based on a recent paper by Seppo Iso-Ahola (2017) from the University of Maryland on the nature of psychological science. While I do not agree with everything he asserts, his claims are definitely worth contemplating.
Do Psychological Phenomona Exist?
According to Iso-Ahola, “a phenomenon is a fundamental psychological process that has theoretically deduced antecedents and consequences and thereby helps explain human cognitions, feelings, and behaviors.”
Such processes are essentially dynamic—they change over time and vary with inner and outer conditions: thoughts, emotions, behaviors, attitudes, social groups, physical environments, the weather, etc. “By their nature, psychological phenomena are subtle, elusive, and often brief in time.”
Now, the scientific method can, by principle (and much to the vexation of atheists), never prove the non-existence of a phenomenon, because it is impossible to take all potentially confounding variables into account. This is particularly true for psychological phenomena: for how could one allow for all the innumerable cognitive processes going on in every living human’s brain at all times?
Thus, Iso-Ahola continues, a psychological phenomenon exists—as a theoretical construct—if it can be logically deduced from a psychological theory.
The role of experiments, then, is not to prove the (non-)existence of psychological phenomena, but only to find out more about their natural (non-statistical) variability, about what causes, facilitates, or inhibits their emergence and what modulates their frequency, intensity, duration, and subjective experience.
Moreover, since pressing fingers on a keyboard (as in most experimental tasks) may produce nothing but artificial laboratory phenomena, psychological phenomena should always be tested both in the laboratory and in real-world environments.
Iso-Ahola also finds it critical to embed empirical studies within a larger psychological theory; for example, embedding experiments on ego depletion within a larger theory of self-control failure, rather than investigating the phenomenon as an isolated laboratory effect. His rationale: Stand-alone effects do not always represent full psychological phenomena.
Are Psychological Phenomena Reproducible?
As Iso-Ahola argues further, “empirical studies are mainly evaluated for their theoretical relevance and importance, and less for their success or failure to exactly reproduce the original findings.” (A problem I see with this perspective is that it seems to promote scientific confirmation bias. What do you think?)
It is impossible, so Iso-Ahola, to strictly reproduce psychological phenomena in laboratory settings. Even minor differences in the lab environment, experimental procedure, measurement equipment, time of measurement, experimenters’ behavior, or wording of instructions can profoundly impact the (non-)appearance of an effect, because psychological phenomena are so dynamic, subtle, elusive, state-dependent, and context-sensitive.
Simply put, reproducibility presumes that psychological phenomena are stable, which they are not; thus, they are not fully reproducible. Furthermore, reproducibility is hindered by sampling and measurement errors as well as by the different conclusions that are often drawn from the same results. Iso-Ahola concludes, “Reproducibility in psychological science is unattainable for conceptual, theoretical, methodological, and statistical reasons.”
As a cognitive scientist who has often had to deal with complex dynamical systems, I think that those are not fundamental problems, but rather the great and intriguing challenges of experimenting with human subjects. Without the basic scientific principle of reproducibility, would we not end up with an empirically informed, yet fundamentally armchair psychology?
Repeated failures to replicate a finding may not prove a phenomenon’s non-existence, but they do raise doubts as to whether it is worth continuing experimental testing of the phenomenon and letting it influence our theoretical understanding of human cognition, particularly if its definition has repeatedly been systematically refined.
If we believe more in a theory than in experimental findings, then our belief is philosophical rather than scientific. This is not a problem in itself, but it is problematic for those who want psychology to be a legit science. My opinion: Let psychologists theorize while cognitive scientists, who have greater methodological diversity, take over the scientific study of the human mind. In the end, we will certainly need the intellectual efforts of both.
The Paper
Iso-Ahola SE (2017). Reproducibility in Psychological Science: When Do Psychological Phenomena Exist? Frontiers in Psychology 8(879), doi: 10.3389/fpsyg.2017.00879.