Paul E. Meehl

"My thesis in a nutshell: "There is no convincing reason to assume that explicitly formalized mathematical rules and the clinician's creativity are equally suited for any given kind of task, or that their comparative effectiveness is the same for different tasks. Current clinical practice should be much more critically examined with this in mind than it has been.""

"Because physical theories typically predict numerical values, an improvement in experimental precision reduces the tolerance range and hence increases corroborability. In most psychological research, improved power of a statistical design leads to a prior probability approaching ½ of finding a significant difference in the theoretically predicted direction. Hence the corroboration yielded by "success" is very weak, and becomes weaker with increased precision. "Statistical significance" plays a logical role in psychology precisely the reverse of its role in physics. This problem is worsened by certain unhealthy tendencies prevalent among psychologists, such as a premium placed on experimental "cuteness" and a free reliance upon ad hoc explanations to avoid refutation."

"a fairly widespread tendency to report experimental findings with a liberal use of ad hoc explanations for those that didn't "pan out." ... especially tempting in the "soft" fields of (personality and social) psychology, where the profession highly rewards a kind of "cuteness" or "cleverness" in experimental design, such as a hitherto untried method for inducing a desired emotional state, or a particularly "subtle" gimmick for detecting its influence upon behavioral output."

"A zealous and clever investigator ... undismayed by logic-of-science considerations and relying blissfully on the "exactitude" of modern statistical hypothesis-testing, has produced a long publication list and been promoted to a full professorship. In terms of his contribution to the enduring body of psychological knowledge, he has done hardly anything. His true position is that of a potent-but-sterile intellectual rake, who leaves in his merry path a long train of ravished maidens but no viable scientific offspring."

"Owing to the abusive reliance upon significance testing—rather than point or interval estimation, curve shape, or ordination—in the social sciences, the usual article summarizing the state of the evidence on a theory (such as appears in the Psychological Bulletin) is nearly useless."

"the null hypothesis in the life sciences is almost always false—if taken literally— in designs that involve any sort of self-selection or correlations found in the organisms as they come, that is, where perfect randomization of treatments by the experimenter does not exhaust the manipulations. Hence even “experimental” (rather than statistical or file data) research will exhibit this if interaction effects involving attributes of the persons are studied. Consequently, whether or not the null hypothesis is rejected is simply and solely a function of statistical power."

"Most so-called “theories” in the soft areas of psychology (clinical, counseling, social, personality, community, and school psychology) are scientifically unimpressive and technologically worthless. [...] Most of them suffer the fate that General MacArthur ascribed to old generals—They never die, they just slowly fade away."

"It is simply a sad fact that in soft psychology theories rise and decline, come and go, more as a function of baffled boredom than anything else; and the enterprise shows a disturbing absence of that cumulative character that is so impressive in disciplines like astronomy, molecular biology, and genetics."

"In the developed sciences, theories tend either to become widely accepted and built into the larger edifice of well-tested human knowledge or else they suffer destruction in the face of recalcitrant facts and are abandoned, perhaps regretfully as a “nice try.” But in fields like personology and social psychology, this seems not to happen. There is a period of enthusiasm about a new theory, a period of attempted application to several fact domains, a period of disillusionment as the negative data come in, a growing bafflement about inconsistent and unreplicable empirical results, multiple resort to ad hoc excuses, and then finally people just sort of lose interest in the thing and pursue other endeavors."

"As is well-known (and not disputed by Popper), when we spell out in detail the logical structure of what purports to be an observational test of a theoretical conjecture T, we normally find that we cannot get to an observational statement from T alone. We require further a set of often complex and problematic auxiliaries A, plus the empirical realization of certain conditions describing the experimental particulars C [but] in social science the auxiliaries A and the initial and boundary conditions of the system C are frequently as problematic as the theory T itself."

"Isn’t the social scientist’s use of the null hypothesis simply the application of Popperian (or Bayesian) thinking in contexts in which probability plays such a big role? ... since an output variable such as adult IQ, or academic achievement, or effectiveness at communication, or whatever, will always, in the social sciences, be a function of a sizable but finite number of factors... Putting it crudely, if you have enough cases and your measures are not to-tally unreliable, the null hypothesis will always be falsified, regardless of the truth of the substantive theory."

"a theory that has seven facts for it and three facts against it is not in good shape, and it would not be considered so in any developed science."

"Sir Ronald has befuddled us, mesmerized us, and led us down the primrose path. I believe that the almost universal reliance on merely refuting the null hypothesis as the standard method for corroborating substantive theories in the soft areas is a terrible mistake, is basically unsound, poor scientific strategy, and one of the worst things that ever happened in the history of psychology."

"You may say, “But, Meehl, R. A. Fisher was a genius, and we all know how valuable his stuff has been in agronomy. Why shouldn’t it work for soft psychology?” Well, I am not intimidated by Fisher’s genius, because my complaint is not in the field of mathematical statistics, and as regards inductive logic and philosophy of science, it is well-known that Sir Ronald permitted himself a great deal of dogmatism."

"Meehl was an American psychologist who studied the successes and failures of predictions in many different settings. He found overwhelming evidence for a disturbing conclusion. Predictions based on simple statistical scoring were generally more accurate than predictions based on expert judgment. A famous example confirming Meehl’s conclusion is the “Apgar score,” invented by the anesthesiologist Virginia Apgar in 1953 to guide the treatment of newborn babies. The Apgar score is a simple formula based on five vital signs that can be measured quickly: heart rate, breathing, reflexes, muscle tone, and color. It does better than the average doctor in deciding whether the baby needs immediate help. It is now used everywhere and saves the lives of thousands of babies."