Predicting Mental Defects: Does the Model or the Expert Win?

/Predicting Mental Defects: Does the Model or the Expert Win?

Predicting Mental Defects: Does the Model or the Expert Win?

By | 2017-08-18T16:59:11+00:00 June 14th, 2014|Behavioral Finance|2 Comments

Clinical intuition and test scores as a basis for diagnosis

  • Klehr, R.
  • Journal of Consulting Psychology, 13, 34-38
  • An online version of the paper can be found here
  • Want a summary of academic papers with alpha? Check out our free Academic Alpha Database!


The present study has compared the ability of a group of 15 experienced clinicians to diagnose mental defectives and schizophrenics on the basis of purely qualitative interpretation of test responses, with the diagnostic results of applying an objective scoring technique to the same material. The material evaluated consisted of the CVS test responses of 36 subjects: 12 schizophrenics, 12 mental defectives, and 12 normals. The results indicate that the performance of the clinicians under these circumstances is comparable to the use of an objective scoring technique, both performances being significantly better than chance. A control group of graduate students did not differ significantly from chance.


In this experiment 36 subjects were chosen. The subjects were broken into 3 categories:

  1. Schizophrenics
  2. Mental defectives
  3. Normal

15 advanced level psychologists who had long experience in clinical testing were selected to evaluate these subjects’ “normality.” As a control, 15 inexperienced graduate students were also asked to evaluate the 36 subjects. There are two data sources:

  1. Results from a standardized intelligence test shown to predict mental defects (CVS)
  2. Experience and intuition.

The results for the experiment are in Table I. On the left axis we have the actual diagnosis and then we have the performance of the different groups (clinical experts, grad students, and the systematic approach (test)).

How does this work?

Take line 1. As a benchmark, any group should get 4/12 correct identifications because the 3 categories are spread evenly among each group of 12. Of the 12 Schizophrenics, experts (labeled Clin.) called 6.6/12, the system got 6/12. Experts said .7/12 Schiz. were defectives; 4.7/12 were normals. The system said 1/12 were defectives; 5/12 were normal. The students were no better than chance. For calling mental defectives, the experts got 8.8/12 and the machine got 10/12. For normals, experts hit 8.8/12, the machine called 7/12. Klehr_1949.pdf - Adobe Reader_2014-06-09_09-50-01 This study highlights a few aspects of models versus experts:

  1. Experienced experts and the model have similar performance.
  2. Inexperienced experts have poor performance.


  • The views and opinions expressed herein are those of the author and do not necessarily reflect the views of Alpha Architect, its affiliates or its employees. Our full disclosures are available here. Definitions of common statistics used in our analysis are available here (towards the bottom).
  • Join thousands of other readers and subscribe to our blog.
  • This site provides NO information on our value ETFs or our momentum ETFs. Please refer to this site.

Print Friendly, PDF & Email

About the Author:

After serving as a Captain in the United States Marine Corps, Dr. Gray earned a PhD, and worked as a finance professor at Drexel University. Dr. Gray’s interest in bridging the research gap between academia and industry led him to found Alpha Architect, an asset management that delivers affordable active exposures for tax-sensitive investors. Dr. Gray has published four books and a number of academic articles. Wes is a regular contributor to multiple industry outlets, to include the following: Wall Street Journal, Forbes,, and the CFA Institute. Dr. Gray earned an MBA and a PhD in finance from the University of Chicago and graduated magna cum laude with a BS from The Wharton School of the University of Pennsylvania.


  1. karl shewmake June 14, 2014 at 11:09 am

    As an expert in medical diagnosis, and known as the doc who will figure it out, I do not think this study applies to the financial world. Among other arguments, the specificity rule would apply. the financial world has too many variables and is so different than the medical.

    • Wesley Gray, PhD June 14, 2014 at 2:49 pm

      Hi Karl,
      Thanks for the input. As an actual expert in the field, your comments related to these studies are very helpful.

      In the context of financial markets, it seems that simple models are likely to BEAT experts (e.g., a low-p/e large-cap stock strategy over the past 30 years would place you among a very select group of mutual fund managers that were able to beat the S&P 500).

      At least the doctors in this study gave the model a decent fight!

Leave A Comment