Behavioural Science Blog

The study of Hillerbrand and Claiborn is a 2 (level of expertise)x3 (levels of problem structure) design. 17 expert and 15 novice psychologists with good diagnostic abilities read well structured, ill structured or random structured cases. Consistency of the relevance of the information supplied varied between the cases. Relevance was the extend to which information necessary for a particular diagnosis was identified in the case. As dependent measure the accuracy of the diagnose was used, as well as the number of diagnoses, the rationale for the diagnosis and the predictions for future behaviour.

Significant differences between clinicians and novices were on ratings of their knowledge, confidence and perception of the cases’ clarity. The cognitive process however did not seem to differ. Clinical reasoning skills have been found to be present in lay-persons. It might therefore be hard to distinguish an expert from a layperson, as experience is not a sufficient condition for expertise. On the other hand you can wonder is a second or third year student is a lay-person. Certainly the training received so far will have had some impact on the cognitive processes at hand (hopefully).

The task itself is also important to consider: Whereas clinical therapists mostly have a face to face conversation with the client, in experimental studies (such as this one) the interaction is often substituted by a text about a clinical case. This way of data perception might be more appropriate for a undergraduate student, who is used to reading cases, but less used to direct therapeutic contact.

In the study of Brammer participants came from a doctoral program (n=83) or were graduate students of psychology. With the use of a computer program the structure of information exchange between the therapist and the client was simulated. After reading the case the participants could ask questions that were answered by an artificial intelligence that used pregenerated answers. As the participants asked more questions, the answers supplied additional information concerning the case. The diagnosis was sent to a panel of experts who voted the fit a the diagnosis on a 4-pooint Lickert from unlikely to definitive. The possible diagnoses were restricted to major depression, Bipolar II disorder, alcohol abuse, acute stress disorder, borderline personality disorder, nicotine dependence.

The number of diagnostic questions asked were related to correct diagnosis, as well as the level of training and the years of experience. Due to the correlational evidence no causal assumption can be made. For example the number of diagnostic question was also related to the total number of questions. Path analysis revealed there was no direct effect of level training or years of experience, instead the effect was mediated by the numbers of diagnostic questions asked.

Although the task is not a real interaction with full non-verbal communication, the interaction simulated between clinician and client was differed from a pure evaluative task. The clinician was also required to perform a skill.

The third study also compared novice and experts, but introduces intermediate level experienced clinicians as a control group. All 10 cases used as material were prerated by a panel of experts. The DSM diagnosis and the discussion was given for every case. Difficulty was enhanced by cutting back the length of the description by two thirds.

The time it took for the participants to reached a conclusion (indicate the diagnostic classification) was measured. The dependent measurement was the number of correct answers each participant gave.

The pattern that emerged form analysis can best be explained by a third-degree polynomial function. Therapists without any experience start of giving about 40% correct answers. As experience increases a decline in the percentage of correct answers is visible (less then 30% in between the second and third year of inexperience). The curve then mounts again and nears asymptotic level at about 40% correct answers.

Both experts and novice took longer to come to a conclusion than psychologists of intermediate level. This pattern also emerged in the previous study where the number of (diagnostic) questions was related to categorization quality.

Other studies who compared sub experts and experts in their cognitive structure of decision making found that there was only a difference, when time was limited. This might be the case on a face to face interaction, when decisions to pursue a specific topic have to be taken during the interaction. It might well be that the sub experts are equally able to judge the criterions of mental illness in a written form but would fail to achieve this knowledge in a real life interaction, because they do not have the conversational skills necessary.

I believe that the most important factor is to have a clear understanding of the scientific knowledge of a specific illness and be able to get all related information necessary from the client. This might be easier with a well-structured problem than with a difficult case including multiple comorbid illnesses, because the possible causal relations are much not too complex.

This hypothesis is also supported by the findings of the study of Brammer, where the quality of the diagnosis was related to the numbers of diagnostic questions asked. The difference between experts and novices can be found in the process of information gathering, rather than evaluation of the acquired data. Training new therapists might indeed by quite straightforward teaching them to ask more questions and shape the questions in a way so that the answer yields diagnostic value.

I imagine standing in china in front of a coke-vending machine. If I am an expert (and if I know Chinese) I know which button to push to get the coke. The way I see it giving cases to lay-persons is like telling exactly which button to push. Taking out the coke is easy and I do not expect Chinese and Europeans to do that differently. So the most important thing, the general background knowledge, that makes you remember to ask about manic episodes when hearing of depression, is not considered in these studies. The crucial questions to study how clinical psychologists make diagnostic classifications, might not be the ones that are asked, but those that are not asked.