Behavioural Science Blog

The Science of Human Behaviour

The Downside of the Glorious RCTs

with 2 comments

RCTs are great and every empirical scientist loves them. Everyone? No there is a small village in the south of France…Well to be honest I believe that RCT’s are just a hype. Of course they were great in reducing a lot of statistical problems and have helped psychology to become a “respected science” and earn a place in between mother philosophy and father medicine. In the retrospective the big advances in psychology have all been made by individuals that used single case designs (Freud, Piaget, Skinner…). Why is that the cause?

As much as RCTs can tell us about the statistical differences between groups they are not very good at telling us what the processes at hand are. RCT are also often conducted in special settings and with high treatment fidelity and lots of resources, something the “real world” often does not have to offer. All of this makes it complicated to derive any practical treatment value from them. If there is some practical value to it, one will have to search for it by reading the whole article and giving a few hours of thought to it.

Scientists write for scientists (or rather they write to please their peer-reviewers). This is a mindset quite far from the therapist that lives in a world in which all the factors that are excluded from RCT (for obvious reasons), like: economical and technical problems, comorbidity, lack of resources, converge and interact with each other.

The mindset of the therapist often does not entail thoughts of simple causality of X influencing Y, but of a multidimensional system in which all factors interact with each other. Thus the information value derived from RCTs might seem huge for the scientists, but low for the therapist that needs support in his treatment decisions.

Written by Martin Metzmacher

December 5, 2008 at 2:00 pm

2 Responses

Subscribe to comments with RSS.

  1. I like your ideas very much in this piece. I see your point that RCTs are oftentimes so controlled that they aren’t applicable outside the lab. For example, Marsha Linehan developed Dialectical Behavior Therapy (DBT) for treating border personality disorder (BPD). According to her RCTs, the therapy is very effective for treating these patients, but in reality, is it useful? This is still up for debate. How do you ensure that the therapists are trained accurately? Where does the funding for her research come from, and is this applicable in the real world? What is considered progress for someone with BPD? All these questions make this therapy controversial.

    Even more pertinent than my first comment, isn’t there an ethical complication that being trained in this “gold standard” of therapy for BPD is only done through Marsh Linehan at the institute, Behavioral Tech, which she founded? If this therapy is really cutting edge in treating clients with BPD, shouldn’t it be taught at the universities that train our clinicians?

    … just something to think about…


    September 26, 2009 at 6:53 pm

  2. Hi Christina,

    thanks a lot for your lengthy comment. Indeed very interesting thought… I am pretty sure that the RCT as the golden standard will disappear the same way that alpha=5% is currently not regarded as “proof” anymore.

    I am really looking forward to publishing your interview!

    Martin Metzmacher

    September 27, 2009 at 5:52 pm

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: