The Allure of Evidence-Based Practices: 3 Things You Must Consider

It’s easy to understand the allure of evidence-based practices. As counselors, we want to use techniques and approaches that are going to help our clients. We want to help people get better in the most effective manner possible.  Insurance companies and other third-party payor sources want counseling done in the most efficient way possible. And of course none of us want to do anything that might actually harm someone.

I just got back from the first ever national conference for Veterans Treatment Courts called Vet Court Con. There was plenty of talk about evidence-based practices and what works and what doesn’t when working with this population.

We heard from Doug Marlowe about the importance of evidence-based practices and what the research says about best practices in alternative court programs. I also learned a lot from VA psychologist Brian Meyer about the application of these practices in the field. Like most good conferences, I came home with some new information but more importantly, with a greater understanding of what I don’t know.

The idea behind evidence-based practices is pretty simple and sounds promising enough. You find the problems your clients are facing then open your tool box of evidence based practices to apply the appropriate proven intervention for the problem. You apply the intervention as prescribed and your client gets better. Of course it’s not that simple, but you can see the attraction.

As a clinical director, I’m a big fan of evidence-based practices. The practices give a framework of what should be taking place in therapy and a standard by which to measure if and how well that is happening. The use of evidence-based practices can help protect clients from counselors using practices that are unproven at best and harmful at worst.

As a counselor, I’ve successfully used these practices to address specific needs. However, I’ve also sometimes felt more like a technician than a therapist when following a manualized curriculum. Using a manulaized approach can seem pretty sterile and somehow less than genuine.

Additionally, some approaches don’t fit well with some counselors.The majority of evidence-based practices are somehow related to Cognitive Behavioral Therapy (CBT).  Counselors with foundations in other approaches, such as psychodynamic, gestalt or existentialism, may have a difficult or at least uncomfortable experience adopting CBT practices.

Some counselors are completely opposed to any talk of evidence based-practices. They argue that the use of manulaized approaches take the human warmth and genuineness out of counseling. Others have completely embraced evidence-based practices and argue that using anything less is malpractice.

Of course the truth is usually someplace in the middle. With an understanding that the push toward evidence-based practices is probably going to be with us for a while, I’d like to review some important considerations starting with a definition of some terminology.

1. What is an evidence-based practice?

In the truest sense, an evidence-based practice is a manualized counseling intervention, often including counseling scripts, that has been shown effective for a specific population with a specific disorder. In order to use evidence-based practices appropriately, the script and schedule should be followed as indicated.

In reality, very few of us actually ever use an evidence-based practice in this sense of the term. I’ve seen very few practitioners actually deliver an evidence-based intervention exactly as prescribed to the exact population it was tested for.More often than not, what we actually use are evidence-informed practices.

Another category, called promising practices, includes interventions that seem to have some level of efficacy but have not been thoroughly tested. Most evidence-based practices start off as promising practices and are used in a limited fashion until sufficient evidence can be gathered.

Evidence-informed and promising practices are critical for the advancement of the counseling profession. If all we ever used were evidence-based practices, we’d never have anything new.

So where do we find information on the evidence? SAMHSA maintains The National Registry of Evidence-Based Programs and Practices.  Each listing provides descriptive, research and contact information about each listed program or practice.

Not all effective interventions are listed on the SAMHSA registry. One of the three common approaches used by the Veterans Administration in the treatment of PTSD is Cognitive Processing Therapy (CPT). The approach was initially developed by Dr. Patricia Resick for working with rape victims. CPT is not listed on the SAMHSA registry but is based on techniques from Cognitive Behavioral Therapy, which is.

In short, few practitioners are actually using evidence-based practices in the truest sense of the term. Even when the practice is evidence-based, it’s application is likely not. More often than not, what we do in practice probably falls in the categories of evidence-informed or promising practices.

2. What populations were tested?

This question can be a little tricky. Most approaches indicate applicability for various age groups or genders. A closer look at the research may indicate a very limited testing population. Additionally, most research is conducted on people with a single identified problem.

Participants with atypical representations or other complications are usually screened out of the research group. Research on PTSD treatments usually include participants without any co-occurring problems.  Research on treatments for substance use disorders typically screen out anyone with depression or PTSD.

As we all know, very few clients present with a simple or single diagnosis. Co-occurring is the rule rather than the exception. Unfortunately, evidence-based practices are almost never tested on patients with multiple or complicated problems.

One exception is Seeking Safety, a curriculum that, to my knowledge, is the only evidenced-based curriculum for patients with co-occurring PTSD and Substance Use Disorder. Unfortunately, the quality of research (see next section) is not as high as I would like.

The Matrix IOP curriculum was designed and tested for those with stimulant use disorders. However, it is commonly used on clients with a history of using a variety of substances other than stimulants.

Once a practice is considered evidence-based, it will likely be modified and adapted to other uses or populations. Matrix IOP has been adapted by Hazelden for use with teens and young adults but has not been sufficiently tested to be listed by SAMHSA. Is it evidence-based? Not really, but it is based on evidence-based practices (complicated huh?).

The SAMHSA listing for Prolonged Exposure (PE) therapy indicates that three of the four studies used to determine quality of research were conducted on women and the fourth on a non-US population. None of the research used by SAMHSA to determine quality of research indicates studies with a US male population. However, PE is another of the three common approaches used by the Veterans Administration in treating PTSD.

Again, in order for a practice to be considered evidence-based, the population it’s being used on should match the population with which it was tested. While it’s impossible to match your clients exactly to the participants in a research study, they should come pretty close. More often than not, that isn’t going to be the case.

3. What is the quality of the research?

Finally, what did the research actually say? SAMHSA assigns grades to different categories on quality of research including validity, reliability, data analysis and several others. Ratings on quality are listed on a scale from 0.0 to 4.0 and can vary considerably.

The research on Trauma-Focused CBT indicates relatively high quality research, scoring around 3.7 on most measures. The quality of the Matrix IOP research didn’t fare as well with scores closer to 2.0 on most measures and 1.5 on a couple. On a zero to four scale, that’s not very good.

Eye Movement Desensitization and Reprocessing (EMDR) is a third commonly used approach in the treatment of PTSD. While the quality of research listed for EMDR is a respectable 3.2 on most measures, some argue it is no more effective than previously existing forms of therapy.

In fact, the parts of EMDR that seem to be effective are the components shared with other therapies including cognitive behavioral therapy and imaginal exposure. As cited in an article in Scientific American quoting a Harvard University Researcher,” What is effective in EMDR is not new, and what is new is not effective.”

Finally, most research reports the effectiveness of therapy for those completing treatment. While some studies consider dropout rates in the results, others do not. For example, Prolonged Exposure (PE) and Cognitive Processing Therapy (CPT) seem to have similar effectiveness rates on those that complete However, PE has a higher dropout rate than CPT, which certainly complicates matters.

In spite of the problems and having considered these three questions, I still think clients are better served when we use evidence-informed and promising practices. As counselors, we need to clearly understand what the evidence is, what the evidence means and how we can use the information to best help clients.

We can’t and shouldn’t take something that purports to be evidence-based and apply it to populations that may or may not benefit from that approach without understanding what we’re doing. We should also understand how one approach compares to another and find the most useful approaches we can effectively use.

I’ve heard some counselors with negative attitudes towards evidence-based practices. I don’t think we’ll ever be reduced to technicians applying scripted interventions but we should at least have evidence to support what we’re doing. If an intervention can’t be supported by some type of evidence, we probably shouldn’t be using it.

I hope this article can generate some discussion and additional questions on this topic. As always, I’d love to hear about your experiences and thoughts on the matter.