Anytime a health or fitness practitioner is confronted with scientific evidence that collides with their beliefs about the practices they use to “treat” pain, or to “fix” the issues they believe are “causing” pain, they’re quick to cite their “in practice” experience in an attempt to 1) discredit the science and 2) to convince others their beliefs about their practices are true.
Such justifications are more common, and are taken more seriously, by alternative medicine practitioners and by rehabilitation and fitness professionals who use a variety of “corrective exercise” practices to “fix” what they believe to be postural dysfunctions that are causing or contributing to pain. I consider this arena of “corrective exercise” to be the “alternative medicine” of the fitness field, as these are both, mainly belief-based arenas that rely heavily on anecdotal evidence. Not to mention, many advocates of these various fringe health and corrective exercise practices completely reject controlled testing as a valid means for arriving at reliable conclusions and consider “in-practice” experiences as the only reliable path to determining whether an intervention is beneficial for pain relief.
It’s clear that the practitioners who say things like “I’ve seen it work” and “so and so wouldn’t be who they are if what they were teaching didn’t work,” not only feel these types of anecdotal arguments provide solid “proof” that their practices are valid. But also feel these arguments offer them justification for denying the scientific evidence when it contradicts with the evidence of their experience. However, there are several realities that demonstrate why anecdotal evidence, in the arena of corrective exercise interventions for fixing dysfunctions that are claimed to cause or contribute to pain, doesn’t offer “proof“ of those claims. These are realities the many smart, well-meaning, health, fitness and rehabilitation practitioners who present to their “in-practice” experiences as “proof” don’t’ seem to think about or acknowledge. I must oblige them to.
In this article I’m first going to highlight the several glaring problems with these “it works in practice” arguments, in order to shed light on why they don’t come close to demonstrating that the claims these practitioner make about their practices are valid. And, I’ll also show why the only thing the wide variety of these “its true because I’ve seen it work in practice” claims do serve to provide “proof” of is a deeper psychological reality that undermines all of these anecdotal arguments.
Reason #1 Why Anecdotal Evidence Isn’t Proof of an Intervention’s Validity: We Can’t Rule Out Other (More Likely) Explanations as the Cause for the Pain Relief.
When people see pain relief during or immediately after receiving questionable health practices, it could very easily be due to the placebo effect. And, we also have regression to the mean, which is a natural phenomenon, whereby when things are at their extremes (such as one’s pain levels), they are likely to settle back to normal (i.e., settle back into the middle), or regress to the mean.
As I said in this post on regression to the mean, “Things like non-specific low back pain unfold in a non-uniform manner. These issues tend to come and go; you have good days and bad days, good weeks and bad weeks, with episodes of higher pain and periods of improvement. So, when some sort of alternative medicine treatment or corrective exercise intervention is introduced very soon after a flare up in a person’s symptomology; when you see improvement – as you surely will from something like non-specific low back pain, and many other ailments – you’ll naturally assume that whatever you did when your symptoms were at their worst must be the reason for your recovery. And, it’s these periods of relief that give rise to erroneous perceptions of a treatment’s effectiveness, whether the treatment actually is (objectively) effective or not.
You see, there’s no doubt theses practitioners are witnessing their clients and patients getting pain-relief in their “in practice” experience. But, with anecdotal testimony alone, these practitioners have no reliable means of distinguishing whether they’ve seen improvement through the placebo effect and regression to the mean or not. Not to mention, if the special, “corrective exercise” interventions have been used along with general exercise applications, as the usually are, one has no way of knowing whether it wasn’t the general exercise applications that have caused the (short term, or long term) improvements.
As I said in that same post, “Regression to the mean might very well be the true explanation for one’s pain reduction or relief, but these practitioners simply cannot reliably tell whether their clients or patients were going to get better anyway. Therefore, they cannot not honestly answer “no, it wasn’t due to regression to the mean,” or due to any other things like the placebo effect, or to their bias toward positive evidence, etc. that often cause all of us to misperceive or misinterpret the evidence of our own experiences. “
The fact these practitioners cannot rule out other, more likely, alternative explanations as the cause of the outcomes they’ve experienced shows that anecdotes only tell us that one has witnessed an outcome, but these stories do absolutely nothing to demonstrate the validity of the intervention(s) any practitioner is ascribing as the cause of the outcome.
Additionally, what I just addressed above also raises concerns with the fact that so many alternative health practitioners, and fitness and rehabilitation professionals think the only explanation for the demonstrations they’re seeing in these various corrective exercise courses is the one the instructor is providing.
Reason #2 Why Anecdotal Evidence Isn’t Proof of an Intervention’s Validity: Wrong Information Was (and still is) Taught In the Exact Same Way.
Speaking of these various corrective therapy educational courses: Each prior generation of smart, well-meaning practitioners were being guided by information provided in courses just like the ones we have today – information which is now deemed to be highly incomplete and mistaken – using these very same reasons (e.g., “it works in-practice”) to justify the information they were providing.
This undeniable reality demonstrates that people can absolutely teach courses all over the world, and become recognized names, despite the fact that the information they’re teaching is incomplete and erroneous. In other words, despite the fact that theses course were teaching flawed information, they were still able to convince many well-educated and passionate practitioners (just like you and I) that it was valid information. Think about that.
This look back at recent history really drives home the point made in #1, because it demonstrates that we’ve always been very good at inventing practices, however incomplete and mistaken; at providing good sounding explanations to justify these practices, however erroneous, and at becoming entranced with those explanations.
Reason #3 Why Anecdotal Evidence Isn’t Proof of an Intervention’s Validity: The Existence of So Much Conflicting Information Demonstrates That Someone is Necessarily Wrong.
As an experienced fitness educator who teaches at fitness conferences and clubs throughout the world, the most common complaint I hear from fitness professionals is not that they have trouble finding information on this or that training topic; it’s “there is so much conflicting information out there, I don’t know what to believe.” And, I most often hear this complaint when it comes to the variety of conflicting claims we have on offer in the corrective exercise arena, in regards to how the body works when it comes to the nature of pain and dysfunction, and how to go about “fixing” it.
It’s important to note that when we say there’s “conflicting information,” what we’re really saying is that there are two or more parties making mutually conflicting knowledge claims about how the nature of reality on given topic. It’s crucial to realize this because when we do, we realize that these conflicting claims cannot all be correct. If one is correct, than all other mutually conflicting claims by default must be false. There is no way around this.
Keeping in mind the reality that was just discussed above in #2, about the past generations of smart practitioners being taught information that was false: Due to the high amount of conflicting information in the corrective exercise arena, as to the nature of pain and dysfunction, we know for a fact that it all can’t be correct. Therefore, someone is necessarily wrong. The existence of mutually contradictory knowledge claims about how the body works guarantees this.
Some practitioners may mistakenly believe they’re getting around this fact by taking their favorite methods from multiple corrective exercise schools of thought, and integrating them together. But this is ignoring that fact that each of these school of thought has its own principles – principles they claim are observable, predictable and universal. And, its these principles that inform there decisions, and therefore guide the methods they use for each individual client/ patient. So, the fact that one chooses to use a variety of methods, which do ceryainly vary based on the individual, does nothing to change the fact that each school of thought makes important, conflicting claims about the universal principles of how the body works, in regards to what causes or contributes to pain and dysfunction.
Here’s an example focusing on four (of the many) different schools of thought we have on offer in the corrective exercise arena:
When it comes to effectively correcting “dysfunction,” we have a school of thought that claims we must “stretch specific tight muscles and strengthen specific weak muscles,” while we have another school of thought telling us, “don’t stretch muscles, stretch patterns or fascial lines.” While another, different school of thought claims, “don’t stretch at all because muscle tightness is secondary to muscle weakness.” While, we have yet another, different school of thought claiming that “it’s not a stretching or strengthening issue because the mobilizers are acting as stabilizers, therefore it’s a timing (i.e., motor control) issue.”
None of these four (mutually conflicting) claims above are “methods.” They are the principles each of these different schools of thought claims about how the body works, in regards to the nature of dysfunction, and how we should go about “correcting” it. It’s these principles that each school of thought use to guide their methods, and these directly conflicting principles cannot be reconciled.
With that distinction made, between principles and methods; since the laws that govern the human body are the same for everyone, each of the four mutually conflicting claims (used in the example above) about how the body works, in regards to the nature of dysfunction and pain, cannot all be correct. As I established above, if one is correct, than all other mutually conflicting claims must therefore be wrong.
If you’re still not clear on this reality, understand that it’s no different than one party claiming that smoking doesn’t increase your risk of lung cancer while another party claims that smoking does increase your risk of lung cancer. Someone has got to be wrong, as these are two mutually conflicting principles of how the body works in regards to the nature of how smoking affects us. Of course, we know the body works in such a way that smoking does indeed increase one’s risk of developing lunge cancer.
With that example in mind: Just because I take a course from both schools of thought on smoking, and decide that I like a few of the tactics (i.e., methods) each school of thought uses to help people to quit smoking, doesn’t change the fact that, when it comes to their objective (knowledge) claims as to whether or not smoking increases risk of lung cancer, one of them is wrong. I hope you see how that example is analogous to the idea that, just because a fitness pro or rehab specialist picks a few of their favorite exercises interventions (i.e., methods) from various corrective therapy schools of thought, and integrates these applications together, doesn’t change the fact that the conflicting claims each of these schools of thought make about the principles of the body can’t all be correct.
Put simply, the fact there’s so much conflicting information in the arenas of corrective exercise, and alternative health, when it comes to claims about how the body works, in regards to the nature of pain and dysfunction, demonstrates that, just like in recent history (see #2), there are practitioners who are misjudging the evidence of their own experiences; they’re mistaking things like the placebo effect and regression to the mean for a genuine treatment effect. And, as established above (in #2), although many of these schools of thought have been successful at impressing believers with their explanations and demonstrations, we know from all the conflicting knowledge claims that some, if not all, of them are teaching information that’s incomplete and mistaken.
Reason #4 Why Anecdotal Evidence Isn’t Proof of an Intervention’s Validity: The More Conflicting Information, The Higher the Probability of Being Mistaken.
Even if we assumed that a particular corrective exercise school of thought did get it right in their claims about the principles of how the body works in regards to the nature of pain and dysfunction, and wasn’t misjudging the evidenced of their own experience; given the sheer diversity of conflicting claims on offer, every corrective therapy practitioner should expect that they’re mistaken purely as a matter of probability.
If it were only the four different, conflicting schools of thought we discussed in the example provided earlier, one only has a 25% chance of being correct, but has a 75% chance of being in the school of thought that’s mistaken. Of course, we all know very well that there are far more than only four different schools of thought within the corrective exercise arena, so the odds are even less likely that one’s particular beliefs are the correct ones about how the body works, in regards to the nature of pain and dysfunction.
It seems to me this undeniable reality would give these various corrective exercise practitioners a great deal of pause, and make them more cautious about expressing their certainties about how the body works, but it usually doesn’t.
Reason #5 Why Anecdotal Evidence Isn’t Proof of an Intervention’s Validity: Different Schools of Thought Use the Very Same Arguments, Giving Us No Reliable Path to Distinguishing Who’s Right.
Although, as we established above, we know that someone has to be wrong, most passionate corrective exercise practitioner, along with many alternative health practitioners, argues that they’ve got it right. And, by doing so, they’re also claiming, whether outright or not, that its those other practitioners who follow conflicting schools of thought who are the ones who are following incomplete and mistaken information. In other words, if you want to know what’s wrong with a particular corrective exercise school of thought, just ask a practitioner who follows a different, conflicting school of thought.
As they argue amongst themselves, while also attempting to refute the science, and while refusing to acknowledge that there are other (more likely) explanations for the results they’re seeing, and while failing to embrace what recent history has taught us. And, while failing to be humbled by the fact that they’re are lots of other, well-meaning practitioners that are just as smart (if not smarter) than they are, who hold mutually conflicting beliefs. While also ignoring the high probability that they could be mistaken; all of these practitioners continue to justify their certainties by using the exact same arguments as one another: “I’ve seen it work in practice experience,” and “so and so (insert name of teacher) wouldn’t be who they are if what they were doing didn’t work.”
This brings to light the fact that each of these various, conflicting schools thought are all operating without overwhelming evidence for any of the claims they may be making about the principles of how the body works, in regards to the nature of pain and dysfunction, and how we should go about fixing it. Lets face it, if one school of thought did have good scientific evidence, there be no need for them to rely so heavy on anecdotes.
The fact is: Using anecdotes to justify the validity of a given health practice has little to nothing to do with objective evidence or reason, therefore it can be directed with equal force towards belief in any practice. Yet, for some reason, practitioners place little or no value in the beliefs of conflicting schools of thought. In other words, many practitioners in these various schools of thought recognize how weak theses anecdotal arguments are because the reject them from others. And, this is despite the fact that those schools of thought have no more or less evidence than their own.
The point here is: Because each school of thought uses these very same lines of arguments in attempt to “prove” the validity of there differing, often contradictory principles, offers us no objective pathway to distinguishing which, if any, of their knowledge claims are objectively true. Remember: Surely they can’t all be true.
In other words, corrective exercises practitioners who follow different schools of thought, and make mutually contradictory claims, all use the same exact lines of argument in attempt to demonstrate that the claims they make are true: they appeal to anecdotal experience (e.g. “I know its true because I see it working in practice” and “so and so is super smart, and they certainly wouldn’t have become known if what they were teaching wasn’t accurate”). So they offer us no reliable method of determining the truth of any of their claims, as claims are only as good as the evidence to support them, and all of these competing schools of thought provide the same exact same types of evidence. Therefore, offering no reasonable justification for accepting any of their claims.
Note: What I feel constitutes “reasonable justification” to (provisionally) believe that a claim is objectively true, is something I discuss in detail HERE.)
Before moving on to the next point, it’s important to note that, although each of these conflicting schools of thought can’t all be right, they can all be wrong. And, as far as I’m concerned, along with many other skeptics, all these conflicting claims made by the various corrective exercise schools of thought cancel one another out, as none of them provide any more of less evidence. They can all be generally mistaken; be different expressions of the same flawed premise; be different versions of an element of truth mixed with a heavy dose of pseudoscience and sloppy thinking. And, to me, that’s currently the safest position to take, when we consider that, in addition to the realities provided above, the majority of scientific evidence that has been done in the arena of alternative health and corrective exercise practices hasn’t demonstrated these interventions to be as valid and as reliable “as advertised.”
Don’t get it twisted! Skeptics, like yours truly, are not saying that we “know” the claims made by these fringe health practitioners aren’t true, unless they’ve already been shown to be demonstrably false. What we are saying is no evidence or argument has been produced that’s persuasive; there’s no one who’s provided any reason to think that they are true. This is a critically important distinction to make.
What All of the Conflicting Anecdotal Evidence Does Prove
You can’t “disprove” personal experience, which is why no one here is trying to do so. I have no reason not to believe that practitioners have had the experiences they’re sharing. The argument here is with the explanations they provide. The goal isn’t to diminish people, its simple to help people recognize that they there could be other possible explanations. The mere fact that someone, or lots of people, had an experience is no way increases the likelihood that this experience is from the reason they say it did, as I’ve clearly demonstrated above in the five points provided. And, we know this because people are having the same experiences using all sorts of different, and often mutually conflicting practices, which brings me to what all these anecdotal claims do prove.
The fact that practitioners from every school of thought claims be achieving the same results (e.g., pain relief, fixing dysfunction), but uses mutually conflicting principles to get there reveals a deeper psychological reality that makes a mockery of all these “it works in practice” based claims. And that reality is called Confirmation bias, which is a filter through which you see a reality that matches your expectations.
Research in social psychology has clearly shown that our observations and beliefs are not the result of years of rational, objective analysis, but are the result of years of paying attention to information which confirmed what we believed while ignoring information which challenged our preconceived notions. There are many forms of confirmation bias, such as the bias towards positive evidence, which describes our innate tendency to detect relationships (between two variables) that are not there because we overvalue evidence that only confirms a given hypothesis. It’s because of our innate fallibility in judging the evidence of our own experiences, which is clearly demonstrated by all the conflicting claims in the corrective exercise arena, that we have terms like, “Self-deception” and “self-delusion.”
In short, all of the conflicting anecdotal evidence in the corrective exercise arena, along with the alternative health arena, demonstrates how unreliable uncontrolled observations in these arenas are. And, they only serve to demonstrate the fallibility of humans at accurately judging the evidence of our own experiences when it comes to things like health interventions.
Additionally, all of the conflicting anecdotal evidence in the corrective exercise arena also demonstrates another undeniable reality, which I provided in my Why Smart Trainers Believe Stupid Things: (Part 3) Regression to the Mean, article:
Without objective, corroborative evidence from other sources, 10 anecdotes are no better than one, and 100 anecdotes are no better than 10. Anecdotes are told by fallible human storytellers, which is why we have sayings like, “the plural of anecdote is not data.” The most reliable means we have for determining which health interventions actually do objectively work, and which ones are ineffective, but may still appear to “work,” is through scientific testing.
Finally, as I said in this article, “This is not saying that science is always right, nor is this to say that anecdotal experience isn’t important – it is! But we must test experience against the evidence. And, when they don’t line up, the science in no way makes the outcomes we see “in practice” any less real. It just means the explanation(s) we have given as the cause for why we had the experiences are likely wrong. In other words, the effect experienced (likely) wasn’t caused by what we thought.
As a skeptic, I’m not at all beyond using anecdotes. However, what I tend not to do (anymore) is make unjustified and unjustifiable claims about those experiences. I spent several years doing that, and I was wrong each time for doing so. And, I’m wrong on any occasion where I slip up and do it in the future… being human means I will occasionally slip up.”
