Editorial comment on Statistical Meta-analysis

The Massachusetts CFIDS/ME and FM Association would like to bring to the attention of our readers that the type of study conducted by the group of German doctors discussed in this article is referred to as a "meta-analysis." This term refers to the statistical analysis of a group of individual studies.

The authors of a meta-analysis re-analyze the results of many smaller studies (with only the limited information available in each published report of a study, not the original data from each) pooled as if a larger study had been done with all of the patients in the smaller studies.

Thus a meta-analysis will take results from 10 studies of 10 patients each and claim to have valid results from 100 patients, but without having access to the original data for each of the 100 patients.

Thus a meta-analysis, in itself, is not an actual scientific study, but a type of averaging of averages from many different studies.

Therefore, from a mathematical and also a medical standpoint, trying to get "effectiveness" data comparing drugs from varying studies that only compared a single drug to placebo is not really valid. There are too many factors that can throw one's conclusions off such as variations in study length, severity of FM in patients of different studies, numbers of dropouts in a study due to severity of side-effects, etc.

So the German meta-analysis is similar to comparing apples to oranges (placebo), persimmons to oranges and redwood trees to oranges, and then coming up with conclusions about how apples, persimmons and redwood trees compare to each other.

On the other hand, the German meta-analysis provides a broad review of a large number of studies. It is worth considering its conclusions, but they really shouldn't be taken as "gospel."

To put things even more into perspective, some time ago a meta-analysis of treatments for CFS found that Cognitive Behavioral Therapy (CBT) was the "best" treatment for CFS.

The facts that most of the studies analyzed used the Oxford definition of CFS and compared CBT to placebo, and only a few of the analyzed studies looked at other treatments and patients identified by the 1988 or 1994 definitions of CFS, were ignored.

Thus, the conclusion of a meta-analysis can be heavily influenced by the number of studies of a certain treatment outweighing the number of studies looked at in the analysis that used a different treatment. We don't know if that is the case in this German study, but it could be.