controversy over georgia 2007 reeves/empirical 2005 def

Discussion in 'Fibromyalgia Main Forum' started by simpsons, Apr 23, 2009.

  1. simpsons

    simpsons Member

    received this through co cure and thought as tom kindlon says that -

    "Some people may be interested in this material at this time given the
    meeting on the CDC Chronic Fatigue Syndrome Research Program on Monday and
    that comments on the plan have to be received by May 1"

    CO-CURE] ACT,RES: Pre-publication correspondence/reviews on Georgia 2007 Prevalence study (including comments on Empirical/Reeves 2005 definition)
    Inbox
    Add star
    Tom Kindlon
    <tomkindlon@oceanfree.net> Thu, Apr 23, 2009 at 6:13 PM
    To: CO-CURE@listserv.nodak.edu
    Reply | Reply to all | Forward | Print | Delete | Show original
    When this paper was published:

    "Prevalence of chronic fatigue syndrome in metropolitan, urban, and rural
    Georgia.
    Reeves WC, Jones JF, Maloney E, Heim C, Hoaglin DC, Boneva RS, Morrissey M,
    Devlin R.
    Popul Health Metr. 2007 Jun 8;5:5. PMID: 17559660"
    one could read the pre-publication comments at the paper's page:
    http://www.pophealthmetrics.com/content/5/1/5
    (there was a little box that said "Pre-publication History" (or something
    like that)).

    This appears to have disappeared.

    In his article, "Problems with the New CDC CFS Prevalence Estimates"
    http://tinyurl.com/2qdgu4 i.e.
    http://www.iacfsme.org/IssueswithCDCEmpiricalCaseDefinitionandPrev/tabid/105
    /Default.aspx
    Leonard Jason Ph.D refers to these pre-publication comments:
    ~~~~~~~~
    In Peter White's Dec. 3. 2006 review
    (http://www.biomedcentral.com/imedia/1083914155124266_comment.pdf) of Reeves
    et al.'s (2007) article, he states: The use of physical function, role
    physical and social function sub-scales is consistent with the International
    Study criteria for CFS, which states that the illness "results in
    substantial reduction in previous levels of occupational, educational,
    social, or personal activities." (Reeves et al, 2003). The use of role
    emotional is not, since it specifically asks about change in function "as a
    result of any emotional problems". And later White states that "In order to
    make these important criteria consistent with other studies, I think the
    authors need to re-analyse their data, omitting this sub-scale."
    ~~~~~~~~

    The link he quotes still works. But I'm not sure anyone would be able to
    now find these or other comments - they don't show up on Google searches,
    for example, neither are they (as I said) linked from the paper itself
    anymore.

    So I have appended links to the pre-publication history.
    This include comments/criticisms of the Empirical/Reeves (2005) definition.

    I've also included the first set of comments by Prof. White on that paper.

    Some people may be interested in this material at this time given the
    meeting on the CDC Chronic Fatigue Syndrome Research Program on Monday and
    that comments on the plan have to be received by May 1.

    Tom Kindlon


    ~~~~~~~~~~~~~~~
    Files relating to pre-publication correspondence:
    "Prevalence of chronic fatigue syndrome in metropolitan, urban, and rural
    Georgia.
    Reeves WC, Jones JF, Maloney E, Heim C, Hoaglin DC, Boneva RS, Morrissey M,
    Devlin R.
    Popul Health Metr. 2007 Jun 8;5:5. PMID: 17559660" one could read the
    pre-publication comments at the paper's page:
    http://www.pophealthmetrics.com/content/5/1/5

    Version: 1 Date: 3 December 2006
    Reviewer: Peter White
    http://tinyurl.com/dakhlz
    i.e.
    http://rapidshare.com/files/224877099/1083914155124266_comment.pdf

    Reviewer: Andrew Lloyd
    Version: 1 Date: 17 December 2006
    http://tinyurl.com/dyx7b3
    i.e.
    http://rapidshare.com/files/224877102/7377902351258282_comment.pdf

    Version: 2 Date: 16 January 2007
    Author's response to reviews: see over
    http://tinyurl.com/cbmfkj
    i.e.
    http://rapidshare.com/files/224877103/1831163344128749_comment.pdf

    Version: 2 Date: 13 February 2007
    Reviewer: Peter White
    http://tinyurl.com/cuvuee
    i.e.
    http://rapidshare.com/files/224877104/1425508902132288_comment.pdf

    Version: 2 Date: 11 March 2007
    Reviewer: Andrew Lloyd
    http://tinyurl.com/cdv8x3
    i.e.
    http://rapidshare.com/files/224877105/6221089501354307_comment.pdf

    Version: 3 Date: 5 April 2007
    Author's response to reviews:
    http://tinyurl.com/cx7wc4
    i.e.
    http://rapidshare.com/files/224877106/1296233610138832_comment.pdf


    ~~~~~~~~~~~~~~~~~
    http://www.biomedcentral.com/imedia/1083914155124266_comment.pdf
    Or
    http://tinyurl.com/dakhlz
    i.e.
    http://rapidshare.com/files/224877099/1083914155124266_comment.pdf


    Reviewer's report

    Prevalence of Chronic Fatigue Syndrome in Metropolitan, Urban, Title: and
    Rural Georgia

    Version: 1 Date: 3 December 2006

    Reviewer: Peter White

    Reviewer's report:

    1. Is the question posed by the authors new and well defined?
    Yes, it is, in that the novel aspects are:
    - the state-wide sampling
    - the study of prevalence across metropolitan, urban and rural populations
    - the screen question being unwellness, not fatigue
    - the use of standardised questionnaires to measure symptoms

    2. Are the methods appropriate and well described, and are sufficient
    details provided to replicate the work?
    No; see major compulsory revisions.

    3. Are the data sound and well controlled?
    Not yet; I think the authors need to review their analyses.

    4. Does the manuscript adhere to the relevant standards for reporting and
    data deposition?
    Yes.

    5. Are the discussion and conclusions well balanced and adequately supported
    by the data?
    I think the authors need to be more cautious in their interpretation of
    their findings of a much higher
    prevalence than previously found. They should also refer to similar findings
    of prevalence found in other
    countries.

    6. Do the title and abstract accurately convey what has been found?
    Yes

    7. Is the writing acceptable?
    Yes.


    Major Compulsory Revisions (which the author must respond to before a
    decision on publication can be
    reached)

    1. The methods leave out non-English speaking people, which may have led to
    an under-sampling of ethnic
    minority groups, and possibly CFS, if there is a difference in those who do
    not speak the majority language.
    I note the authors weighted the sample to allow for ethnic differences. The
    authors should mention this in
    discussion.

    2. The authors state that, "..those with a score ? well-population medians
    on the general fatigue or reduced
    activity scales of the MFI were considered to meet fatigue criteria of the
    1994 case definition." This means
    that it would be possible to meet the fatigue criterion without significant
    fatigue; i.e. with reduced activity
    alone. This is inconsistent with the international study criteria for CFS,
    which require: "clinically evaluated,
    unexplained, persistent or relapsing chronic fatigue (of least 6 months
    duration) that is of new or definite
    onset" (Reeves WC, Lloyd A, Vernon SD, Klimas N, Jason LA, Bleijenberg G,
    Evengard B, White PD,
    Nisenbaum R, Unger ER. Identification of ambiguities in the 1994 chronic
    fatigue syndrome research case
    definition and recommendations for resolution. BMC Health Services Research
    2003;3:25 (published 31
    December 2003). http://www.biomedcentral.com/1472-6963/3/25 ). In order to
    provide prevalence data consistent with
    previous criteria for CFS, the authors need to re-analyse their data for
    general fatigue alone on the MFI, if
    that is their preferred sub-scale to measure the severity of fatigue.

    3. The authors state that, "Functional impairment was assessed by the
    medical outcomes survey short
    form-36 (SF-36) [13]. For classification as CFS, those with a score ? 25th
    percentile of population norms in
    the physical function or role physical, or social function, or role
    emotional subscales of the SF-36 were
    considered to have substantial reduction in activities as specified in the
    1994 definition." The same criticism
    as in the paragraph above applies to their choice of SF36 subscales. The use
    of physical function, role
    physical and social function sub-scales is consistent with the International
    Study criteria for CFS, which
    states that the illness "results in substantial reduction in previous levels
    of occupational, educational, social,
    or personal activities." (Reeves et al, 2003). The use of role emotional is
    not, since it specifically asks
    about change in function "as a result of any emotional problems". I note
    that "role emotional" had the lowest
    correlation coefficient of any of the SF36 subscale scores with any of the
    three measures of CFS in one of
    the authors' previous studies (Wagner D, Nisenbaum R, Heim C, Jones JF,
    Unger ER, Reeves WC:
    Psychometric properties of the CDC symptom inventory for the assessment of
    chronic fatigue syndrome.
    Population Health Metrics 2005, 3:8). In order to make these important
    criteria consistent with other studies,
    I think the authors need to re-analyse their data, omitting this sub-scale.

    4. There is a further problem about the choice of thresholds for both
    fatigue and functional impairment. For
    fatigue, the authors have chosen a median split from population norms, i.e.
    50 % of the population would
    have a score of equal or greater than this. In contrast, they have chosen
    the bottom quartile score of
    population means. Why the discrepancy? This needs justification. Since the
    criteria for CFS include
    "severe" fatigue and dysfunction, it makes sense to make both thresholds the
    lower quartile from a
    population sample.

    5. Furthermore the authors need to refer to the studies from which they took
    these population norms. I am
    aware of several population studies of working age adults using the SF36,
    but they give slightly different
    results, and the readers need to know which one they chose and why. I am not
    aware of any population
    studies of working age adults using the MFI. This reference should be given.

    6. The authors state that, "For classification as CFS, those reporting ? 4
    case defining symptoms and who
    scored > 25 on the SI case definition subscale were considered to meet
    accompanying symptom criteria of
    the 1994 case definition." Having read the relevant paper, I am still
    uncertain by both what is meant by the
    "SI case definition subscale" and what a score of 25 means. This needs both
    clarifying and justifying.

    7. What proportion of the three different geographical sample households did
    not have a working telephone
    at the time of sampling? This is important information in order to
    approximate the selection bias towards
    greater economic income. I would assume that this data would be available.

    8. What proportion of those screened by telephone was fatigued but not
    "unwell"? This should be reported.

    9. How did they work out the numeric value of the weightings? It would help
    the reader to give more details
    about this.

    10. The authors give mainly univariate analyses results with limited
    interaction data. We need to see
    multivariate analyses to model all the examined factors that may influence
    the prevalence of CFS, so that
    confounding and interactions can be excluded or explored.

    11. The authors describe similar findings across black and white people, but
    do not describe the ethnic
    groups adequately. What do they mean by "black"?


    Minor Essential Revisions (such as missing labels on figures, or the wrong
    use of a term, which the author
    can be trusted to correct)

    1. Reference 12 needs revision.


    Discretionary Revisions (which are recommendations for improvement but which
    the author can choose to
    ignore)

    1. The description of the numbers and proportions found in sampling,
    telephone assessment and clinical
    assessment is complicated and it is hard to follow the numerators and
    denominators. I think readers would
    be helped by an algorithmic figure with the numbers (%) at each stage, with
    both true and false negatives
    and positives given at each stage of assessment.

    2. The difference between metropolitan and urban areas seems somewhat
    arbitrary to this non-American
    referee. I suspect some further justification would help non-American
    readers

    What next?:
    Unable to decide on acceptance or rejection until the authors have responded
    to the major
    compulsory revisions

    An article of importance Level of interest in its field

    Quality of written English: Acceptable

    Statistical review: Yes, and I have assessed the statistics in my report.

    quick reply to
    To: Tom Kindlon <tomkindlon@oceanfree.net>


    [This Message was Edited on 04/23/2009]
  2. QuayMan

    QuayMan Member

    Thanks for that.

    If you're a bit lost, it might be a difficult piece to start on.

    An easier document to understand criticisms of the definition might be Leonard Jason's article at:
    http://tinyurl.com/2qdgu4
    i.e.
    http://www.iacfsme.org/IssueswithCDCEmpiricalCaseDefinitionandPrev/tabid/105/Default.aspx
  3. QuayMan

    QuayMan Member

    One interesting aspect to this piece is that Peter White was part of the 4-person external panel that reviewed the CDC.

    Peter White is no shrinking violet - he challenges documents if he doesn't agree with them such as the NICE Guidelines in the UK or before that the CMO Report.

    Yet, in the external review, the empirical definition was listed as a strength - and no concerns were raised about the definition. Here's the exact quote in case people don't believe me:

    "Research Program

    Strengths to be further encouraged

    1. The review panelists judged that over the past decade, the CDC team has amassed a very important body of research work on CFS. In particular, CDC has led the world in defining the illness, including improving the quantitative precision with which the illness is defined—through development of a standardized case definition. While some have recently criticised the recent standardization of the research criteria for CFS, the panelists believed that the CDC’s work on psychometric operationalization of the existing 1994 case definition should improve the reliability of research."