College of American Pathologists
Printable Version

  Statistical associations don't prove clinical utility


CAP Today


March 2008
Feature Story

One of the cardinal principles in the evaluation of pharmacogenomics for determining warfarin sensitivity is that statistical associations do not prove clinical utility, says Margaret Piper, PhD, MPH, director of genomic resources at the Technical Evaluation Center, Blue Cross/Blue Shield Association. "Proof of clinical utility requires evidence that dosing according to genetic test results has incremental clinical benefit compared to usual practice," Dr. Piper explains. Current data related to pharmacogenomics testing for warfarin consists largely of associations, such as the association between out-of-range INRs and adverse events. Unfortunately, associations can be misleading (Ware JH. N Engl J Med. 2006; 355: 2615- 2616; Wald NJ, et al. Br M J. 1999; 319: 1562- 1565; Pepe MS, et al. Am J Epidemiol. 2004; 159: 882- 890).

"Results indicate that the genetics of warfarin account for a substantial proportion of the variation in final warfarin dose," Dr. Piper says. "But we don't have any prior experience or prior models to say that the amount accounted for is enough to make a clinical difference. For some of these initial applications the bar for evidence might be very high. As more experience accumulates," she says, "the bar may come down."

Dr. Piper was a consultant for the recent ACMG review of evidence on pharmacogenomics for warfarin (www. acmg. net/ AM/ Tem plate. cfm? Se ction= Home3& Template = / CM/ ­Content Dis play. cfm& Content ID= 2263). In the executive summary, the report says: "No study has yet shown this intervention to be effective in reducing the incidence of high INR values, the time to stable INR, or the occurrence of ­serious bleeding events." Many studies are underway, the report notes, but until results are available "the clinical utility [of this intervention] is still unknown." In the ACMG report, cost-benefit estimates are also presented and a tenta­tive conclusion offered: "[T]he costs of testing may be more than twice the expected health care savings under this scenario." The report offers this evalu­ation of a much-cited cost-benefit estimate in a joint American Enterprise Institute/ Brookings Institution study: "After review, it appears that these ­authors made several choices in their modeling that may not be warranted, or may need further documentation." Details are provided.

It is important to understand the difference between an evidence review and Food and Drug Administration clearance of an assay, Dr. Piper says. "Policy determination starts with the evidence base, and focuses on evidence for improved outcomes," she says. This approach is not part of the FDA's review of devices, which is limited to an "excellent and much-needed" evaluation of technical assay performance, she says, and establishing that there is some evidence of clinical validity. The FDA does not, however, evaluate clinical utility.

-William Check, PhD