College of American Pathologists
CAP Committees & Leadership CAP Calendar of Events Estore CAP Media Center CAP Foundation
 
About CAP    Career Center    Contact Us      
Search: Search
  [Advanced Search]  
 
CAP Home CAP Advocacy CAP Reference Resources and Publications CAP Education Programs CAP Accreditation and Laboratory Improvement CAP Members
CAP Home > CAP Reference Resources and Publications > CAP TODAY > CAP TODAY 2008 Archive > Statistically speaking, federally mandated PT is unreliable
Printable Version

  Statistically speaking, federally mandated
  PT is unreliable

 

CAP Today

 

 

May 2008
PAP/NGC Programs Review

B. Lane Jr., MD

Seventeen years after proficiency testing in cytopathology was mandated by the Clinical Laboratory Improvement Amendments of 1988, annual nationwide testing began, in 2005, as required by the Centers for Medicare and Medicaid Services. There is widespread agreement among medical professional organizations that the federally mandated cytopathology proficiency testing program is ineffective, and legislative efforts to change the program have been introduced in Congress. Meanwhile, annual proficiency testing in cytopathology continues unabated.

In a recent article, George K. Nagy and Sonya Naryshkin review basic statistical principles of testing theory and evaluate the validity and reliability of the current CLIA ’88-mandated cytopathology proficiency testing format (Nagy GK, Narysh­kin S. The dysfunctional federally mandated proficiency test in cytopathology: a statistical analysis. Cancer Cytopathology. 2007; 111: 467–476). The authors explain basic principles of dichotomous and non-dichotomous testing. They illustrate a simple binomial error model and show how it can be applied to cytopathology proficiency testing. They show how the size of the test set and the competence of individual examinees directly affect the accuracy of a testing system.

The authors provide statistical evidence that the current cytopathology proficiency testing system misclassifies substantial numbers of cytopathologists. Using basic statistics they show why most first-attempt failures are not caused by insufficient cytopathology skills but instead are due to the poor validity and reliability of a 10-slide test. They also show through basic statistical principles why a 10-slide test with three attempts allowed is an ineffective method to identify less-than-competent cytopathologists. They conclude that the current federally mandated proficiency testing system cannot reliably measure the expertise of cytopathologists, and they advise that if cytopathology proficiency testing is to achieve its intended purpose it needs to be completely redesigned, with the participation of experts in modern test theory.


Dr. Lane, a member of the CAP Cytopathology Committee, is a pathologist with Southeastern Pathology Associates, Brunswick, Ga.
 

Related Links Related Links

       
 
 © 2014 College of American Pathologists. All rights reserved. | Terms and Conditions | CAP ConnectFollow Us on FacebookFollow Us on LinkedInFollow Us on TwitterFollow Us on YouTubeFollow Us on FlickrSubscribe to a CAP RSS Feed