Return to CAP Home
Printable Version

  Cytology screening limits, by the numbers

 

CAP Today

 

 

 

October 2010
Feature Story

Anne Paxton

“Cut-rate Pap mills process slides using screeners with incentive to rush.” Like a key unlocking a door, that November 1987 Wall Street Journal headline could be said to have opened the way to CLIA ’88 and the whole new era of laboratory regulation. And from the start, the maximum workload limit for cytotechnologists—no more than 100 slides in an eight-hour day—has been a linchpin of CLIA.

But with the advent of semiautomated cytology screening devices, the workload limit began to be bent, both in federal regulations and in the field—and not always in a clear manner. So it was a relief to many labs when a July 27 alert from the Food and Drug Administration clarified an aspect of cytology workload limits that has been unsettled for several years: exactly how the Centers for Medicare and Medicaid Services will count the slides processed using semiautomated devices when calculating the workload. “What CMS has done is basically leave people with no way to misinterpret the limits,” says Ann Moriarty, MD, a cytopathologist with AmeriPath, Indianapolis, Ind., and chair of the CAP Cytopathology Committee.

CLIA’s 100-slide-per-day limit was set long before the semiautomated devices, often called image-assisted screeners, arrived on the scene. When the FDA approved the first such device in 2003, the ThinPrep Integrated Imager made by Hologic, it set the limit at 200 slides maximum in a 24-hour day, including the time spent on manual review of slides, with a requirement that the total not be screened in less than eight hours. The BD Diagnostics FocalPoint GS Imaging System, which came along five years later, was assigned a limit of 170 slides in 24 hours, within a minimum eight-hour day.

In the systems’ package inserts, the manufacturers took their cue from the FDA. “CLIA ’88 didn’t really address the automated instruments. The guidelines that the manufacturers provided, which were what the FDA approved, were different from what was in the CLIA guidelines,” says Michael Henry, MD, director of cytology at Mayo Clinic in Rochester, Minn., and a member of the CAP Cytopathology Committee.

There has been confusion on this score ever since the semiautomated instruments came out, Dr. Henry says. “When CMS was inspecting labs, its recommendation for counting was different from the exact wording in the manufacturers’ guidelines. But the agency didn’t really come out and say, ‘This is how you have to do it.’ We asked CMS to clarify this about a year ago, and they were then in contact with the FDA, and that’s how this clarification came about.”

The CMS was encountering problems in cytology laboratories during inspections, says CMS spokesperson Ellen Griffith. “In a number of surveys, CMS found there were inconsistencies with the way labs were counting slides when using these semiautomated devices. In the manufacturers’ package inserts there were one or two lines saying what the workload is, but not how to count the slide, and that’s an integral part of the whole process.” Labs found the instructions as written to be unclear, she says. In August 2009, the agency asked the FDA to clarify the policy, and from now on the CMS expects that the package inserts will contain the information on calculating workload. Labs would then be expected to follow the manufacturer’s instructions per CLIA requirements, Griffith says.

While the clarification comes from the FDA, the true enforcer of screening limits is the CMS, says Robert Bredt, MD, CAP Laboratory Accreditation Program Mid-South regional commissioner and medical director of the Women’s Health Labs with the Texas Department of State Health Services, San Antonio. “They’re the ones that go out and do inspections and hold labs accountable. And there’s been a lot more scrutiny lately on workload recording and limits. CMS has put more emphasis on going through page by page and line by line, and making certain that neither the federal limits nor the laboratory-set limits are exceeded.”

Using the semiautomated devices, the cytotechnologist or pathologist initially does a “field of view” (FOV) review, with the device zeroing in on fields that the computer has deemed to fall outside of its pre-determined parameters. The problem is this, says Dr. Moriarty: “The cytotechnologist is whizzing along and says a slide needs a full manual review [FMR]. But how do you count that slide? Do you count it as half a slide because it was on the image-guided instrument? Or do you count it as just a single slide because it’s manually reviewed?”

It has tended to create problems with lab inspections, she points out, because CMS inspectors would count workload one way, while “CAP inspectors were going by the letter of the law from FDA, which was a limit of 200 as long as they kept the image-assisted screening instrument record separate from the manual screening record.”

The FDA, CMS, and CAP are now in agreement that a slide with a field of view review and a full manual review counts as 1.5 slides for purposes of workload calculation. “If the reviewer does not see any abnormalities, you can find the case negative, and that would be considered .5 slides,” Dr. Henry says. But if the reviewer sees an abnormality, a full manual review is required. In that case, “you can either do it on the microscope you are at, or take it to another microscopy station. Either way, at that point you are doing something more than an automated review that adds one full slide to your count.”

The idea of adjusting the blanket 100-slide workload limit actually began with some of the nongynecologic slides, Dr. Bredt says. Starting several years ago, the notes on primary screening of nongynecologic liquid-based slide preparations (CYP.08500) in the CAP cytopathology checklist specified that for the purpose of workload reporting, each slide may be counted as half of a slide, “provided the cells are dispersed over one half or less of the total slide area.”

“So there was already a precedent in place of not counting every single slide as one if it would take less time to screen them,” Dr. Bredt says. With fluids, such as pleural fluid, for example, “because they weren’t very cellular preparations, they often lent themselves to being collected in liquid, and it was relatively easy to screen them, particularly if they had low cellularity.”

There is still a lot of confusion and, at many labs, inadequate systems for keeping track of these numbers, he says. “Because it’s a fairly manual process to track these, a number of labs have set up their own ways to do it, and some are better and worse than others. But it was never really clear until this e-bulletin came out whether to count slides as one, or one and a half.”

One of the major improvements that CLIA ’88 brought, in Dr. Henry’s view, was recognition of the need to cap cytology workloads. “There is a limit to the number of slides you can reasonably look at in one day,” he says. “With the new automated equipment, it’s more difficult to figure out how to count those, and what we don’t know right now is what that means for accuracy and sensitivity.”

In cervical cytology, three major shifts have occurred in the 20 years since CLIA took effect, he points out. “First, the vast majority of Pap tests in the U.S., well over 90 percent, are done with liquid-based collection systems, either ThinPrep or SurePath. And second, in the last few years we’ve seen the advent of these automated screening devices. Third, we’ve seen the introduction of HPV testing as an adjuvant test in cervical cytology.”

Have these changes led to improvement in disease detection? “I think there have been incremental improvements. There is debate in the literature as to whether or not going to liquid cytology has truly improved the pick-up rate of disease, and I think we’re still trying to see whether automated instruments will improve disease detection. That still hasn’t been completely answered yet,” Dr. Henry says.

Nor has the question over the impact of proficiency testing. “Some published statistics show that with initial failures of PT, you get rid of some pathologists or cytotechnologists who probably shouldn’t have been screening, but after that, PT may or may not improve overall cytology screening, because the test is very basic—and it has to be in order to be fair.”

As for the cytology workload regulations, there continues to be disagreement in the field over whether the screening limits make sense, Dr. Bredt says. “Some folks argue that a cytotechnologist screening a slide should not be penalized if there are abnormalities found as far as slide screening—that if the cytotechnologist does a field of view review and then goes back for a manual rescreening, he or she already has a ‘gestalt’ on the slide, and the review should be counted as one slide rather than one and a half. And there’s some merit to that argument—that there should essentially be some credit for the work that went into that field of view screening.”

The requirement that cytotechnologists also have individual screening limits is sometimes overlooked, Dr. Bredt says. For example, “if you have a cytotechnologist who has difficulty with a high screening rate, you might set them down to 80 slides a day rather than 100.” Some states have lower caps than CLIA’s as well; “if the lab is located in or is screening California-originated slides, they’re obligated to limit screening of those slides to 80 per day.”

But “you can’t just say everybody is at 100 and that’s the way it is. You have to look at each cyto­tech­nologist’s individual past performance. There has to be documentation that each six months you’ve actually evaluated them and set limits.” He estimates that 95 percent to 99 percent of cytotechnologists are set at the maximum limit. On the other hand, “There are a number of labs and field directors who feel 100 is too many for anybody to screen. And some pathologists and medical directors have quite the opposite view and think that 100 is arbitrary, that some cytotechnologists function quite well at higher limits and that in fact they might function better at higher limits, because they get into a rhythm.”

Another requirement sometimes overlooked is that anytime a pathologist is doing initial screening, 10 percent negative rescreening, or five-year negative lookback screening (the requirement to review all negative slides from the previous five years of a patient with a new high-grade dysplasia), “they are obligated to do workload reporting just as cytotechnologists are,” Dr. Bredt says. That includes setting workload limits based on performance, recording slides screened, recording hours spent, and prorating slides if needed.

“If you screen only four hours a day, you’re limited to 50 slides, not 100. I’ve had a number of labs argue, ‘Well, we never exceed 100 slides a day or 100 a month so we couldn’t possibly get into problems with screening limits.’ And that’s not true because it has to be prorated. To take an extreme example, if your lab did one cytology slide a year, you could theoretically exceed the screening limits if you screened that single slide in four minutes or less.”

The inspectors do ask to see any workload recording documents that the laboratory may have, Dr. Moriarty says. But more important than screening limits, Dr. Moriarty believes, is the assessment of performance and assignment of individual workload limits. “The technical supervisor, the pathologist in charge of the cytopathology labs, needs to do that every six months for every cytotechnologist.”

She doubts that most cytotechnologists would hit the maximum limits in a regular day, because most do not screen for eight hours; at most, cytotechnologists working a full workday might screen for seven or 6.5 hours. “You have to prorate, and that makes recording the workload when you have to do it in two separate areas very onerous. It can be done, but it’s just more complicated.”

Dr. Moriarty would worry about people in a gynecology lab who might hit the 100-slides-a-day mark consistently. “But if you do nongynecologic cytology, if you screen a lot of thyroid cases or fine-needle biopsies from many sites, it’s pretty common in large-volume labs to have your cytotechnologists have higher screening rates because it’s a whole different process. But once again, CLIA dumped everything in one bucket. Cytology is cytology whether you’re doing Pap tests or thyroid nodules, and those screening limits still apply.”

“If the pathologist is the primary evaluator of any cytologic material—it doesn’t have to be just Pap tests but any cytologic material—if they do the initial screening, they have to keep a workload record,” Dr. Moriarty says. “And I think we’ve tried to do a good job of educating, but some pathologists do not realize this applies to them if they do the primary screening.” However, she notes, if pathologists are working alone and are the screeners, they do not have to do a 10 percent retrospective review—”because that would just be themselves looking at their own cases.”

The prorating method has been the source of much controversy. “I don’t think this has really been studied,” Dr. Moriarty says. “The formula has not been rigorously determined, but, then, neither were the original numbers. They were just decided upon without any scientific backing.”

Recent studies on screening and workload have found that as volume increases, the sensitivity of the Pap test diminishes, which makes sense, Dr. Moriarty says. “Essentially the more time the cytotechnologist is able to spend on the slide, the better the sensitivity. And therefore the workload limit should be set with a clear understanding of the sensitivity of the test, at whatever level you’re setting it at. If you can get 95 percent sensitivity at 40 slides a day, and that’s the sensitivity you want, you need to set the limit there. But that’s a really low number compared to what many people are doing in this country.” In fact, she suggests, this may explain the difference between European cytological screening and that in the United States. “The American labs tend to push for higher productivity, and oftentimes, in the European labs, they are not working under the same pressure for workload.”

Unfortunately, in the U.S., there is some danger that workload limits can be considered quotas or “something to shoot for,” Dr. Moriarty says. Most studies have shown productivity increases with image-assisted screening, usually about 30 to 50 percent more than the cytotechnologists would do without the instrument. When the first image-assisted screening device was approved, “there were rumors that a cytotechnologist or a company owner would say, ‘OK, we have image-guided screening. You have to do these 200 slides today, and you can’t leave until they’re all done.’” The cytotechnologist would not exceed the 200-slide limit as the Pap tests were all “image assisted,” and the hourly workload was not exceeded because they could spend as much time as necessary doing the Pap tests. However, she says, the pressure exists for the ­cytotechnologist to get as many Pap tests done as possible as fast as possible. Although Dr. Moriarty doesn’t know of anyone who practices this way, “you could see the potential for abuse. For people who would do large amounts of Pap tests, if they wanted a certain turnaround time, they could potentially wiggle around the workload limit that way.”

The laboratories in which the workload limits were exceeded are usually of two types, in Dr. Bredt’s experience. “Either they are really pushing the envelope and demanding a lot of productivity out of their cytotechnologists,” he says, “or they may be fairly low volume but just don’t have the machinery in place to do a thorough workload recording and checking.”

In either case, the regulations are clear on what should happen when the limits are exceeded. “It’s pretty obvious, from the CMS inspections I’ve seen in the past, that if the cytotechnologist exceeds limits on a particular day, all the slides screened on that day have to be rescreened as part of the corrective action,” Dr. Bredt says. “And evidence that rescreening was done has to be submitted before the lab can pass inspection.”

However, he adds, “I’ve never, ever seen a lab that has caught someone screening too many slides.” When he does inspections, “I’m always getting a calculator to review the numbers, and when I find something out, it has never been noticed by the lab beforehand. It comes as a complete surprise to them.”

Cumbersome workload recording forms may be at fault, he believes. “The forms don’t allow someone to quickly determine if workload limits have been exceeded. They’ll look and make sure they haven’t gone over 100 slides a day, but when it comes to taking the number of hours that slides are screened into account, somebody along the line has to sit down with a calculator.” Sometimes labs are unwittingly not adhering to the limits. For example, Dr. Bredt says, “it needs to be emphasized that if a lab lowers a cytotechnologist’s screening limit, adherence to that lower limit is required just as though that limit were the maximum.”

Despite their weaknesses, workload limits have proved much more effective than proficiency testing in improving cytology, Dr. Moriarty says. “The PT that we’re required to take, I feel, has not had an impact on quality—other than that people who take the test have gotten a lot better at taking the test. But there is very little evidence the PT of individuals has an effect on the quality of the lab.”

Designing a test to assess whether someone is competent to sign out Pap tests is extremely difficult, she points out, because of the interobserver variability that occurs. “It’s a screening test done by humans, so there is no classic case you can send to everyone and get the exact same answer.” The CAP’s proficiency testing program is close to that ideal, because it has been field validated, meaning the slide has gone around to a large number of labs and 90 percent of people agree on the diagnosis. “And that’s great,” Dr. Moriarty says. “But what PT does is select the absolute best slide to be tested on; it doesn’t really identify those people who can’t perform in their daily work.”

Workload limits are one of the key reasons cytology quality has risen since 1988, she contends. “Retrospective reviews, careful consideration of abnormal rates, and workload limits were important. And considering the pre-1988 abuses, where cytotechnologists were paid by the case or had quotas of over 200 manual slides per day, yes, cytology has improved.”

Enforcing workload limits must continue to be a CAP priority, Dr. Bredt says. When it comes to laboratory inspections and accreditation, one of the problems is that CMS can be variable, and some CMS inspectors may emphasize different parts of the regulations more than others. The same is true of CAP inspectors, he adds. “But my philosophy, when I do a CAP inspection, is if a CMS inspector is coming after me, I want the lab if they were subsequently inspected by the toughest CMS inspector there to pass with flying colors. Because there are a certain number of validating CMS inspections required in order for CAP to retain its deemed status, the worst thing you can do, as a deemed inspection agency, is to have a poor correlation with a subsequent CMS inspection, just because you made the assumption they’re not going to look heavily at the workload.”

Dr. Henry stresses that workload limits are only part of a complete quality assurance program for the cytology lab. “Your cytotechnologist competency evaluation is part of that, your 10 percent rescreen, retrospective reviews, and benchmark comparisons all make sure your individual cytotechnologists and pathologists are performing appropriately. They’re all part of making sure your overall cytology program is good.”

In the grand scheme of things, the most important thing to focus on is not the number of slides but the quality of the interpretation, Dr. Moriarty says. “The advantage of having the newly produced guidelines is it decreases confusion over how to count slides. But it’s really important that people remember the whole reason to have workload limits is for patient safety.”


Anne Paxton is a writer in Seattle.
 

Related Links Related Links