Pulling waived tests back
  to safety’s shore

November 2005

Feature Story

Karen Lusky

Under CLIA ’88, waived testing has become like a largely unregulated stepchild long left to its own devices, a growing number of increasingly sophisticated ones that the law says should pose a negligible risk of producing a wrong result—and no reasonable risk of harming a patient if performed incorrectly.

The College has long argued that no test meets that legislative definition, which is why its Laboratory Accreditation Program doesn’t differentiate between waived and nonwaived testing. Even an out-of-date urine dipstick or one stored in an open container might fail to detect blood or protein in a patient’s urine, resulting in a missed opportunity to diagnose chronic kidney disease or renal cancer.

To offer waived testing under CLIA, a physician office, nursing home, or other setting has only to do two things: obtain and maintain a certificate of waiver, and perform tests following manufacturers’ instructions. The more than 105,000 waived labs nationwide aren’t routinely inspected, which means a good portion of waived testing is in the hands of non-laboratorians in sites where CLIA inspectors seldom tread.

And pilot studies and surveillance by the U.S. Department of Health and Human Services over the past few years have found that a persistent percentage of CLIA-waived labs aren’t following manufacturers’ instructions for performing waived testing, including running quality controls when the manufacturer specifies it. The findings have prompted a series of ongoing efforts by various HHS agencies to shore up the quality and safety of waived testing.

In the latest development, the Centers for Disease Control and Prevention is gearing up to promulgate recommendations for good laboratory practices, or GLPs, for waived laboratories. The agency was, at CAP TODAY press time, planning to publish the recommended practices in a November 2005 Morbidity and Mortality Weekly Report as a reference document and starting point for disseminating the information more widely.

The GLPs essentially follow the recommendations developed by a Clinical Laboratory Improvement Advisory Committee workgroup co-chaired by CAP president-elect Jared Schwartz, MD, PhD, director of the Department of Pathology and Laboratory Medicine at Presbyterian Healthcare System, Charlotte, NC. CLIAC approved the workgroup’s recommended lab practices last February, and agreed the CDC should publish them in the MMWR this year.

In addition, the Food and Drug Administration’s recently released draft guidance for CLIA waiver applications includes key CAP and CLIAC recommendations and other provisions designed to ensure waived tests approved in the future meet the spirit and letter of CLIA ’88. Steve Gutman, MD, director of the FDA’s Office of In Vitro Diagnostic Device Evaluation and Safety, says the “guidance intends to provide information on the design and performance issues that need to be addressed to ensure a test is simple and has an insignificant risk of an erroneous result.”

For example, the FDA’s draft guidance encourages manufacturers to participate in developing and promoting “good lab practice guidelines by developing training and educational programs for the end operator.” The guidance places more emphasis on quality control, recommending manufacturers provide information on the general purpose of QC and the value of using it as a broader system of quality assurance.

Following CLIAC’s recommendations, the FDA’s draft guidance also changes the paradigm for field studies. Manufacturers should vet their products, it says, in actual intended clinical sites where nurses, medical assistants, doctors, and others with limited or no hands-on training in conducting lab tests use the devices on actual specimens collected from their patients. The guidance document used now by the FDA to approve waived devices allows manufacturers to conduct field studies in which untrained lay people (high school students or factory workers, as examples) test devices on artificially contrived samples.

The FDA draft guidance is intended for manufacturers and is a separate process from what the CDC is doing in publishing good laboratory practices, says Devery Howerton, PhD, chief of the CDC’s Laboratory Practice Evaluation and Genomics Branch. But even though the FDA is working from a different perspective, all of the HHS agencies with responsibility for CLIA oversight have efforts underway with a common goal of strengthening waived testing.

In fact, the CDC’s efforts to disseminate GLPs is expected to augment the Centers for Medicare and Medicaid Services’ ongoing annual on-site visits to two percent of waived sites annually. The CMS initiated the random inspections in 2002 after a pilot study it conducted in eight states showed 32 percent of a sample of waived labs didn’t have current manufacturers’ instructions for performing testing; and 32 percent didn’t perform QC for testing when the manufacturer required it. Twenty percent had cut occult blood cards and urine dipsticks. The CMS’ findings generally mirrored those of monitoring data reported by the CDC and a study by the HHS Office of Inspector General.

The CMS’ ongoing announced visits appear to be paying off, according to the agency’s survey data from 2002 to 2004. Twelve percent of the labs the CMS surveyed during that period did not have current manufacturers’ instructions—and 21 percent didn’t perform QC when the manufacturer required it. But problems persist, says Judy Yost, director of CMS’ Division of Laboratory Services, which she attributes, in part, to high lab personnel turnover in waived settings, training shortfalls, and lack of clear instructions for test performance.

The CMS plans to continue its annual on-site visits, though CLIA ’88 doesn’t offer any “big sticks” for enforcement when it comes to waived test sites. However, Yost says, if there is risk of harm to patients—immediate jeopardy—or the laboratory is testing beyond the scope of waived tests, then the CMS might propose and impose an enforcement action. But the CDC’s plans to disseminate good laboratory practices for waived testing could have a more far-reaching effect on the quality of testing in waived settings. “Since we all strongly believe that many of the issues in certificate-of-waiver labs can be addressed with education,” Yost tells CAP TODAY, “the MMWR document will facilitate our and others’ abilities to reach these labs with educational information.”

The central premise behind the CDC’s effort is that health care providers want to do the right thing for patients. “The problem is that often the staff doing the [waived] testing don’t know that they don’t know good lab practices,” says Paula Garrott, EdM, CLS(NCA), a member of the CLIAC workgroup that developed the good laboratory practices. “Even physicians serving as directors of the waived settings often have little or no background in good lab practices,” says Garrott, professor emeritus of clinical laboratory science at the University of Illinois, Springfield and a partner in Laboratory Consultants Inc. of Illinois. “And non-laboratory trained individuals doing the testing may not understand clearly the role of quality control in ensuring accurate results.”

The CDC’s Dr. Howerton characterizes the MMWR recommendations as “a fairly comprehensive compilation of a number of recommended practices for all phases of the total testing process—before, during and after testing.” For example, “the GLPs provide a list of considerations [one should make] before introducing waived testing or a new waived test,” she says. “These include the regulatory and physical requirements, costs and benefits of a test, personnel considerations, how the test would impact workflow at the site, and whether the test meets the intended use for its patient population.”

The GLPs place the responsibility and accountability for performing waived testing on the physician operating the waived lab, which is how Dr. Schwartz believes it should be. “The primary care physician representation in our CLIAC workgroup felt as strongly about that as did the laboratorians and pathologists,” he says.

The medical director also must be sure the people hired to do the testing are adequately trained, though CLIA ’88 doesn’t require any training for staff to perform testing in waived sites. In addition, “the lab must have a mechanism to ensure the tests are being performed in a consistent manner that yields reliable results,” Dr. Schwartz says. “This requires ongoing quality control and monitoring—not just performing QC for a test but also a program perhaps using external benchmarks to make sure people in the lab are performing to some standard, for example, proficiency testing.”

Dr. Schwartz says the analytical phase for waived testing is “probably less of a problem than the preanalytical in that the tests themselves are usually self-contained and easy to operate if the user reads and follows the manufacturer’s instructions.”

The CDC is weighing how to disseminate the GLPs in ways that might be more specific or targeted to certain groups performing waived testing. Dr. Howerton notes, for example, that the CMS already provides information on its Web site that explains the requirements for performing waived testing under CLIA. “CMS also has a one-page list of good laboratory practices that surveyors give out during surveys of waived sites,” she adds. “So CDC may provide supplemental materials to what CMS is providing for aspects of waived testing.”

CLIAC has proposed a number of ideas for getting the GLPs in the hands of people who perform waived testing. One idea is to ask manufacturers to endorse the GLPs at the bottom of their test instructions, referencing the document and including a link to the Web site, Dr. Schwartz says.

He does foresee that the GLPs could eventually become an accepted standard of practice. In that case, physician offices that adopt the practices might receive reduced rates on their liability insurance. Or a managed care organization could demand a participating provider doing waived testing follow the guidelines if it wants to care for the MCO’s patients, he says. “[CLIAC] is optimistic that this document will be well accepted as a reference on how to operate a lab doing waived tests to ensure the results are consistent and reliable,” says Dr. Schwartz.

One challenge to capturing clinicians’ attention about shortfalls in waived testing performance has been its “no harm, no foul” aspect that stems, in part, from a lack of published studies linking negative patient outcomes to erroneous waived test results. “The CDC does reference one report in MMWR,” Dr. Howerton says, “where transmission of hepatitis B occurred in a nursing home because the caregiving staff were reusing lancet holders on patients to obtain blood specimens for blood glucose testing.”

But a lack of published or easily accessible data doesn’t mean there isn’t a serious problem with waived testing performance, says Dr. Schwartz. In fact, he adds, outcomes data related to problems with waived testing is an area “rife for research.” While a media disaster involving waived testing seems like a remote possibility, it could happen, cautions Charles Root, PhD, laboratory and compliance consultant and president of CodeMap, Barrington, Ill. Say a patient being treated at a Coumadin clinic dies because of an erroneous INR reading from a waived device, he says, and investigators find that no one was performing quality controls on the device or following the manufacturer’s instructions. That could open the door to media coverage of a largely unregulated class of testing, Dr. Root says.

The potential for patient safety mishaps involv ing waived testing has become an increasing concern as breakthrough technology paves the way for point-of-care tests that the crafters of CLIA ’88 would have relegated to science fiction. Dr. Root notes that people aren’t particularly worried about the simple tests that give a positive or negative result like rapid strep tests. They worry more about waived status for CBCs and, in the future, possibly even a waived chemistry analyzer, though no one has thus far aimed for the latter, he says. (Chempaq plans to file a waiver application for its Chempaq XBC hematology analyzer, which offers a WBC, three-part WBC differential, and hemoglobin, as soon as the FDA approves its 510k for the device, according to Jakob Moller Jensen, business development manager for the Copenhagen-based device-maker.)

The draft FDA guidance is not only designed to ensure waived devices provide accurate test results when used by untrained laboratory personnel, but also to provide end users with more tools to perform the tests correctly. For one, the FDA draft guidance requires manufacturers to provide more precise language about QC in their instructions. “Manufacturers [currently] use terms in their instructions, such as ‘routinely perform’ QC, which tells the end-user nothing if the person has no lab background,” says C. Anne Pontius, MBA, CMPE, MT (ASCP), president of Laboratory Compliance Consultants Inc., Raleigh, NC. “For example, does routinely mean once a week, once a month, or when the person has time, which may be never? The FDA is asking manufacturers to use more specific instructions, such as perform QC every other day.”

The FDA draft guidance also talks about manufacturers supplying test kits with “quick reference” instructions (preferably laminated) written at no higher than a seventh-grade reading level. “The lab can post those instructions, which are usually in pictograph form, so staff will have a guide in front of them to follow in performing the testing,” says Carolyn Jones, director of technology and regulatory affairs for AdvaMed, which represents device-makers.

Also in the guidance are recommendations for manufacturers to consider “providing innovative mechanisms to provide technical assistance to labs, and to ensure they understand the labeling information.” In that regard, a lot of vendors of waived devices are already providing online training, at the end of which the user gets a certificate, says Erika Ammirati, president of Ammirati Regulatory Consulting, Los Altos, Calif.

Laboratory consultant Sheila Dunn says manufacturers could improve waived testing performance by disseminating more learning tools, such as colorful wall charts and laminated cards with specific instructions. For example, when performing a test, “sometimes you wipe off the first drop of blood and sometimes you don’t—it depends on the test,” says Dunn, president and CEO of Quality America, Asheville, NC. Even if the CDC or the CMS disseminates small laminated cards spell ing out good lab practices, for example, those instructions won’t contain directions for performing a specific waived test, she adds. Manufacturers also have a better shot than government agencies, she says, at getting the educational materials directly into end-users’ hands.

And perhaps distributors even better than manufacturers. Dr. Schwartz says the CLIAC workgroup developing the GLPs realized that distributors could have an impact on the quality of waived testing by offering online and other training, because distributors sell the products and interact with customers more than manufacturers.

The draft FDA guidance requires manufacturers to conduct hazard risk analyses and risk assessments consistent with how their products are going to be used. “That is not specifically spelled out in the CDC guidance in place now, so from a quality standpoint, it is a good step by the FDA to ensure high-quality products are on the market,” says Warren E. Pinckert II, president and CEO of Cholestech Corp. in Hayward, Calif., which makes a system focused on heart disease and diabetes for the physician office.

In a hazard analysis, the manufacturer thinks through all of the things a user could do with or to a product to produce a false result, explains Juliet Carrara, director of regulatory affairs and quality assurance for Cholestech. The device-maker then develops a fail-safe (or lock-out feature) or fail-alert mechanism to prevent operators from getting a wrong result when they do one of those things to the product.

The guidance doesn’t allow a test requiring sample manipulation to be waived, which is already true under the CDC waiver guidance in place now. “That was one thing the manufacturers wanted changed, so that a waived test could involve some simple specimen manipulation, such as spinning down a urine or blood sample,” says AdvaMed’s Jones. “But CLIAC opposed that as being beyond waived testing. CLIAC supported only basic non-technique-dependent specimen manipulation for waived testing.”

The most controversial component of the draft guidance from the manufacturer’s standpoint are the field testing requirements for waived products, which CLIAC championed. Dr. Gutman tells CAP TODAY the CLIAC recommendation was based on the fact that the study of devices with potential waived users testing real patient samples over time in a multitasking environment would generate more valuable information than lay users testing contrived samples.

Dr. Root says that if manufacturers follow the on-site trials, they will have to assign someone to a physician office for a month to get enough data, “and that’s an expensive proposition.

“But the draft FDA guidance is just that—guidance,” he adds. “It tells the manufacturer how to approach getting their product waived, but the FDA still evaluates each submission separately.”

After the FDA analyzes comments on the draft, which are due in December, it will issue another guidance in a year or so, Dr. Root estimates. (Dr. Gutman declined to say when the final guidance might come out.) In the meantime, Dr. Root adds, manufacturers will gravitate toward using the draft guidance. “They may negotiate a less stringent trial than the one spelled out by the draft. But if manufacturers incorporate at least the spirit and principles of the draft guidance, they will have a better chance of getting clearance the first time.”

Even without regulatory changes, the FDA can put higher-stakes waived tests in a de facto separate or hybrid category, which is how the agency handled rapid HIV as a waived test. The CDC’s Dr. Howerton notes that “part of the uniqueness [of rapid HIV] is that the FDA put sales restrictions on the test so that sites offering it are required to have a quality assurance program and some limited training for staff performing the test.” Other waived tests in the future could have those kinds of restrictions, she says.

Dr. Schwartz characterizes the FDA’s approval of rapid HIV as a waived test as a “major wake-up call and one of the factors increasing the sense of urgency in the industry for developing GLPs for waived testing.” But in his view, the waived testing performance problem isn’t because of the tests on the market. “Many of the tests are excellent. It’s how the tests are used and ensuring the user performs the testing correctly,” Dr. Schwartz emphasizes. “But from an analytical standpoint, the tests are solid in their performance. Many of the tests are used in hospitals as point-of-care tests.”

As for whether the CLIAC-developed GLPs improve the quality of waived testing performance, only time—and data from the CMS’ continuing random surveys of waived sites—will tell. “Education has its place,” says Ammirati of Ammirati Regulatory Consulting. the whole issue of improving quality ultimately comes down to compliance. And regulators have to get out in the field. CMS has shared information that once a waived lab gets inspected by CLIA surveyors—whether it’s fear or anger or loathing—people do seem to improve.”

Yost agrees: “Nothing replaces the on-site visits of the surveyors. One-on-one is always the best mechanism to educate these folks because they are all different,” she says.

Dr. Root predicts that if future CMS surveys show that waived testing sites are following the GLPs—and using manufacturers’ instructions, including QC—then the agency will see the educational approach as successful. “But,” he says, “if CMS runs into a wall or Congress gets all over the agency over a negative outcome due to waived testing that hits the media, then Congress could add an amendment to the CLIA law that further regulates waived testing.”