To cite or not to cite-determining what's right

March 2001
Originally published in CAP TODAY

Vida Foubister

Eight deficiencies found during CAP inspections tend to pop up frequently, but the solutions used to rectify them can be as unique as the laboratories themselves.

That's the message Thomas F. Ruhlen, MD, west central region commissioner for the CAP Laboratory Accreditation Program, conveyed to laboratory inspectors in an educational audio conference last fall.

"Something we keep running into is complaints from labs who are inspected. They'll say, 'The inspector insisted that we do it their way, that we use this format, or that we have to have our signatures in such a way,'" says Dr. Ruhlen, medical director of laboratory services at Olathe Medical Center, Olathe, Kan. "We're trying to get out the word that there's more than one way to meet the requirements and that inspectors should be a little more open-minded."

Most important, laboratories need to ensure that they are fulfilling the requirements of the Standards for Laboratory Accreditation. "The checklists are just tools to assess whether they're meeting the Standards," Dr. Ruhlen says. "So it's really not appropriate for an inspector to get hung up over some minute detail when the lab's truly meeting the intent of the Standard." Based on a review of the CAP laboratory inspection database, Dr. Ruhlen identified the eight checklist questions that most often result in deficiencies. (They are listed below in no particular order.)

1. An inspection scenario: Nurses at a point-of-care testing site that performs whole blood glucose monitoring manually transcribe results onto a flow sheet in the patient's record. The flow sheet lacks preprinted reference ranges, and, despite the nurse supervisor's assurance that these ranges are transcribed along with the results, they're missing in about 80 percent of the patient records.

What action should the laboratory inspector take? The inspector can choose to:
    Not cite a deficiency because the laboratory is in compliance.
    Not cite a deficiency but address the problem informally.
    Not cite a deficiency but make a written recommendation.
X Cite a deficiency.

In this case, Dr. Ruhlen says the laboratory inspector should cite a deficiency because the nurses working at the POC testing site have failed to meet this Standard for about 80 percent of patients tested. "They're not even coming close to accomplishing the goal, so they need to try harder or try some other system," he says.

The POC Testing Checklist question specific to this scenario is 30.0450: "When applicable, are all patient results reported with accompanying reference (normal) or interpretive ranges?" "Typically, nurses are doing the testing and reporting the results on a nursing form," Dr. Ruhlen says. "When the nursing committee implemented that form, they had no idea they had to put reference ranges on there."

An obvious solution is to revise the flow sheet to include the reference ranges. But Dr. Ruhlen recognizes this isn't always practical. "There may not be room on the form; it may take months to get forms approved; or they may have preprinted a five-year supply of forms," he explains. "There are any number of things."

In those cases, the POC testing site might want to find another solution. One option would be to use a sticker or a rubber stamp, with the reference ranges printed on it, to place this information on the existing form. Another option: Develop a form that lists the glucose reference ranges and is a permanent part of the patient's chart. But Dr. Ruhlen, whose laboratory did this several years ago for all of its tests, says it can be "very difficult to keep these forms up-to-date and even on the chart."

Those sites that choose to continue recording references ranges with test results should develop a new procedure that explicitly states this step is a requirement. "If you're going to rely on them to manually transcribe the reference ranges each time, how do you know they're doing it between now and the next inspection?" asks Dr. Ruhlen. "That would probably require a policy describing what the expectation is and how it is to be accomplished but should also include some kind of documented tracking mechanism to ensure ongoing compliance." This might be accomplished using a quality improvement indicator, he says, and he adds that the reference ranges must be reported with qualitative and quantitative test results.

2. An inspection of a laboratory's safety program reveals that the criteria used to evaluate the effectiveness of the chemical hygiene plan aren't fully described. The documentation, signed each year by the laboratory director, states that the chemical hygiene plan has undergone a review that includes an evaluation of the plan's effectiveness.

An inspector could respond by citing a deficiency. But if the laboratory personnel can readily describe how the effectiveness of their plan is assessed, it might be more appropriate to make a written recommendation. "They're probably doing what they're supposed to do, and I would just recommend that they formalize it," Dr. Ruhlen says.

The Laboratory General Checklist question specific to this scenario, 01:7050, asks, "Is there annual review and evaluation of effectiveness of the laboratory's chemical hygiene plan?" Sometimes the annual review is simply not documented, says Dr. Ruhlen. More often, however, the annual review has been documented, but it doesn't contain any evidence that it includes an evaluation of the plan's effectiveness, he adds.

How can a laboratory evaluate its plan's effectiveness? One way would be to track the number of accidents involving hazardous chemical spills or mishaps. The results of periodic safety audits also could be compared to see if the number of noted deficiencies is stable or decreasing. Employees' compliance with and knowledge of the elements of the chemical hygiene plan could be monitored as part of their annual competency evaluation.

3. A freestanding laboratory that does not collect patient specimens on site has a documented evacuation plan. It addresses the needs of workers with disabilities but overlooks the needs of visitors to the laboratory. When asked, the laboratory manager claims this isn't necessary because patients are never on site.

Though an inspector could choose to cite a deficiency, he or she typically will make a written recommendation. "Sometimes you go into a lab and there's just chaos," says Dr. Ruhlen. "They really don't understand a lot of this checklist content; they're not doing a very good job; and they do need to be cited on these things so they understand it's important."

"In some places everything is hunky-dory and you find just one or two things [that need improvement]. That's a little bit different," he says. "If the lab, overall, is doing a pretty good job and this seems to be a minor oversight, then I personally don't get excited about getting them for the letter of the law and citing a deficiency. A place like that is going to be more likely to respond adequately to a written recommendation."

General Laboratory Checklist question 01:7080 asks, "Is there a comprehensive, documented and workable evacuation plan, including specific plans for any persons with disabilities?" Sometimes laboratories do not have an evacuation plan. More often, however, the evacuation plan is simply not comprehensive enough. As in this scenario, laboratories may fail to include plans for visitors. Plans for persons with disabilities also are commonly omitted. Laboratories should address this deficiency by developing a detailed evacuation plan and educating their employees about its requirements.

4. The immunology section of a laboratory uses a direct antigen detection kit without built-in controls for its urine human chorionic gonadotropin pregnancy tests. External positive controls are run every day; negative controls are not. When asked, the laboratory supervisor explains that negative controls aren't run if a patient result is negative because that demonstrates the kit is capable of producing a negative reaction.

   Though the intended response is to cite a deficiency, the supervisor's response "actually makes some sense," Dr. Ruhlen admits. "Our checklist commissioner sent a very similar scenario to all the regional commissioners and asked, 'How would you handle this situation?' The results were pretty interesting because there was significant variation in the way commissioners handle it. Many said, 'No, it doesn't meet the requirements of the checklist,' but there were quite a few that said, 'When we think about it, we can't see why that's not valid.'"

   Quality control continues to evolve, and some longstanding guidelines tend to reflect tradition more than science. "The technology has got to the point where some of those old quality control rules [might not] apply. This is one of those examples where maybe you don't need to have a negative control with some of these tests," says Dr. Ruhlen.

   The Commission on Laboratory Accreditation, however, has not yet reached that conclusion. Unless that checklist question is changed, the laboratory in scenario No. 4 is considered to be not in compliance with the current Standard.

   This scenario is further complicated because the two relevant questions in the proposed checklist are significantly different from existing Immunology & Syphilis Serology Checklist question 06: 3430. Once the proposed checklists, which have been approved by the Commission on Laboratory Accreditation, are approved by the Health Care Financing Administration, they can be used in the field for inspections. (The proposed checklists are available on the CAP Web site.

   "Currently we're using checklists that are two-plus years old for the actual inspection," Dr. Ruhlen says. "But if a laboratory protests when it gets cited on something, then we refer to the proposed checklist. If what's in the proposed checklist is more lenient than what's currently in the field, then we go ahead with what's in the proposed checklist. If what's in the proposed checklist is more stringent, then we stay with what's currently in the field."

   The current immunology checklist question that refers to direct antigen testing QC asks, "Are known positive and negative controls run on each day of analysis with each run of specimens for all qualitative or semi-quantitative serologic tests (e.g., rheumatoid factor, rapid plasma reagin, ASO, hCG, heterophile antibody, etc.)?" As such, it doesn't distinguish between those tests that have built-in controls and those that don't.

   The proposed immunology checklist replaces this question with two new ones that make this distinction. They are: "For tests that do not include built-in positive and negative controls, are known positive and negative controls tested on each day of analysis for all qualitative or semi-quantitative antigen/antibody tests (e.g. rheumatoid factor, rapid plasma reagin, ASO, hCG, heterophile antibody)?" and "For tests that do include built-in positive and negative controls, is a positive and negative external control tested with each new kit lot number or different shipment of a given lot number for all qualitative or semi-quantitative antigen-antibody tests (e.g. rheumatoid factor, rapid plasma reagin, ASO, hCG, heterophile antibody, etc.)?"

   "The proposed checklist simply clarifies the requirements-it doesn't change them," says Dr. Ruhlen. If a test kit does not include built-in controls, then external positive and negative controls need to be run each day of use. If the test kit includes built-in controls, then external positive and negative controls need to be run only with each new test kit. These internal controls, however, are sufficient only if the test system has been classified as a waived or unmodified moderate-complexity test under the Clinical Laboratory Improvement Amendments of 1988. In addition, the manufacturer's recommendations must be followed.

5. This scenario involves review of a laboratory's immunology section documentation of new reagent testing for the lab's infectious mononucleosis kits. To parallel test new kits, the laboratory runs known positive and negative patient specimens from the previous day using reagents from the new kit.

In this case, the laboratory's procedure is considered an acceptable practice and should not be cited.

The Immunology & Syphilis Serology Checklist question that addresses this is 06:3315: "Are new reagent lots checked against old reagent lots or with suitable reference material before or concurrently with being placed in service?"

Laboratories sometimes don't run a parallel check. "When the old reagents expire, they simply start using the new reagents," says Dr. Ruhlen. "Another common problem is that laboratories do the parallel testing, but their documentation is weak or nonexistent."

Furthermore, laboratories often take the controls from an old kit and test them using the reagents from a new kit. "The controls may have what we call lot specificity, meaning the reaction has something to do with the manufacturer's lot that it's from," Dr. Ruhlen says. "So you're making a possibly invalid assumption that control material is going to react just like patient material. The ideal is to test patient samples or some kind of reference material that you know is going to act like a patient sample."

To address these problems, laboratories should run parallel tests with known patient samples, as in scenario No. 5, or with reference material that gives positive and negative results. They also should use a separate log sheet to document this process. "The idea is that before you start using the new kit, while you're still using the old kit, you'll run those kits side-by-side," says Dr. Ruhlen. "So you can test patients using the old kit and report it off the old kit, but run the test with the new kit to be sure you're getting the same answers."

Laboratory inspectors sometimes misinterpret this checklist question, requiring laboratories to run parallel tests with both patient specimens and reference materials when either alone is sufficient.

6. A laboratory's temperature and maintenance records for a chemistry instrument are initialed and dated by the section supervisor each month. However, there is no evidence of a weekly review.

That's acceptable, says Dr. Ruhlen, and thus the laboratory should not be cited. "The current checklists in the field do state a requirement for weekly review by a supervisor and then monthly review by the laboratory director or designee," he explains. "But the Commission on Laboratory Accreditation recently approved a change in the proposed checklist that requires monthly review by the laboratory director or designee. Any more frequent review is up to the discretion of the laboratory director."

Automated/General Chemistry Checklist question 03:2010, which also appears in many of the other checklists, asks, "Is there evidence of active review of results of instrument maintenance and function, temperature, etc., for routine procedures on all shifts?"

Inspectors commonly find that laboratories fail to include the temperature and instrument maintenance records as part of their QC review. Another problem is that documentation of this review is sometimes incomplete, and some laboratories forget to complete the review every month. We've found that some laboratories are "keeping temperature records on their refrigerators and their heating blocks, but no one looks at them," adds Dr. Ruhlen.

To resolve these problems, instrument maintenance and temperature records should be included in QC review; a laboratory director's approval for a designee to perform this review must be documented; the scope and frequency of a laboratory's QC review should be described in its QC plan; and laboratories should diligently carry out these reviews as prescribed.

7.  & 8. In this scenario, the chemistry section's procedure manuals are written in the NCCLS' format. While the first page of each procedure includes evidence that an annual review has been conducted by the chemistry supervisor, the laboratory director's signature is not on any of the procedures.

The two Automated/General Chemistry Checklist questions that address this scenario are 03:2100 and 03:2110: "Is a complete procedure manual available at the workbench or in the work area?" and "Is there documentation of at least annual review of all policies and procedures in the automated chemistry laboratory section by the current laboratory director or designee?"

"We see a whole variety of problems with procedure manuals that constantly come up," says Dr. Ruhlen. "Sometimes the procedures are nonexistent or woefully out of date, or they're not complete or detailed enough. Sometimes we see manufacturers' inserts being used without any kind of modifications to make them specific for the laboratory. Sometimes the manuals aren't available in the work area-they're put somewhere in a supervisor's office, well away from the workbench. Sometimes work cards are in place that don't correlate with the procedures as outlined in the procedure manual. Annual review documentation is a problem. And then sometimes we see problems with the use of electronic manuals."

Here's what Dr. Ruhlen suggests laboratories do to comply with the Standards.

• Write the procedures in the NCCLS' format. That means they must include statements on the test's principle, clinical significance, specimen type, required reagents, calibration, quality control, procedural steps, calculations, reference ranges, and interpretation.
• Tailor manufacturers' inserts and manuals to meet the specifics of the laboratory. This includes information on controls, reporting instructions, reflexive testing, critical values, and reportable ranges.
• Keep current manuals in the work area and keep work cards up-to-date.
• Follow the checklist requirements for documenting the annual procedure manual. "There's basically two ways to go about this," says Dr. Ruhlen. "One is to date and sign each procedure either on the first page or the last page of the procedure, or the alternative is to date and sign each procedure in the table of contents. It's not acceptable to have a single page at the beginning of a procedure manual that says, 'All the procedures in this manual have been reviewed, dated, and signed.'"
• If electronic manuals are used, keep records that document annual reviews and specify who is authorized to make changes.
• Ensuring the laboratory meets these requirements and reviews the resulting procedure manuals annually is a logistical problem, Dr. Ruhlen admits. "We have lots of procedures and they come and go," he says, "so it's always an ongoing problem. That's why procedure manuals are one of the most common deficiencies."