Made to order: how to add
  new tests

December 2003
Cover Story

William Check, PhD

Amid the rapid pace of technological innovation in laboratory medicine, one could do worse than to follow the advice of the 18th-century English poet Alexander Pope:

Prudent advice indeed. And pertinent to today's clinical microbiology laboratory, where conventional culture and immunoassay are giving way to molecular methods for detecting bacterial and viral pathogens. But, as is often true of good advice, there is a catch-converting it into practice. Just when should a laboratory adopt newer methods? That is, for a given pathogen, how does an individual laboratory decide which method is optimal for it right now?

Anyone looking for a simple, one-size-fits-all answer shouldn't ask Angela Caliendo, MD, PhD, medical director of microbiology/ molecular diagnostics in the Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Atlanta. "I would say that there is no perfect test for every laboratory," says Dr. Caliendo. Although she is speaking specifically about tests for respiratory viruses, her comments apply more generally.

"Depending on your patient population and whether you are an on-site laboratory or servicing multiple hospitals as an off-site laboratory, how you choose to do testing will vary," she says.

What doesn't vary, of course, is the need to provide results rapidly, so the results can influence clinical care. "You can provide that in a variety of ways," Dr. Caliendo says. "You need to decide which is best for your laboratory."

Christine Ginocchio, PhD, lists seven criteria for test selection: availability and expertise of technical staff, space limits (more pertinent to molecular methods), test volumes, required turnaround time, clinician demand, equipment costs (a real-time PCR instrument can cost $100,000), and the extent of the requirements for assay verification and validation. To complicate matters, "Even in the hospital setting, laboratory testing is becoming, in part, a business," says Dr. Ginocchio, director of microbiology/virology and molecular diagnostics, North Shore-Long Island Jewish Health System Laboratories, New York. Low reimbursement and rising health care costs drive financial concerns.

"Years ago we would look at a new laboratory test and if it was exciting, technically good, and clinically relevant, we didn't think so much about how much it cost to run," Dr. Ginocchio says. "We would just use it. Now budgets are tight, and every test we bring in not only must benefit the patient but should be profitable for the hospital as well."

Every laboratory has money-losing tests that are necessary for patient care, but those losses have to be made up in tests that are profitable.

Decisions will become only more difficult as tests with improved characteristics enter clinical practice. Michel G. Bergeron, MD, professor and chair of the Division of Microbiology and Infectious Disease Research Center at Laval University, Quebec City, says, "Due to the fact that microbiological identification still takes around 48 hours, infectious disease management is empirical today. Moreover, clinicians do not consult microbiological results unless patients do not respond to initial therapy." Dr. Bergeron has developed a faster molecular method. His goal is to provide tests that detect and simultaneously identify, in less than one hour and directly from clinical samples, multiple microbes responsible for infections and many antibiotic resistance genes. The first commercial test from his work is for group B streptococcus, or GBS (Bergeron MG, et al. N Engl J Med. 2000; 343: 175-179).

Methods may be novel, but the traditional ways of evaluating them remain the best, says Michael Loeffelholz, PhD, director of Public Health Laboratories in the Arkansas Department of Health, Little Rock. "I would say that if someone is considering implementing a molecular test, a new analyte or a new test they haven't done, they really need to look at the peer-reviewed literature," he says. "Particularly if it is a commercially developed assay, you need an objective source to get a good idea of test performance and workflow." Don't adopt a new molecular test just because it is molecular and new. "A particular molecular test is not always as necessary in some settings as in others," Dr. Loeffelholz says.

Molecular methods are useful for detecting several clinically important pathogens: respiratory viruses, Bordetella, enterovirus and herpes simplex virus in the central nervous system, and GBS in pregnant women.

"In 10 years we have moved eons in turnaround time for detection of respiratory viruses and with more sensitive assays," says Dr. Caliendo, discussing the details of selecting a rapid test for laboratory diagnosis of viral respiratory infections. Early diagnosis can affect patient management by leading to focused care, infection control, and appropriate use of antivirals. "We finally have some antivirals for respiratory viruses," she notes. Rapid tests include antigen identification, rapid culture, and molecular methods.

Patient population and specimen type affect test sensitivity. Children shed higher titers of virus, 10⁵ to 10⁷ pfu/ mL, within the range of sensitivity for rapid enzyme immunoassays, or EIA. "As you get away from the pediatric population," Dr. Caliendo notes, "you lose sensitivity."

To illustrate the interaction of age with collection method, Dr. Caliendo cites data for detecting influenza A virus (flu A) using an EIA compared with a 14-day culture as the gold standard. In children, sensitivity was 90 percent with nasopharyngeal washes; in a geriatric population, sensitivity was 50 to 60 percent with nasopharyngeal or throat swabs (Landry ML, et al. J Clin Microbiol. 2000; 38: 429-430). Adding method as a variable, both a fluorescent antibody (FA) method and an EIA detected all 15 positive nasopharyngeal aspirates in children; for nasopharyngeal swabs in adults, FA's sensitivity was 90 percent, much higher than the 66 percent for EIA.

Based on these and other data, Dr. Caliendo says, "you really don't want to use throat swabs in these rapid assays." Her laboratory refuses them.

Newer, very rapid EIA tests (15 to 45 minutes) are available for flu and respiratory syncytial virus from Becton Dickinson, Biostar, Binax, ZymeTx (which offers a waived test), and others. In general, very rapid tests are less sensitive, with reported figures from 50 to 90 percent, varying by brand. One brand has reported sensitivity of 85 percent; Dr. Caliendo says this occurred under "ideal" conditions in a pediatric hospital. Specificity ranges from 80 to 90 percent.

A second category is fluorescent antibody methods, sometimes differentiated into indirect (IFA) and direct (DFA) fluorescent antibody tests. These assays use pooled monoclonal antibodies to detect flu A/B, major parainfluenza virus serotypes, RSV, and adenovirus. FA tests are of high complexity, require more highly trained personnel, and are run in batches. Their turnaround time is hours to one day.

Figures show higher sensitivity for FA tests than for very rapid tests: flu A, 70 to 95 percent; flu B, 60 to 85 percent; RSV, 70 to 95 percent; PIV, 65 to 85 percent. Adenovirus, at 50 percent sensitivity, may be an exception. One study compared a very rapid EIA, the BD Directigen flu A+B kit, with cytospin DFA in adults. Of all the DFA-positive specimens, 52 percent were Directigen-positive. Positivity rates by number of cells were: <10 cells, 10 percent; 11-100, 44 percent; >100, 92 percent; >1,000, 100 percent.

Another advantage of FA assays is that they can be used to detect multiple pathogens with one test. In a transplant center, for instance, one test can look for both flu and parainfluenza virus.

Rapid culture is done by the shell vial method, in which cultures of multiple cell lines are stained at 24, 48, and 72 hours. Because of the need for multiple cell lines, shell vial cultures are "pretty cumbersome," in Dr. Caliendo's words. A recent innovation is the use of a mixture of mink lung cells and A549 cells (R-Mix, Diagnostic Hybrids), which allows all respiratory viruses to be detected on one cell culture. Cells are stained at one and two days. Dr. Caliendo calls the use of mixed cell lines "a major improvement in rapid culture methods for respiratory viruses." Sensitivity remains about the same as with conventional cell culture and is superior to DFA, with the possible exception of respiratory syncytial virus, which is efficiently detected by DFA.

The real advantage of mixed cell culture is more rapid results. One analysis showed positive results being reported in an average of 1.4 days using R-Mix compared with 5.2 days for conventional culture. On average, 60 to 100 percent of positives turned positive at day one using mixed cell culture. Dr. Caliendo says in addition to getting positive results earlier, the negative cultures are reported earlier, at 48 hours.

Summing up, EIA assays are very rapid but less sensitive and limited to flu and respiratory syncytial virus; FA testing is rapid and useful for a wider range of respiratory viruses; and rapid culture with mixtures of cell lines is "a major improvement over conventional culture methods," Dr. Caliendo says, though it still has a slower turnaround than FA and EIA tests. "We use very rapid tests in the ER in the winter respiratory virus season," she says. For inpatients, she uses FA and rapid culture.

Turning to molecular testing, there are two major options for RNA amplification, nucleic acid sequence-based amplification (NASBA) or reverse-transcriptase PCR (RT-PCR).

BioMérieux's NASBA system, called NucliSens, with detection by electrochemiluminescence or molecular beacons, is available for flu A/B, parainfluenza virus 1-4, and respiratory syncytial virus. It takes several hours to complete a batch. Since these are open kits and include only the basic reagents for nucleic acid isolation, amplification, and detection, laboratories need to provide their own primers and probes. Analytical sensitivity is <1 TCID⁵⁰ or 1-<100 RNA copies; there is no cross-reactivity with other respiratory pathogens. What is lacking for NASBA, Dr. Caliendo says, is extensive clinical evaluation.

The second molecular option is RT-PCR. Prodesse markets a multiplex RT-PCR kit (Hexaplex) for RSV, flu A/B, and parainfluenza virus 1-3, as well as one that simultaneously detects 51 strains of adenovirus (Adenoplex). Hexaplex is more sensitive than culture for RSV and flu A and equally sensitive for PIV. It is slightly less specific than culture-for example, 97.4 percent for RSV. However, Dr. Caliendo cautions that such numbers may be misleading because culture is less sensitive than molecular methods. How does one adjudicate when culture is negative and PCR is positive? In-laboratory-developed assays for real-time RT-PCR have been reported. One such method was as sensitive for respiratory syncytial virus detection as DFA and culture-augmented DFA in nasopharyngeal aspirates from children (Whiley DM, et al. J Clin Microbiol. 2002; 40: 4418-4422).

Dr. Caliendo concludes that molecular methods are rapid, with a two- to six-hour turnaround time, and sensitive and specific. Drawbacks are that multiplex assays can be difficult to design and that it is probably too expensive to do molecular testing as frequently as DFA. Molecular methods also require technologist expertise and can be labor-intensive, particularly for the initial extraction step.

"I don't want [laboratory directors] to think they all have to do molecular testing for respiratory viruses," Dr. Caliendo says. "There is no perfect test." She advises laboratorians to think critically when reading the literature. "Look very closely at the methods," she says, noting patient population, specimen type, culture technique, and assay technique will influence the testing sensitivity and specificity. Finally, ask yourself, "Is this realistic? Would I really do it in my laboratory?"

Dr. Ginocchio emphasizes the external challenges that laboratory directors face when deciding which tests to implement: At the same time that costs, test volume, medical staff demands, and regulations are increasing, budgets, technical staff, reimbursement, and approved tests are decreasing. Personnel shortages are particularly vexing, Dr. Ginocchio says: "We have positions open that we can't fill."

As Dr. Caliendo enumerates, test options include DFA, direct antigen tests, culture, and molecular-based methods. Optimal choices might vary by setting, Dr. Ginocchio suggests. A small community hospital might prefer FDA-approved tests with rapid turnaround time, such as off-the-shelf DFA and antigen detection assays, plus limited culture. To this menu, a large community hospital or university hospital could add molecular assays, both FDA-approved and commercial non-FDA-approved, and in-laboratory-developed molecular assays. A core reference laboratory, considering the need for rapid turnaround times and a competitive edge, would use off-the-shelf assays and more in-laboratory-developed molecular assays for a broad range of test analytes and low-volume niche testing.

Addressing molecular tests specifically, Dr. Ginocchio notes the virtual alphabet soup of amplification methods-PCR, RT-PCR, TMA, SDA, and NASBA. For any amplification method, nucleic acid extraction is "the most labor-intensive step," she says. Semiautomated or automated extraction robots are available from Qiagen, Gentra, Abbott, BioMérieux, Roche, and others.

Detection can be done by a traditional endpoint method (colorimetric, luminescent, or electrochemiluminescent) or by real-time technologies, including dyes such as Syber Green, FRET probes, TaqMan probes, and molecular beacons. "What we are moving to in the laboratory is real-time," Dr. Ginocchio says. Simultaneous amplification and detection favorably affects turnaround time and thus patient care. Real-time PCR entails a major initial outlay for the instrument, "but you save a tremendous amount of technologist time and reduce TAT greatly," Dr. Ginocchio says. Equipment for real-time non-PCR-based applications such as NASBA and TMA is significantly less expensive since the reactions are isothermal and don't require a thermocycler. "And you reduce the risk of contaminating your laboratory with amplicons," she says. Also, it is easy to develop applications for real-time assays. Many real-time PCR instruments are available, such as the Corbett RotorGene, Cepheid Smart Cycler, Bio-Rad iCycler, Roche LightCycler, Applied Biosystems 7000, and Stratagene Mx4000. The BioMérieux EasyQ system is a real-time instrument for NASBA applications.

To detect RNA viruses, Dr. Ginocchio uses NASBA and molecular beacons; for DNA viruses, she does real-time PCR. Molecular methods are ideal for laboratories without classical virology services, those that don't have the ability to grow viruses in tissue culture, Dr. Ginocchio says. (Even in laboratories that do perform viral culture, molecular methods often provide a safer alternative to culture-for identifying the coronavirus that causes SARS, for example.) And molecular methods have been demonstrated to be more sensitive than traditional methods in "dozens of journal articles, if we're just talking about respiratory viruses here," she says.

For instance, one group looked for respiratory viruses in bronchoalveolar lavage fluids from 43 patients with hematological cancer and pneumonia. By culture or antigen detection, nine pathogens were found in eight samples (19 percent detection rate); PCR found 17 pathogens in 15 samples (35 percent), a significant increase. Other investigators verified that PCR and multiplex RT-PCR are more sensitive for RSV, parainfluenza virus, rhinovirus, and adenovirus (Freymuth F, et al. Clin Diagn Virol. 1997; 8:31-40; Osiowy C. J Clin Microbiol. 1998; 36:3149-3154).

On balance, Dr. Ginocchio says, although DFA and rapid antigen tests are generally fast (less than two hours), their sensitivity and specificity are typically lower than those of amplified tests. Rapid antigen tests are fastest (less than 30 minutes) but least sensitive. Real-time molecular assays, on the other hand, are much faster than conventional amplification assays and can provide results from start to finish in as little as 1.5 hours, which is comparable to DFA.

A major consideration in selecting test method is cost. Dr. Ginocchio lists these representative setup costs for various methods: antigen tests, $0 to $200 (pipettes); DFA, $15,000 to $25,000 (biohazard facility, cytospin, microscope); viral culture, $30,000 to $40,000 (add centrifuge); molecular, $30,000 to $250,000 (extraction and amplification instruments). Test costs range from $20 to $50, with molecular tests on the high end. Using R-Mix for viral cultures adds acquisition cost but saves money overall.

Molecular tests often require greater technical expertise depending on the complexity of the assay and on whether the assay is commercially available or developed in-house. Dr. Ginocchio had two positions open for a year in the molecular laboratory. Her approach was to develop more defined, standardized protocols for in-laboratory assays she developed so that less-experienced technologists could readily perform them. Real-time molecular assays are coming out with more well-developed protocols; however, all laboratories need to do in-house verification and validation studies.

A second challenge is the lack of FDA-approved tests. "Often, manufacturers don't want to go through the expensive FDA approval process, particularly for assays without significant test volumes," she says, "so we must verify and validate these assays and run numerous appropriate controls to ensure the validity of our test results and the technical competency of our staff." Her solution: "We need to put pressure on manufacturers to bring more assays to the FDA." This is particularly important for smaller labs that don't have the ability to develop in-house assays or perform the more in-depth verification and validation studies required for non-FDA-approved assays.

Overall, Dr. Ginocchio says, molecular methods cost more initially but offer greater sensitivity with equal or better specificity, are faster, and save money in technologist time.

One place where molecular methods are valuable is in diagnosing pertussis, especially in light of the atypical presentation seen in adolescents and adults who now contract this infection due to waning immunity. Newer, more sensitive tests are contributing to the recognition of pertussis as a re-emerging infection, Dr. Loeffelholz says. Molecular methods are valuable also for their specificity: While most pertussis is due to Bordetella pertussis, one percent to 30 percent of cases are caused by B. parapertussis.

The tools used to diagnose Bordetella infections are culture, DFA, serology, and molecular. Dr. Loeffelholz last July surveyed participants on the ClinMicroNet e-mail discussion list, most of whom are laboratory directors, about Bordetella testing capabilities; 35 laboratories responded. Of the 24 laboratories using nonmolecular methods, 13 used culture and DFA, eight used culture alone, one used DFA, and two sent samples out. None depended on serology. Eleven laboratories, or 35 percent, offered molecular methods, chiefly PCR. Four laboratories were using conventional PCR and seven had adopted real-time PCR. "The real-time bandwagon is rolling on," Dr. Loeffelholz says.

(Dr. Loeffelholz was surprised to find that four of five public health laboratories were using PCR, compared with only seven of 25 hospital laboratories. "Historically, one thinks of public health laboratories as entrenched in more traditional methods, in part due to restricted funding," he says. The reality of bioterrorism, he adds, "has made a huge impact" on money available to public health labs. This funding has enhanced testing capabilities outside of bioterrorism.) Culture provides high specificity and variable sensitivity, but is affected by specimen transport, since Bordetella is labile.

Culture provides isolates for antibody susceptibility testing, erythromycin resistance (which occurs at a very low rate), and strain typing in outbreaks.

DFA is more rapid, but has poor sensitivity and variable specificity (positive predictive value <50 percent has been reported). The Centers for Disease Control and Prevention does not recommend DFA as a primary diagnostic test for Bordetella.

Serology may be the most sensitive method, but optimal specificity requires paired specimens, to detect seroconversion or a fourfold rise in the antibody titer. However, it is not always possible to get a patient to return in two to four weeks. There is no FDA-cleared serology test in the United States and no standardized interpretation criteria.

Among molecular methods, PCR predominates. Primary amplification targets are insertion sequences (IS), pertussis toxin, porin, and RecA. One insertion sequence, IS481, is present at 80-100 copies/ cell (which increases sensitivity) but is not specific for B. pertussis; it is also found in B. holmesii. IS1001 is present at 20 copies/ cell and is almost specific for B. pertussis. Conventional and real-time PCR multiplex assays are available for B. pertussis and B. parapertussis (Sloan LM, et al. J Clin Microbiol. 2002; 40: 96-100). Of the 11 laboratories in the survey offering molecular methods, only three included B. parapertussis.

Dr. Loeffelholz delineated four possible solutions to the nonspecificity problem. First, use a target specific for B. pertussis, such as RecA, porin, or pertussis toxin. Second, use real-time PCR and do melting curve analysis. Third, use both IS481 and IS1001, with this scoring: +/-, B. pertussis; -/+, B. parapertussis; +/+, B. holmesii (Templeton KE, et al. J Clin Microbiol. 2003; 41: 4121-4126). Fourth, use IS481 and report positive results with a disclaimer. Macrolides are effective against B. holmesii.

Dr. Loeffelholz reviewed 14 studies comparing PCR to culture. PCR ranged from 1.1- to 9.6-fold more sensitive than culture. However, it is difficult to pinpoint a true multiple, he says, since studies from the 1990s did not always resolve discrepant results. Lower multiples typically occurred in studies done in pediatric populations or when specimens were transported to the laboratory in less than 24 hours. These conditions can influence which test a laboratory adopts.

Asymptomatic people who are PCR-positive for Bordetella present another conundrum. We can assume that B. pertussis bacteria or DNA are present in the specimen but that the person was transiently colonized and didn't develop disease because of a preexisting immunity. What is the clinical relevance of a positive PCR result in this person? We might call it a clinical false-positive. Whether such a person can transmit disease is not known. Clinical false-positives are particularly common in children: 92 percent of PCR-positive children one to six years old and 33 percent of those seven to 12 years old never develop symptoms. Should they be treated? Are they Bordetella reservoirs? No one knows.

The problem of possible PCR false-positives also arises during therapy. In infants and neonates culture positivity decreases to zero after seven days, while 56 percent of specimens remain PCR-positive at this time (Edelman K, et al. Pediatr Infect Dis J. 1996;15:54-57). Dr. Loeffelholz says these figures illustrate the challenge of "interpreting positive results from sensitive tests that don't require the presence of viable organisms and that don't always agree with clinical response."

He concludes that molecular diagnostics offers high sensitivity and detection of more Bordetella infections while helping clarify the organism's epidemiology. Remaining challenges include standardization and quality assessment.

Speaking more generally about whether to adopt molecular tests, Dr. Loeffelholz advises, "Consider your patient population. Traditional methods might perform adequately." Consider, too, how you will implement real-time molecular tests. "If a method has a two-hour turnaround time, does that mean a technologist is going to be standing there with open arms 24/7 to receive specimens as they come in and get a result out in two hours?" Dr. Loeffelholz asks. "Probably not."

Think about volume and specimen batching. Ultimately, adopting molecular testing will require a comprehensive approach. "You need several tests to justify setting up a molecular capability," Dr. Loeffelholz says. One or two is not enough.

Amidst all the talk of technology, Dr. Caliendo brings the discussion back to its critical core. “The most valuable resource that we have in our labs is our technical staff,” she reminds us. “We need to focus on them.”

William Check is a medical writer in Wilmette, Ill.