College of American Pathologists
CAP Committees & Leadership CAP Calendar of Events Estore CAP Media Center CAP Foundation
 
About CAP    Career Center    Contact Us      
Search: Search
  [Advanced Search]  
 
CAP Home CAP Advocacy CAP Reference Resources and Publications CAP Education Programs CAP Accreditation and Laboratory Improvement CAP Members
CAP Home > CAP Reference Resources and Publications > CAP TODAY > CAP TODAY 2004 Archive > Playing by the rules with expert software
Printable Version

  Feature Story

title
 

cap today

Playing by the rules with expert software

November 2004
Anne Paxton

A lot has changed in the core laboratory of Oklahoma University Health Sciences Center and OU Medical Center since total automation was installed last March. The lab no longer has to pour samples into secondary tubes or pull tubes off racks, and specimens are positively identified from order, collection, and testing through to reporting. The staff is less stressed because there are fewer exception samples to handle and more time to process the few they do have. The CEO of the university hospital likes to show off the laboratory to Joint Commission inspection teams because the reduced risk of error is there for all to see.

But one of the most striking changes is that virtually all the laboratory’s testing is now performed in real-time—first in, first out. In most cases, there’s no queue for a stat test to jump ahead of. "When a specimen shows up, it’s processed immediately," says Kenneth E. Blick, PhD, director of clinical chemistry. "I tell the doctors they can still order tests stat—but they won’t get them any faster."

But the credit doesn’t just go to automation. He says expert systems technology has also been key to helping the lab dispense with test queues and batch processing—and become a workplace less prone to panic. Years ago, up to half of the laboratory’s work at Oklahoma University Health Sciences Center and OU Medical Center was ordered stat, creating what Dr. Blick describes as a "downward spiral."

"Once you have to worry about putting stats ahead of routine work, that delays the routine work, then you have technologists trying to do the tests while the physician calls looking for it, and here they are on the phone, which delays the tests even more. Then tomorrow those routine tests are ordered stat," he says.

Only a few years ago, the shift that took place at OU that made it possible to eliminate stat testing couldn’t have taken place, Dr. Blick says, because the connectivity and expert decision-making software weren’t there. "It was a good idea at that point, but nobody was really ready to implement it properly," agrees Gregory Vail, CEO of Data Innovations, a 15-year-old data management company that focuses on the clinical laboratory market. "Now we’re really hitting our stride, and people are seeing dollar savings in preventing medical errors."

Expert systems emulate advice that an expert, such as a technologist, would provide, but they do it more efficiently and consistently. The term refers to any software application that takes a user’s request for advice, applies a set of rules to it, and responds according to a knowledge base. The rules engine, sometimes called an inference engine, is the hub of this "rules-based technology."

Rules take the form: If some condition is true, then perform a certain step, else perform a different step. For example, consensus rules developed recently by the International Society for Laboratory Hematology include the following: If an MCV test result is <75fL or >105fL for an adult, and it’s the first time, and the specimen is <24 hours old, then conduct a slide review.

Other test results might lead to a request for a fresh sample or a verification of the sample’s identity. If none of the conditions apply, then the result can be released. (See www.islh.org for a complete list of the 41 hematology rules.)

Software companies in the laboratory field, however, point out that consistently identifying all conditions that trigger these rules is nearly impossible to do manually. Of the 41 hematology rules, 18 are based on instrument flags that are not numerical results and are often not clearly displayed by the lab information system. Others specify limits with regard to specimen age, which could require time-consuming searches for information comparing run-time to draw-time (also not routinely displayed by the LIS). "Without a computer addressing these specific rules issues, expecting a laboratorian to spot all these conditions would be time-consuming and probably unreasonable based on all the other issues technologists deal with day to day," says Curt Johnson, vice president of sales and marketing at Orchard Software.

That’s one reason why the most common use of rules in clinical laboratories to date is to auto-verify or clinically validate results. But actions can also include "notifying somebody that something happened, adding new orders to patients’ test files, canceling orders, creating results, getting a certain set of data, and based on that, having a rule to create new results, or adding comments," says Arthur Hauck, MD, physician executive at Cerner Corp. Cerner’s Millennium LIS platform has offered clinical validation since early 2003, and clinical validation is now available on its Classic platform as well.

Virtually all computer programs rely on if-then rules, or decision trees. So what’s unique about rules-based technology? Says Data Innovations’ Vail, "Once you get down to the computer doing the logic, there probably is not that much of a difference" between any computer program and expert systems. Data Innovations sells its products to both LIS vendors and instrument vendors. Instrument Manager is the company’s main product, with imExpress offering a simplified interface designed for small LISs. "But in a computer program," Vail continues, "you’re ’hard-coding’ the branching. With a rules engine such as we’ve implemented, the branching is constructed at the user levels." They’re using drop-down boxes to pick data elements they want to operate on, he says, and then picking operators to check those elements against another data element or a fixed user-defined value. "It’s not hard-coded; you’re really building rules as you need them," he says. "And we execute those rules on a real-time basis as data comes through, so you get much more flexibility and usability."

A data-management system is not an LIS. "An LIS typically is a big dollar investment in the U.S., and very hard to change, whereas a data-management solution like ours is much less expensive," Vail says. "So you can bring us into a situation where you have an older LIS and make big changes to the workflow of the laboratory at a lot smaller price and implementation time." Data-management systems are also valuable during LIS downtime, he says. "With the sophistication of laboratories today, when the LIS is down you can’t run the lab. By queuing up all the results, this lets you continue testing until the LIS is back up."

What most labs are looking to Data Innovations for, Vail says, is auto-verification. "We do that through a simple yet very powerful rules engine. The rules engine has a ton of data elements, operators, and actions for the user to choose from."

The rules can take popular data elements like patient sex or age and combine them with individual tests, a group of tests for one instrument, or tests for the same specimen across different instruments. "Some systems force you to write too generic a rule—for example, for this test between X and Y value, release the result. That’s probably not 100 percent the right way to do it, because the range that delineates what a good normal value is might change by age or sex or both or more. In fact, we’ve taken it all the way to the level where one of the data elements is ’doctor,’ so the rule may vary by doctor as to what they think is normal."

Having to do computer programming would frighten most people, Vail says. "This is at a nonprogrammer level." It’s a Windows form with "if" and "then" boxes with drop-down menus. It guides the user through entry of the rules. For example, if the user picks "test result of," then it automatically prompts the user to give the test code, then to pick an operator such as greater than, equal to, less than, and so on. "It’s constructing rules for you as you go," Vail says.

With the graphical user interface, the "then" box allows the user to click on action boxes to have the system, for example, hold the test for verification, dilute and perform a rerun, or perform reflex ordering. "One of the big keys to our rules engine is that it’s so easy for the user, yet at the same time so flexible and comprehensive." Vail cites as an example the fact that Data Innovations was not involved in developing the consensus rules of the International Society for Laboratory Hematology but, as it turns out, the rules can be fully implemented via the company’s rules engine.

When Vail got his computer science degree in 1987, the prediction was that artificial intelligence would let computers learn to perform human tasks without being programmed with the logic for each decision. "We haven’t seen that come to fruition anywhere," he says, "but that doesn’t mean the idea of expert systems or rules engines isn’t valuable. Instead of going for ’pie in the sky’ artificial intelligence systems, people have backed down and said, hey, rules engines have a lot of benefits."

They do—and the farther out you place expert rules technology, at the point of care or when a physician or patient event needs to occur, the more impact and value they provide, says Mark Spencer, vice president of Horizon departmental solutions for McKesson Corp.

Horizon software products include an application for the LIS, Horizon Lab, which provides wireless handheld applications for specimen management and positive patient ID. One new area for McKesson expert applications has been the integration of laboratory and pharmacy data to provide clinical support at the point of care.

Often a patient is started on antibiotic therapy before the culture result is complete. Within Horizon Lab, a microbiology technologist is able to view a patient’s medication profile. "With discrete microbiology data, a rule will trigger an alert to the pharmacist’s work queue,"Spencer explains. "The pharmacist can review the culture and antibiotic sensitivities, then alert the physician to recommend changing therapy." Time is saved because the information comes real-time from Horizon Lab.

Horizon Lab also displays the cost of susceptible drugs, which will be on the patient report, and online results inquiries. "We’ve chosen to indicate using dollar signs and have also sorted the results in cost-code order so the physician views the less-expensive medications first," in hopes of influencing order behavior, Spencer says. "That’s an example of using expert rules and information to marry laboratory and pharmacy data."

Rules-based technology can bring software that is much more tailored to the laboratory, explains Dr. Blick, who for many years has also been director of laboratory information systems. "’Hard-coded’ software might be written by somebody sitting in a cubicle who doesn’t even work in the laboratory but writes a laboratory software solution. Sometimes it’s not a good fit, or it’s written in a language that’s not adaptable and has to be reprogrammed and re-compiled." If it’s a legacy LIS, it might be written in a language no one programs in anymore.

"So when you’ve got this legacy fixed solution that’s not solving the exact problem, people do work-arounds; you sort of trick the software to get what you want it to do. Technologists are very good at that," Dr. Blick says.

Software like that can handle 80 to 90 percent of the problems, but there’s that 10 or 15 percent it can’t handle, he says. "Suppose a doctor says, When I send you a sample on a pediatric patient in dialysis and order a creatinine, I want to send it to the reference laboratory and get it done by HPLC. I’m going to want that four times a year."

What are the chances of people handling that correctly? "If I don’t have some kind of scripting language to identify the doctor, the patient, and so on, we’re going to screw up every time it comes down—because it will be on a weekend or holiday when I have a part-time person working," he says.

You end up with a computer with a bunch of Post-it notes on it because the legacy system has code designed to solve problems that may not exist anymore. As Dr. Blick puts it, it becomes a process of "trying to chop a tree down with a hammer."

"If something happens that’s an error," he says, "are you going to in-service everybody to hope they’ll recognize that combination of events when it happens—or put in a rule to trap this event the next time it occurs? You’ll never get someone to remember all the variations that can happen in a laboratory. So I need a layer of decision-making software to support my legacy-coded hard software system to handle the 15 to 25 percent of nuances that hard programming doesnhandle well, instead of using workarounds and Post-it notes." The same applies to auto-validation, Dr. Blick points out, because there it is hard to get people to look at samples and come up with the same grades on hemolysis, for example. "Yet we can get a quantitative measurement and write a rule to decide whether a sample is acceptable or not for which sex and age of patient and for which doctors."

Auto-approval of results is what most people think of when they think expert systems, says Johnson. "We don’t necessarily think that way at Orchard, because if you are going to design a system and use it to improve efficiencies and workflow, it has to be able to think of everything a technologist thinks of when making a decision. If there’s a criterion the med tech is going to use to make a decision and the computer can’t evaluate it, then the med tech won’t trust it." In effect, he says, that means you don’t have an expert system.

Orchard Software is the maker of Aqueduct, middleware that links Beckman Coulter hematology analyzers with lab information systems. The company’s main product is Harvest LIS, and its newest product, Copia, is for outreach, offering Web-based order entry and results retrieval.

Auto-approving normal results is easy, Johnson notes. "It’s the abnormals that take all the time, and if you could offer decision support and matrices on what the next step should be, then you’re offering true benefits to the laboratory." While all LIS manufacturers say they have rules technology, what sets Orchard’s apart, he says, "is the ability to evaluate all the criteria."

In chemistry, for example, the rules can be set all the way down to patient levels. "If the patient has had a recent kidney transplant, you want to make sure the values don’t vary significantly, but you don’t want rules that flag every creatinine." Orchard, he says, puts the following kind of detail into its rules engine: "If Jim Johnson’s creatinine moves 10 percent or X number of units over the next week, flag me."

In hematology, rules might deal in time frames, he says, citing the International Society for Laboratory Hematology’s standards for doing auto-approval. "If the blood is only three hours old and these results are present, do this. However, if it’s seven days old, there would be a different set of criteria because some of the parameters will move." Orchard’s system incorporates all of the International Society’s different rules for hematology.

"We’ve tried to develop our whole LIS to be an expert system, so we have rules not just for test evaluation but also for ordering, billing, and reporting," Johnson says.

Interestingly, a lab doesn’t have to trash its existing LIS to reap the benefits of expert systems. Says Dr. Blick: "We have an older legacy LIS, a Meditech version 4.9.1, which frankly does a good job. We didn’t have to update it. Our LIS was never very good at auto-validation, and we hadn’t really used that feature. We still had technologists doing it, with results reporting delayed 30 minutes to an hour and even longer."

What the lab at Oklahoma University did instead was acquire Beckman Coulter’s middleware to take over auto-validation. The laboratory’s track, which automates 90 different tests, was manufactured by Beckman but also handles Bayer’s Centaur instruments.

"We brought decision-making for auto-validation down to the data link that connects the chemistry analyzers together," making it possible for the lab to extend the life of its legacy LIS, Dr. Blick says. "I thought it would take six months to start realizing benefits," he adds, "but we started seeing enhanced services within one week."

The core lab at Oklahoma University has data to show that for every percent or so it fails to meet its turnaround time targets, it increases wait time in critical care by up to 20 minutes. "So it’s very important that we get our laboratory data back" promptly, he says.

The sheer quantity of rules that some expert systems are able to apply is one of the systems’ most striking features. In fact, says Gilbert Hakim, CEO of SCC Soft Computer, the large numbers are at the heart of rules-based technology.

"There are many different areas of the laboratory in which rules facilitate an improved product generally. But you cannot really get the benefit unless you have massively large numbers of rules. The question," Hakim says, "is not whether you can have a rules engine; it’s really how many rules you can apply, and you have to have thousands of rules apply without slowing the system down."

With an average of 500 to 2,000 physicians at the hospital sites, and 1,000 to 3,000 laboratory tests, Soft Computer’s laboratory information systems employ two parallel processors connected to the Internet. Hakim says this feature allows them to include large environments among the company’s 500 sites. For example, Soft Computer customer Kindred Healthcare has more than 40 hospitals in four different time zones connected to a single database in Louisville, Ky., while Gambro Healthcare, based in Lakewood, Colo., uses SoftLab to handle 20,000 requisitions a day on peak days.

In blood banking, expert systems are used more and more to allow customers to set up rules for temperature requirements for storage and shipment of units from donor collection and component manufacturing to final distribution, says Miklos Csore, vice president of research and development at Wyndgate Technologies, a division of Global Med Technologies. Users of Wyndgate’s SafeTrace donor management system and SafeTrace Tx transfusion service management system, which automate the management and tracking of specimens, orders, and blood products, can decide how they want the system to work by choosing acceptable temperature ranges for blood product storage and distribution, as well as setting up rules for patient and product compatibility, electronic crossmatching, safety checks, and override options.

Similarly, clients can customize their interpretations of industry regulations and standards. "What we’re finding," Csore says, "is that while some regulations are more specific, there can be quite a bit of leeway in how others are implemented. If we have to deal with 10 customers, five will implement regulations one way, and five a little differently, and the system is going to have to handle both."

How long does it take a typical laboratory to set up the rules for an expert system? "It’s not usually an A-to-Z approach," Vail says, estimating that technologists can finish it within a couple of weeks to a month. "People usually sit down and set up a minimal set of rules at the beginning because lots of people like to take things slowly."

"There’s somewhat of a ’feeder factor’ as they test it out, and there’s the factor that they may not know everything they want or can do," he says. "So they start slowly with a few results, see how they work, then the light bulbs go off and they add one rule and another rule, and so on."

Setup depends on the size and complexity of the installation, says Cerner’s Dr. Hauck. "Having written many myself, I could typically do a rule in anywhere from 15 minutes to an hour, depending on how complicated it is."

"But it absolutely does not require any programming. Anyone can do it. Our rules system uses a Windows-based application and templates with actual links referring to data in the database. So when I need to plug a test name or a result name into a rule, I can choose from a list in the database," he says.

"You set up a rule once and in the beginning, but once it’s set up, it does its thing for potentially thousands of laboratory results a day." For a critical result, for example, the rule can be used to either page the clinician or send an e-mail or put a message in the clinician’s inbox.

"In microbiology, scripted work ups allow us to let, say, a gram-positive cocci kick off a cascade of events based on rules we’ve set up," Dr. Hauck says. Hakim says that implementation of Soft Computer’s systems takes from nine to 12 months depending on size. "But we have done rapid implementation that takes less than six months," he adds.

The Misys Laboratory and Misys Insight systems use rules-based technology to auto-file laboratory results in general laboratory and microbiology, and to perform other tasks such as automatically sending out lab result notifications to caregivers, says Debbie Tillman, senior product manager at Misys Healthcare Systems. With Misys Laboratory, the user can enter different rule sets based on patient age, location, test value, and whether quality control has passed or failed, and so on, to allow auto-filing without technologist review. "About 75 percent of the laboratory results filed in the general laboratory are normal, and with ever-decreasing med tech staff, being able to apply these rule sets has proved to be very valuable," Tillman says.

Laboratory personnel using Insight gain workflow efficiencies in results reporting by replacing manual processes such as phone calls, faxing, and printing with an electronic notification system. The majority of Misys customers are hospitals and integrated delivery networks with test volumes exceeding 500 to 1,000 specimens a day. It takes approximately 36 weeks for a typical installation, Tillman says. "Misys implementation trains the customer on the functionality, demonstrates how the rules can benefit their workflow, and how to define new rules."

Wyndgate’s SafeTrace systems take from three months to a year to set up, Csore reports. Using a train-the-trainer approach, Wyndgate’s staff trains a core group of system experts, who in turn train the users. "That way the client project team understands the system and can set up the tables and the system using the rules that work in their operation." Training people in the many new, required standard operating procedures, he says, is probably the chief challenge of an installation.

Industry officials forecast that expert systems will pervade the laboratory. Orchard Software is expanding into anatomic pathology and the molecular diagnostics industry, says Johnson, who predicts the company will release its anatomic pathology product in 2005 and its molecular diagnostic product within three to five years.

The new generation that Soft Computer is working on will automate the back end of the laboratory through workflow engines that make it possible to monitor people and assignments, Hakim reports. The SoftWorkflow module, which Hakim says will be released in early 2005, includes a courier tracking system. Couriers and courier vehicles can be equipped with handheld GPS and Internet palm devices so that managers can electronically order collection and delivery of supplies based on where drivers are located.

More sophisticated logic for auto-verification is in the works at Cerner. "As it develops we’ll be looking at more parameters, making it more comprehensive," Dr. Hauck notes. Beyond the laboratory, he sees a need for expert systems to help clinicians understand the results of laboratory tests and how to work up a particular problem, including the effect that drugs the patient is taking are having on the test result.

Hospital-based laboratories can use rules to gain a competitive edge, says McKesson’s Spencer. "What helps the laboratory differentiate itself is the customized and localized services they can provide, versus a national reference laboratory that basically says ’Here’s the result.’"

The hospital-based laboratory that wants outreach work from the physician market can tailor its service to the way the physician wants to interact. With expert rules, the lab can have physician-specific normal ranges and alerts, and thus deliver customized reports to its physician customers, Spencer says. "But there are also expert rules that are based on past events, so you can start to build ordering patterns, the most commonly ordered tests and normal ranges specific to that physician."

But, in Spencer’s view, the Holy Grail of expert systems right now is computerized physician order entry and adoption. "Moving laboratory content out to the physician is key."

Dr. Blick agrees: "Right now we have nurses and clerks doing orders on the floor, but we’re looking at having physicians do their own ordering and having expert systems guide them." Dr. Blick describes the improved handling of neonatal bilirubin at his hospital to illustrate how great an impact expert systems can have on quality laboratory testing. Before the core lab installed its new systems, neonatal bilirubin created chronic problems.

"We can’t send the baby home with mom and the grandparents unless the bilirubin is below a certain number. But we often got heelstick samples that hemolyzed, and our method gives mostly elevated results on hemolyzed samples. Then we’d have to ask for a second sample from the other heel, and sometimes a third. About now, we have tension."

They started collecting the samples in lithium heparin, and that virtually eliminated the problem. Then they also came up with an enzymatic test on the chemistry analyzer that doesn’t have a problem with hemolyzed samples. "So now," he says, "when there is a hemolytic index of 8 or greater, they get an alert on the screen and the technologist is instructed to take the sample and run it on the instrument with the enzymatic bilirubin with no hemolysis problem. As a result, babies are getting discharged on time."

The expert system is "exponentially better" than the old manual system, he says, stressing the importance of rethinking all the processes in the laboratory. "Don’t use the saying, ’If it ain’t broke, don’t fix it,’" he warns, noting that continuous improvement is a requirement of the CAP and the Joint Commission on Accreditation of Healthcare Organizations. "All processes are essentially broken," he insists. If you don’t see it, "you just haven’t looked closely enough."

Anne Paxton is a writer in Seattle.

   
 

 

 

   
 
 © 2014 College of American Pathologists. All rights reserved. | Terms and Conditions | CAP ConnectFollow Us on FacebookFollow Us on LinkedInFollow Us on TwitterFollow Us on YouTubeFollow Us on FlickrSubscribe to a CAP RSS Feed