|
May 2005
Feature Story
Current QC protocols
Risk management model
EQC Option 4
QC as is for now
When the Centers for Medicare and Medicaid Services outlined three new
options for performing what it calls "equivalent quality control," or
EQC, under CLIA ’88 in January 2004, the agency expected that the options
would better accommodate new technologies, reduce the frequency of QC
for stable test systems, and provide flexibility. What it didn’t expect
was that EQC would be so confusing for so many labs to implement. "After
we released the three EQC options in our CLIA surveyor guidelines, it
wasn’t long before we started hearing concerns. We had calls and e-mails
from laboratories, manufacturers, professional groups, and accrediting
organizations," said Judith Yost, MA, MT(ASCP), director of the CMS Division
of Laboratory Services, speaking at the March 18 "QC for the Future" workshop
in Baltimore. "There is innovative and new technology that we did not
anticipate, and we have created some policies that are perhaps inconsistent
with what the accrediting organizations have," she explained. Recognizing
that it may be time to re-evaluate the direction laboratory QC is taking,
Yost, her fellow CLIA ’88 regulators, and professional organizations—including
the CAP—partnered with the Clinical Laboratory Standards Institute
to convene the "QC for the Future" workshop. "Through this workshop, we
want to generate the framework for the development of future quality control
consensus protocols," said Thomas L. Hearn, PhD, president of CLSI and
associate director of the Division of Laboratory Systems, Centers for
Disease Control and Prevention. The CMS could then potentially use the
consensus protocols to turn the one-size-fits-all CLIA regulation into
one that provides scientifically based QC options for labs that are embracing
innovative technologies.
Current QC protocols
As laboratorians know, CLIA ’88 now requires that two levels of external
QC be performed each day for nonwaived tests, and more frequently for
certain specialty and subspecialty tests. CLIA regulators have long recognized,
though, that this may not be an optimal way to perform QC. "When we first
began to discuss QC under CLIA ’88," Yost said, "both the CDC and CMS
had meetings with nationally recognized experts regarding quality control.
Because of the variation in laboratories, and in the types of technologies
and test methodologies, we were unable to come to any real conclusions,
so we defaulted to the two external controls per day." Every test system
may not need this level of external QC at this frequency, she concedes,
and that’s one of the reasons she and others created the equivalent quality
control options.
The EQC options are detailed in CLIA’s interpretive guidelines. Basically,
the first option applies to test systems with internal QC that monitors
all analytical components of the test process, the second option addresses
systems with internal QC that monitors a portion of the analytical test
components, and the third option can be used for stable test systems without
built-in controls. But the use of these QC alternatives is contingent
on a performance evaluation by the laboratory that institutes the EQC
option, and a test is not eligible if the manufacturer requires more frequent
controls in its package insert. A number of specialty and subspecialty
tests are excluded from EQC as well.
Speaking at the workshop, James O. Westgard, PhD, president of Westgard
QC Inc., pointed out the drawbacks of using the current EQC options. "If
you do any of the EQC protocols and you don’t have any rejection signals,
that is not evidence that things are okay. You have not tested whether
the system would even respond if you had problems," he said, adding that
before regulators think about reducing QC, they should be sure the analytical
performance of current assays is truly acceptable. He provided data showing
that for most of today’s common tests, the quality of the result is three
to four Sigma on the Six Sigma scale, with six Sigma being the goal for
world-class quality and three Sigma being the minimum acceptable quality
for a routine testing process. He showed that the CLIA minimum QC requirement
of two levels per day is appropriate only for methods having five Sigma
quality or better and recommended that methods demonstrate six Sigma performance
for adoption of any EQC option.
In addition, if there is a QC failure when using one of the EQC options,
the lab must repeat the test to determine whether the problem is isolated.
If the problem is a fluke, the lab can continue testing. If not, the lab
must return to the last successful run and repeat tests on all of the
subsequent samples to be sure the results are accurate. With the first
EQC option, labs are allowed to go 30 days without doing QC, which means
if there is an inexplicable failure, they may have to retest samples as
far back as 30 days. This can be a problem for labs that keep samples
for only three to seven days.
"At the moment, no one is pleased with the EQC options," Gerald Hoeltge,
MD, treasurer of the Clinical Laboratory Standards Institute and the CAP’s
liaison to the CLSI, told CAP TODAY. "If you look at the surveyor guidelines,
I believe the approaches detailed there are scientifically difficult to
explain and may be risky to implement, depending on the instrumentation."
What he heard "loud and clear" at the workshop from many in the laboratory
community, he says, "was that you can’t do quality control any less frequently
than the stability of the sample. In fact, he adds, any QC intervals that
are longer than the stability of the sample are out of the question from
the user’s point-of-view.
Thomas L. Williams, MD, director of the Department of Pathology at Nebraska
Methodist Hospital, Omaha, and a member of the Clinical Laboratory Improvement
Advisory Committee, agrees with Dr. Hoeltge’s assessment. "Those of us
on the laboratory side were quite baffled about how to approach EQC. If
you look at the evaluation processes for the three different options and
how many times you do controls, and what you must do if QC fails, none
of that seemed to make sense to any of us," he told CAP TODAY.
Dr. Williams, who was at the workshop, says there’s a disconnect between
the way laboratorians view the EQC options and the way in vitro diagnostics
manufacturers approach EQC. "Those in industry seemed to immediately look
at EQC as a risk management question, while those of us in the laboratory
looked at it as a quality control question," he says. "I’ve reached the
conclusion that for us to proceed with EQC, we really need to learn more
about risk management in the laboratory, and look at EQC as a risk management
challenge."
Risk management model
Why does risk management make sense for clinical laboratories? "It provides
greater insight into the patient safety risks that we’re all trying to
deal with, and it supports systematic, well-informed decision making regarding
QC and other risk controls," said Donald Powers, PhD, a quality systems/regulator
compliance consultant and founder of Powers Consulting in Pittsford, NY,
who spoke at the workshop. "It also fosters quality by design, continuous
improvement, and leads to enhanced predictability of results," he said.
"QC should be thought of as a form of risk control."
In addition, risk management dovetails with other quality system initiatives
that laboratories may be undertaking already. "Risk management has been
incorporated into the regulatory schemes of all the major industrial countries,
and the new European IVD directive places a very strong emphasis on risk
management as an essential requirement," said Dr. Powers, adding that
the quality systems regulation in the U.S. has similar requirements.
Dr. Powers explained that the internal built-in monitoring systems used
in a number of today’s IVD devices exist because of IVD manufacturers’
risk management process. If laboratories would use risk management in
those parts of the process under their control, the partnership could
be powerful. "One of the things that concerns me is that the manufacturers
are doing a lot of risk management, but the laboratories haven’t really
started yet," he said. "However, they are an integral part of making sure
that the risks are minimized to ensure the safety of patients. So we have
to find a way for manufacturers and labs to work together."
If labs want to adopt a risk management approach to QC, guidance is already
available to them through the International Organization for Standardization,
or ISO. ISO standard 14971 is a general life-cycle risk management approach,
and new guidelines for applying the standard to IVD products were developed
recently by ISO TC212 and added to ISO 14971. "Applying risk management
to in vitro diagnostics, where failure has an indirect impact on the patient,
is a little more complicated than applying it to pacemakers or dialysis
machines, where failure causes direct harm. That’s why we needed additional
guidance," Dr. Powers said.
The new IVD risk management guidelines describe one risk model that begins
with a manufacturer experiencing a failure in a quality system, which
leads to a defective IVD medical device. This, in turn, causes the laboratory
to have a failure in its testing process, which then produces an incorrect
result. Since the incorrect result in the physician’s hands can lead to
a wrong diagnosis or an inappropriate treatment, it is considered a "hazardous
situation" in the terminology of the standard. As defined in the standard,
a "hazard" may lead to a hazardous situation, but not all hazards lead
to hazardous situations, and not all hazardous situations lead to harm.
For example, though an incorrect test result may create a hazardous situation,
it won’t cause harm unless the physician uses the incorrect result to
change the course of therapy or treatment in some way.
Risk management sounds like it might be good in theory, but could it
work? Says Dr. Williams, "Delineating the boundary between where the manufacturer’s
responsibility ends and where the lab’s responsibility begins is going
to be difficult, and I think as I understand the risk management concept,
we in the laboratory are going to have to understand enough of our own
risk management environment to know where failures are likely to happen
as their device is used with our operators in our environment."
Communication will be key in the future because labs will need information
from manufacturers to control risks and monitor hazards over the life
of the device. "We want the manufacturers to be able to communicate with
us about what the potential points of risk are in an instrument," workshop
attendee Stephen Sarewitz, MD, CAP Laboratory Accreditation Program checklist
commissioner, told CAP TODAY. Manufacturers, he says, tend to want to
put out the instrument as a black box. "That may or may not be true,"
he concedes, "but since we have responsibility for the result, we believe
we have to be given some degree of insight into how the test works to
make sure it’s working right."
Powers agrees that more open communication is necessary. "The ISO standard
requires manufacturers to disclose any residual risks to users," he says.
EQC Option 4
IVD manufacturers proposed another possible solution to the EQC conundrum,
and its working title is EQC Option 4. "We wanted to propose something
that made scientific sense," said Luann Ochs, director of regulatory and
reimbursement affairs at Roche Diagnostics and chair of the AdvaMed CLIA
Working Group. "So we called CMS, CDC, and FDA into a meeting at AdvaMed
in May 2004, and we proposed what we called Option 4, which we believe
is a scientifically valid process for establishing equivalent quality
control while still maintaining government oversight."
At the moment, Option 4 consists of three steps. First, the manufacturer
performs a scientifically rigorous risk assessment and validation of a
proposed alternative QC protocol that would provide substantial equivalence
to traditional QC. Second, the manufacturer submits the validation data
to the FDA using processes already in place today. The FDA then reviews,
approves, or clears that proposed QC protocol if the validation data are
sufficient. Third, labs use the alternative QC protocol in place of the
CLIA-mandated QC requirements for that assay.
"The objective of Option 4 is to mitigate as many risks as possible through
a combination of fail-safe mechanisms and quality control—internal
and external, if needed—and reduce the risk to minimal or negligible
levels or at least reduce them as well as current traditional QC practices,"
Ochs said.
At the workshop, participants were asked to provide their input into
designing a QC program for the future. Most agreed the initial three EQC
options needed to be changed or eliminated, and that future QC should
be based on scientifically sound data. Many supported the exploration
of Option 4 and listed a number of new tools labs might be able to use
for QC. The CLSI is producing a monograph from the conference that lists
all of the participants’ suggestions.
Laboratorians will have a chance to be heard in the future as a CLSI
subcommittee begins work to determine what constitutes a "scientifically
rigorous" validation to support the Option 4 proposal. This definition
will be part of a consensus document describing data requirements for
alternative QC validation. The proposed title for the document is "Principles
of Manufacturers’ Validation of Risk Mitigation Using Quality Controls."
Another suggestion workshop attendees made was that unit-use devices
become their own separate category for QC purposes. Some felt it was wrong
to require labs to do "old school" QC on today’s innovative devices. "For
unit-use devices," says Dr. Sarewitz, "traditional external quality control
really isn’t applicable, and that was one of the major problems with the
equivalent quality control procedures as initially proposed. CMS didn’t
make a special provision for unit-use devices." Dr. Williams adds, "I
believe unit-use devices are safer to impose limited quality control on
than are the typical flow-through systems we use all of the time in the
laboratory."
QC as is for now
While CLIA regulators sort out the ideas expressed at the "QC for the
Future" workshop, labs can expect QC to remain as it is for now. Also,
because QC policy is unclear at the moment, Yost announced at the workshop
that surveys for the QC arena of CLIA ’88 will remain educational.
Don’t expect CAP checklists to change anytime soon either, Dr. Sarewitz
says. "We don’t need to make any changes to the checklist at this time,"
he says. "Because the situation is unsettled and we don’t know what exactly
is going to come out of CLSI and CMS and other discussions in the next
year or so, we don’t want to make changes because of something we think
might happen and then make other changes in six months if there’s something
new. So we’re going to leave the checklist as is and await development."
In the meantime, the CAP expects to have a voice in the development of
the Option 4 document and future CLSI conferences on this topic. (Dr.
Sarewitz was appointed advisor to the Subcommittee on Validation of Risk
Mitigation that will write the Option 4 document.) Dr. Hoeltge says the
next CLSI Leadership Conference is expected to address risk management,
and he believes the Option 4 project will be the consensus document upon
which the CMS will build much of its future foundation for QC.
"In fact, I think the future of EQC is riding on this CLSI document,"
he says.
Sue Parham is a writer in Edgewater, Md. |
|
|
