- CAP Checklists include detailed standards for PT. The reader can use them as an additional resource.
- Proficiency testing (PT) is defined as determination of laboratory testing performance by means of inter-laboratory comparisons, in which a PT program/external quality assessment scheme periodically sends multiple specimens to members of a group of laboratories for analysis and/or identification; the program then compares each laboratory’s results with those of other laboratories in the group and/or with an assigned value. (adapted from Clinical and Laboratory Standards Institute Harmonized Terminology Database; available at http://www.clsi.org/).
- Alternative assessment is defined as determination of laboratory testing performance by means other than PT, for example, split-sample testing, testing by a different method, etc.
- Functions of PT:
- PT is a form of quality control (QC) in which a testing site receives specimens and submits results from those specimens to (usually) some outside body. It is also called external quality assessment. PT is a form of “peer based review” allowing regulators, manufacturers, professional organizations, and laboratories to compare the ability of individual laboratories to achieve accurate or reproducible results. Peer groups are compared by specific instruments, methods, or with “accuracy based” material (see below). Results can also be compared among similar and different testing systems to assess whether those systems produce accurate or comparable results.
- A more current form of proficiency testing is for all laboratories using any methodology for an analyte to compare their results to a single standard, or “accuracy based” target. A laboratory may appear to have a well controlled testing system, but when its results are compared to peers or a standard method, biases or inaccuracies may be revealed.
- PT is a rich source of information for a laboratory in helping to ensure that a test system performs as it was meant to, especially in comparison to other laboratories.
- There are standard protocols for troubleshooting errors or near misses and these provide a window on a total testing process, (e.g., clerical errors,) calibration errors, equipment preventive maintenance problems, bad reagent lots, etc. PT also helps manufacturers follow their test system performance.
- Although not officially promoted, many users compare PT performance of instruments or test systems when selecting testing options.
- Regulatory Requirements for PT:
- Laboratory participation: CLIA88 requires PT for all ‘regulated’ analytes, (an updated list may be found on the CAP website) but most accrediting bodies require laboratories to participate in PT for any test they run.
- There are a number of sources of PT material and programs and CLIA88 requires a laboratory to be enrolled with an approved program and have all regulated results sent to the Centers for Medicare and Medicaid Services (CMS) so that successes and failures can be monitored and acted upon if necessary. Other regulatory bodies, such as states and professional societies also require that results be reported to them for similar monitoring or intervention. There are established definitions of success versus failure per analyte. These generally include per cent versus absolute deviation from a target value and are derived from total error or historical peer performance decided by experts.
- If PT materials are not available for a test method, laboratories can construct “alternative” sources of PT, such as exchanging samples with another laboratory, retesting previously tested samples in a blinded fashion, comparison testing using a different methodology, etc. Alternative PT is generally required by other regulatory bodies or professional societies for any test the lab runs if an approved source of PT material is not available for that test.
- One of the criticisms of CLIA88 is that is does not require PT for waived testing. PT can be time consuming and expensive and one of the attractions of the waived designation in the United States is that PT is not mandated. However, various accrediting bodies and jurisdictions do require PT for waived tests and “best laboratory practices” require it. Again, this Tool Kit promotes best practices and this discussion encourages PT. Even prior to CLIA67, there was ample evidence that laboratories that participated in PT performed better over time and did better on inspections. Many laboratorians and regulators, especially with the 20th anniversary review of CLIA88, have cited the absence of required PT for waived testing as a real weakness and one of the aspects of CLIA88 that needs correction. Internationally, various bodies support the concept of PT for waived or POC testing.
- CAP considers POCT to be no different than testing within the central laboratory. CAP accreditation requires that PT be performed for all POCT tests in order to assure the quality of the test and the competency of the staff that performs the testing. A list of available POCT proficiency tests available from CAP can be found in the on-line CAP PT catalogue.
- A Laboratory Director or POCT program may promote or require PT. PT is ideally a great source of education and regulatory bodies and professional laboratory societies endorse it as such and promote its expansion. Many POCT sites regard PT as a window on their performance and a source of pride. With good leadership and education, POCT sites can see the value in participation in PT. Again, this Tool Kit encourages directors and coordinators to use it in their programs and educate users about its value. Understanding POC site issues and facilitating PT at those sites is critical.
- Successful participation: Laboratories must maintain a PT sample success rate of 80% for most analytes.
- There may be severe consequences for PT failures and a laboratory may lose the regulatory approval to test for certain analytes based on the type of failure or for persistent failures. Those consequences are outlined in the CLIA regulations. Laboratories may not be able to perform or bill for specific tests that they fail in PT.
- Handling of PT samples: Another important requirement of PT is that it is meant to test a laboratory’s routine function. Therefore, PT samples must be treated as routine patient samples without any special test system preparation:
- PT samples must be rotated among all test operators on all shifts. This requirement may pose a significant hurdle for POCT where there may be numerous sites and users that perform testing. However, in order to gain maximum benefit, PT should enable a laboratory to ensure that a test performed at any time by any staff member is reliable.
- Before testing PT samples, laboratories may not specially calibrate instruments, use fresh reagents, or take special efforts like running PT samples in duplicate if they would not take those actions for regular testing.
- Finally, a laboratory MUST NOT share PT samples or consult with another laboratory about a PT sample before results are officially submitted. If a laboratory consults with, compares its results with, or sends a PT sample to another laboratory there are severe consequences and fines that may result in the loss of accreditation or licensure for that laboratory. Laboratory and POC sites must avoid the appearance of consultation or infractions with PT submissions. Rather than submitting results by facsimile from a central site, it is advisable to submit results from the individual sites of performance. The Laboratory Director is ultimately individually responsible for ensuring compliance with PT requirements and s/he or his/her designee must sign an “attestation” statement to that effect with each PT submission.
- Potential issues with PT and POCT:
- Matrix effects (commutability): One of the difficulties in creating PT samples is securing enough stable material that can be used at multiple sites on multiple different instruments or with different test systems.
- This aspect, or the commutability, of the material is defined as the: “ability of the material to yield the same numerical relationships between results of measurements by a given set of measurement procedures, purporting to measure the same quantity, as those between the expectations of the relationships obtained when the same procedures are applied to other relevant types of material” (ISO 15197, ISO/DIS 17593).
- Providers of PT tests must make materials as close to human samples as possible so that they will act like human samples in test systems. However, preservatives, stabilizers, etc. are needed to manufacture a single material that can be used in different systems and which is stable during shipment and testing.
- The use of such non-physiologic materials may create a “matrix” effect that can alter test results. While a specific PT sample may work well in some instruments, it may give skewed results in others due to this effect.
- Laboratory Directors and coordinators need to be aware of this issue.
- Manufacturers are aware of matrix effects when they evaluate their equipment performance in large PT programs. They can be a valuable source of assistance for sites investigating PT testing problems.
- Recently there has been a movement to adopt “accuracy based” PT derived from human samples to better reflect the nature of actual testing. The targets for these accuracy based materials are set based on standards established by the CDC or other national and international programs, all working to standardize testing results in laboratories around the world.
- Determining test usability:
- PT can provide a great deal of information regarding the accuracy of a given POCT test method compared to other methods for measuring the same analyte.
- It can also provide a useful assessment of the competency of those performing POCT. It cannot, however, determine if a POCT test is usable in a given clinical situation.
- The usability of a test is dependent on how the results of the test will be used in managing patient care.
Handling PT samples in the POCT environment:
- For example a POCT test for glucose may be judged to be acceptable based on PT data for home care, clinic or emergency department use but may not be usable for tight glycemic control in the post-surgical or intensive care environment, where higher accuracy requirements are needed and more sources for error exist.
[Back to Top]
- Various PT materials have different storage requirements (room temperature, refrigeration, etc.) and may require reconstitution and/or thorough mixing prior to testing.
- These pre-analytic aspects of PT may pose significant challenges in the POC environment where there are typically a large number of users who are not familiar with such handling requirements.
- Additionally, PT materials once prepared, may be unstable and delays in testing may produce unacceptable results.
- Many POCT PT failures can be traced to the user not faithfully following the instructions supplied with the PT materials for storage, preparation and timely testing.