- A typical POCT program may have hundreds to thousands of non-laboratorian operators with wide ranges of education, training, responsibilities and understanding of medical conditions and the indications and implications of testing. In addition, there is constant turnover or addition of new operators.
- A challenge of any program is the initial training of these operators and the ongoing assessment of their testing competency.
- This is one of the responsibilities ideally overseen by a POCT Coordinator, in partnership with nursing and other clinical personnel, who can help to construct a meaningful competency assessment process, explain it to operators, motivate them to comply, and document those activities for regulatory compliance.
- The POCT Laboratory Director needs to provide critical leadership and support so that the competency program is effective and meaningful for the varied personnel performing POCT.
- A director’s guidance and support can also be seen as a patient safety effort by helping to ensure that testing is performed properly and operators understand potential errors.
- Competency assessment has been defined as the evaluation of an individual’s knowledge, skills, and correct practice of required work processes and procedures.
- In order to be effective, the assessment should include criteria for determining how the knowledge, skill, or practice being assessed is applied in the workplace.
- Competency assessment has application in many aspects of health care, including evaluating technical skills, critical thinking ability, and interpersonal skills of providers.
- Each of these aspects has an important influence on the quality of testing provided by both laboratory and non-laboratory professionals in the clinical setting.
- It is important to note that competency assessment is not achieved through merely ensuring an individual is educated properly or is qualified physically to perform a task.
- Competency is best demonstrated when an individual consistently performs assigned tasks accurately and in a safe and timely manner, as determined by comparison to a pre-established standard.
- An important aspect of quality systems in the laboratory environment includes the initial training and ongoing competency assessment of personnel.
- The quality system approach, one widely accepted framework, applies a core set of “quality system essentials” (QSEs) that are basic to any organization and to all operations in any healthcare service’s path of workflow”.1
- Training and competency assessment are two important QSEs.
- The quality system approach should also be used when developing training and competency assessment of non-laboratory personnel performing POCT procedures.
- While training primarily provides an individual with the knowledge, skills, and behavior to accurately perform tasks, competency assessment ensures that attributes acquired in training are maintained and performed on a consistent basis.
- Training is paramount in all aspects of the laboratory, including the pre-analytical, analytical, and post-analytical phases of testing.
- The need to train non-laboratory personnel performing laboratory procedures and entry-level laboratory professionals requires little justification.
- However, even an experienced laboratory professional who has previously demonstrated sound theory and basic skills in laboratory practice should engage in a formal training program when there is a significant change in work processes.
- Physicians and intermediate level providers performing PPM (provider performed microscopy) and PPT (provider performed testing, as defined by CAP) should ideally also have training.
- In summary, training is required in the following situations:
- New employees: provide formal training to learn specific work processes and procedures of the laboratory.
- All employees: when there are changes that affect work processes, such as new instrumentation or organizational changes (e.g., increasing the available test menu on the midnight shift).
- All employees: when training needs are identified (e.g., through knowledge errors detected during quality systems monitoring or when competency assessment processes indicate unsuccessful performance by an individual)1
- Table 1. Components of a Clinical Laboratory Training Program1 Adopted with permission from CLSI, name of document, and catalog number, Wayne, PA; 2009 (www.clsi.org)
Type of Employment
o Organization’s quality system
o Quality system responsibilities
o Quality manual
o Service’s path of workflow
o Quality control program
o Quality assurance program
o Occurrence management program for error detection and prevention
o Customer service program
o Good manufacturing practice (GMP): blood bank
o Proficiency testing
o Document management
o Main organization’s system (e.g., hospital information system [HIS] email, scheduling, word processing)
o Department’s system (e.g., laboratory information system [LIS])
o Middleware and device information systems
o PC applications (e.g., spreadsheets, database)
o Online documents
o Other computer applications used in the job (e.g., documentation of training, continuing education, competency assessment)
o Accident reporting
o Emergency preparedness
o Hazardous waste disposal
o Chemical hygiene program
o Infection control (Standard Precautions, use of personal protective equipment, bioterrorism, etc.)
o Radiation safety, where needed
|Work processes and procedures
o Processes in the path of workflow in which the employee works
o Procedures performed
o Medical necessity requirements
o Fraud and abuse reporting
o Health Insurance Portability and Accountability Act (HIPAA)
Pathologist as Technical Consultant:
Designing a Competency Assessment Program
- CLIA88 requirements for competency assessment involve “evaluating the competency of all testing personnel and assuring that staff maintain their competency to perform test procedures and report test results promptly, accurately and proficiently”.4
- Table 2 lists the six procedures for assessing competency as mandated by CLIA88.4 For waived testing, competency assessment may be satisfied for CMS with a sample of the six procedures across the entire waived testing program while currently non-waived testing requires all six elements to be assessed for each test system. An institution or other accrediting bodies may require all six elements to be assessed for any type of testing, so each POCT program should comply with its local requirements.
- Assessment and documentation must occur semiannually the first year an individual performs patient testing and at least annually thereafter. If test instrumentation, methodology, or procedural steps are altered, the individual’s performance must be assessed prior to reporting test results using the new instrument, method, or procedure.4
- Table 2: Methods for Assessing Competency1
- Direct observation of routine patient test performance, including patient preparation, if applicable, specimen handling, processing and testing
- Monitoring the recording of test results
- Review of intermediate test results or worksheets, quality control records, proficiency testing results, and preventive maintenance records
- Direct observation of performance of instrument maintenance and function checks
- Assessment of test performance through testing previously analyzed specimens, internal blind testing samples or external proficiency testing samples
- Assessment of problem solving skills
- Examples of competency assessment elements
- Direct observation of routine patient tests
- When used to assess competence, direct observation of routine patient test performance requires a designated evaluator to actually observe the individual performing the prescribed steps of the procedure.
- While this process may be time consuming, it can be very informative. It is usually documented using a skills checklist written specifically for the procedure.
- Care should be taken when selecting which procedures shall be monitored using direct observation.
- Procedures that require a great deal of critical thinking are good processes to assess using direct observation.4
- Monitoring recording and reporting
- This requirement deals with assessing an individual’s competence in recording and reporting patient test results.
- It may be accomplished in different ways.
- An evaluator may use direct observation of an individual documenting and reporting a patient test result.
- He/she may do a retrospective review using worksheets and computer printouts.
- Review of records
- This requirement includes the review of intermediate test results or worksheets, QC records, PT results, and preventive maintenance records.
- These tasks are already required of the laboratory supervisor (or designee) as an aspect of accreditation and should be incorporated into the laboratory’s quality system.
- Taking the time to carefully assess and document them annually is an efficient way to accomplish ongoing competency assessment.
- Many POCT devices have data management systems or middleware systems that may be used to automate the review of reported patient results and review of records.
- These provide an electronic means for reviewing the work of many testing operators in a relatively short period of time.
- Direct observation of instrument maintenance and function checks
- This requirement involves assessing individual performance of maintenance and function checks that are required on laboratory instrumentation.
- Direct observation assessments are best performed with the aid of a checklist. The manager or designee should develop the checklist using information from standard operating procedures, training materials, and instrument manuals.
- Care should be taken to make the checklists used in direct observation realistic and relevant to the testing process. The checklists should contain the key critical steps in performing the task and those in which errors are likely to occur.5
- Internal blind testing or external PT
- Assessing competence using previously analyzed patient samples, internally created blind testing samples, or external PT specimens is required by CLIA88 for all nonwaived tests.
- The manager should determine the best means of assessment for each task.
- Although time is required to make up internal blind testing specimens, this is an effective means of assessing competency.
- Rotating prior PT specimens among all staff that perform each analysis is another means of accomplishing this goal.
- It is important to note that residual PT material may be used as blind samples and repeated by staff as a competency check only after the submission deadline for the PT event.
Assessment of problem solving skills
- This is only feasible if the stability of the specimen will allow for repeated testing over time.
- Note: For laboratories that are part of an integrated healthcare system, it may be prudent to defer using residual PT specimens for competency assessment purposes until after the PT report has been generated.
- Problem solving skills are critical in the clinical laboratory; therefore, it is important that the laboratory competency assessment program evaluate them.
- One way this can be accomplished is through the use of written examinations where questions are designed around problems to be solved.
- Also effective is the case study approach where either real or fictitious problems are presented via a narrative format and individuals are asked to respond, either in writing or orally during a one-on-one session.
- Both written examinations and case studies may be integrated into an organization’s electronic learning system when available.
Another way of assessing problem solving skills includes review of the laboratory problem log, assessing each individual’s contribution to problem resolution as documented.
- Doing so simplifies access and record keeping.
- This is particularly important when dealing with high volumes of trained users (e.g., integrated delivery network with 4,000+ glucose meter testing operators).
When designing a competency assessment plan, there must be careful consideration to ensure that all six components required by CLIA88 are included for non-waived testing and also for waived testing if required (see above). The plan should also support the laboratory’s quality systems objectives.
- Alternatively, managers can require staff to log their own examples of problem solving situations and review them together during an assessment session.
When assessing large numbers of non-laboratory staff performing POC blood glucose testing, it is critical that an accurate assessment of the time and resources required to implement the plan are considered.
- This requires structure and documentation tools.
- One way of designing such a plan includes creating a table where the subject matter experts (e.g., supervisors) list the six required components and then systematically record and review the work process or analysis to determine which of the six is best used to assess the individual’s performance of that process or analysis.3
Table 3 below provides an example of a competency component review.
- If direct observation is chosen as the best means of ensuring the individual’s accuracy in testing, compliance with documentation requirements and problem solving skills, then time and resources must be budgeted so the plan is implemented consistently throughout the facility. Such plans require active collaboration with leadership from the various units.
- Remember , for all policies, procedures, and processes, it is important to:
- State, in writing, what you are going to do.
- Do what you said you would do.
- All steps in the work process must be considered, including the pre-analytical, analytical, and post-analytical aspects of test performance.
- Assessments should also be designed for each type of job category in the laboratory.
- The specific duties as outlined in the job description for each position type (e.g., nurse, laboratory assistant, medical laboratory technician, medical technologist) are a valuable reference tool for the supervisor when determining which competencies should be evaluated for each level of staff.
Table 3: Example of a Competency Component Review4
- Policy/procedure review
- Direct observation
- Unknown challenge
- Written exam
A and B
POCT Blood Gas Testing
A and D
Central Laboratory Gram Stain
- While no two competency assessment plans will look exactly alike, the process for developing a written plan includes steps that may be applicable to all laboratories. These steps are summarized as follows:3,4
- Define areas requiring competency assessment through analysis of tasks and skills. Include the time intervals for assessment.
- Identify which methods should be used to best assess the areas identified.
- Determine which individuals will perform the evaluation (e.g., laboratory managers, lead technologists, nursing educators).
- Develop documentation tools to accompany the written plan. It must be evident in the documentation what skills were assessed and how they were measured.
- Define the steps for corrective action or remediation when competency is not demonstrated.
- If the laboratory has oversight of POC tests, competency assessment of appropriate non-laboratory personnel performing CLIA defined laboratory procedures should be included in the competency assessment plan.
- What should be done when competency objectives are not met
- The most obvious answer to this question is to “retrain” the employee in the task or objective.
- However, it has been documented that over 80% of errors occur due to flawed or nonexistent work processes.
- These are system errors rather than knowledge errors and are not corrected by additional training.
- Before making the decision to retrain staff, it is critical for laboratory leaders to assess the documented processes and procedures for accuracy and effectiveness. The Clinical and Laboratory Standards Institute (CLSI) suggests using the following questions in the assessment:1
- Have all work processes been clearly documented?
- Are there written procedures for all activities in the work process?
- Are the written procedures clear and easy to understand?
- Does the service have a documented training program for all work processes and procedures?
- Has the employee been trained in the process or procedure in question and was the training documented?
- How was the effectiveness of the training determined?
- Is the employee the only person with this performance problem?
- Flawed, inadequate, or nonexistent work processes identified through this assessment must be addressed and steps to correct the problem(s) implemented.
- Seeking employee input in evaluating the effectiveness of work processes is advised.
- When work processes are substantially changed, retraining of the entire staff may be required.
- Despite the systematic requirements previously discussed, the model of competency assessment used may differ from organization to organization and from testing site to testing site within an organization.
- The model that is selected should be done in consideration of available organizational resources.
- For example, a decentralized model of competency assessment might include collaboration with nursing education to evaluate the competency of nursing staff performing bedside glucose testing as part of an annual nursing education day.
- A more centralized approach calls for central laboratory staff or the POCT Coordinator to perform direct observation of POCT operators in the patient care environment.
- In most cases, however, an integrated “train the trainer” model is effective as it allows for the collaboration of laboratory POCT Coordinators with clinical supervisors to evaluate all aspects of the POCT work processes.
- As a POCT Laboratory Director, the pathologist’s role is pivotal in ensuring a well executed competency assessment plan.
- His or her role is one of advisor, providing support to the POCT Coordinator attempting to implement the plan in daily operations.
- This support may mean helping to overcome institutional obstacles and challenges unique to POCT testing by seeking support and resources from the organization’s administrative and physician leaders.
- A well-written and consistently implemented competency assessment plan is a key component of the POC quality system.
- Whenever possible, electronic means for tracking and documenting competency assessment records should be used to assist in implementing the plan.
- Through the assessment process, POCT Coordinators and clinical supervisors, supported by POCT Laboratory Directors, not only ensure that individual employees perform critical tasks accurately, but they may also identify opportunities for process improvement within the testing site as a whole.
- When accurate and efficient work processes are performed by competent and confident individuals, patients are best served and errors can be avoided.
1.Clinical and Laboratory Standards Institute (CLSI). Training and Competence Assessment; Approved Guideline–Second Edition. Document GP21-A3. CLSI, Page 5, Wayne, PA, 2009.
2. Code of Federal Regulations. 1992. Clinical Laboratory Improvement Amendments of 1988, final rule. Fed. Regist. 42 CFR 493. Available at: http://www.cms.gov/Regulations-and-Guidance/Legislation/CLIA/downloads/apcsubm.pdf - 493.1351. Accessed April 30, 2008.
3. Ellinger P. Competency assessment: strategies to meet the ongoing challenge. Lab Med. 2006;37:176-178.
4. Sharp SE, Elder BL. Competency assessment in clinical microbiology laboratory. Clin Microbiol Rev. 2004;17:681-694.
5. Hudson J. Construction and delivery of an instructional unit. In: Principles of Clinical Laboratory Management: A Study Guide and Workbook. Upper Saddle River, NJ: Prentice Hall; 2004.
[Back to Top]