Monitoring and Evaluation: Selecting Methods and Tools
A wide variety of methods and tools exist that could be used to monitor and evaluate a clinical mentoring program. A trainee's acquisition of knowledge and skills could be assessed through self-report or through a written knowledge/skills test. Motivation and intent to apply knowledge and skills acquired in training could be measured through participant self-assessment, action plans, or interviews. Health care worker delivery of clinical care could be measured via direct observation, supervisor reports, or patient records. The development of skilled clinical mentors might be tracked via numbers of local clinicians providing mentorship services, or via performance scores on a clinical mentoring checklist. The development of host sites for clinical training opportunities could be measured via routine reporting on preceptor trainings hosted or written evaluations of practica by practicum participants.
Most of the methods and tools presented in this toolkit were not designed for basic research purposes. Rather, they were designed to provide “good enough” information about how clinical mentoring activities are being conducted and whether they are having any effect. Successful program monitoring and evaluation activities are those which generate data good enough to use to adjust your mentoring activities to better meet your program objectives.
The monitoring and evaluation plan of your clinical mentoring activities should be informed by your project goals and objectives and by the feasibility of various methods given the context and parameters of your clinical mentoring project.
The first criteria in selecting a method is that it actually measures what it is you want to know (validity). Knowing the number of clinical mentoring sessions your organization has conducted does not tell you anything about what was gained from those sessions. Increasingly high test scores after clinical mentoring sessions indicate that mentoring activities have been successful in increasing provider knowledge and skills. However, this tells us very little about the type of care they are actually providing.
An equally important consideration in selecting a method is that it is acceptable to the mentor, trainee, patients, hospital administrators and other affected staff. Different evaluation methods will be acceptable in different contexts. If clinical mentoring activities form a component of a credentialing program, fellowship program, supportive supervision initiative, or take place in a situation in which the mentor has clear authority and responsibility to evaluate provider competency, then evaluation methods may be more detailed and extensive than if the mentor is a "guest" in the trainee's home facility, and has no clear mandate to certify or supervise the trainee. Most people react negatively to the idea of being evaluated, and so it is essential that the mentor negotiate the method and purposes of the evaluation process with the trainee so that the evaluation method supports, rather than interferes with the work of clinical training.
The third criteria is that your program actually has the capacity to implement the method. It is often not possible to choose the most rigorous method imaginable to evaluate programs because of a variety of barriers and resource constraints. For example, when mentors are present as a trainee cares for his or her patients, this provides a wonderful opportunity for them to observe and note how a trainee delivers services. This is important information gauging whether the mentoring is having any impact and to identify areas for future training. However, this level of data collection is not always feasible given programmatic limitations. Similarly, seeing whether clinical mentoring activities have an impact on patient outcomes would be incredibly powerful data to collect; however, this is especially challenging in low resource settings where patient data collection is usually not very reliable.
- Instructional Evaluation Methods
- File Type:
- Word Document
- Pages:
- 2
- Summative vs Formative Evaluation
- File Type:
- Word Document
- Pages:
- 4
This document presents a widely accepted framework of five general categories for training evaluation. The categories include participant reaction, learning, application of learning in the work setting, improvements in facility systems or patient outcomes, and training cost effectiveness. The framework can help you think about the different kinds of questions you would like to answer with your evaluation activities.
This document summarises the kinds of evaluation questions trainers and program managers can ask before, during, and after their training activities. It is especially written to assist in the design and management of classroom–based trainings but many of the main ideas can be adapted to the clinical mentoring context.