Four primary care teams and 1 rehabilitation team participated in this project. Participants were representative of the various disciplines involved in providing primary care (i.e. physicians, residents, nurses, pharmacists, social workers, physiotherapists, and dieticians).
A total of 35 observers participated and represented a range of primary care disciplines including physicians, nurses, dieticians, occupational therapists, pharmacists, and social workers. Observers all had previous experience working within interprofessional teams.
As predicted, this analysis revealed that the inter-rater reliability (i.e., the ability of raters to consistently discriminate between participants) was substantially greater than the inter-station reliability (i.e., the consistency of the discrimination when comparing one station to another). Internal consistency (an inter-item reliability) was very good for the observer checklist indicating that there is reasonable consistency in the scores assigned across questions.
Using the demographic variables to predict overall performance accounted for considerable variance, which appeared to be driven primarily by the years in the current practice team (i.e., the longer they were in that team the higher they scored on the TOSCE). This suggests that formal IPE training and general experience are less important to team function than is experience working within that particular team.
Many participants indicated that it would be feasible to incorporate the McMaster-Ottawa TOSCE tool into existing education sessions and team meetings to build IPC within the practice. Participants indicated they would likely just use 1 or 2 of the TOSCEs in a 1-hour meeting. Using the TOSCE tool to introduce new providers or trainees to the team was also suggested by several participants.
Most participants felt the 20-minutes allotted to each TOSCE station was sufficient to allow opportunity for all team members to participate in the discussion and to arrive at a patient-care plan. The only exception to this was when a simulated patient/patient role play was used. In these cases, scheduling 30 minutes for the TOSCE was preferable.
The majority of both team and observer participants found the McMaster-Ottawa TOSCE to be a useful tool for building IPC and felt it would be easy to implement within their practice. Key process lessons learned include: the importance of including a physician as part of the TOSCE exercise; the subjectivity of the observation rating scale and the need for clear observation guidelines; the need for, and value of, performance feedback discussion post-TOSCE; and the need to allow more than 20 minutes to carry out a TOSCE that involves a simulated patient.
"Each station was very well suited to primary care and allowed for opportunities for each member of the team to contribute."
Based on the findings of this project, modifications were made to the TOSCEs and the observation checklist to improve their use and uptake.
Impact of McMaster-Ottawa TOSCE: Team and observer participants found the patient cases presented in the TOSCE stations to be appropriate to a team-base approach and relevant to primary care. Participants indicated a number of areas in which participating in the TOSCE project has had an impact. Primarily, the McMaster-Ottawa TOSCE tool allowed participants to gain a better understanding of other team member’s roles in patient care. Many team and observer participants felt they would now be more likely to collaborate with other disciplines they may not have in the past on specific patient cases. One observer illustrated the value of the TOSCE tool by saying,
"It is a good learning opportunity for each profession to be there to understand more about each others roles, skills; any overlap/sharing of roles and for the value of team building, i.e., feeling like an equal, valued and respected member of the team."
Participants provided very few negative comments about the TOSCE tool or process. The main criticism received from a few people was that the TOSCE stations were not authentic. These people felt using actual patient cases from their practice would be more valuable.
Another interesting theme emerged from the observer data: ‘unexpected benefit of being an observer’. This goes beyond just enjoying the experience or finding the exercise interesting. Several observers discussed the direct impact participating in this project has had on their own practice. As one observer said,
"Observing the TOSCEs showed me that it is a great idea to use the TOSCE stations as a tool to show how you would really work together."
- Curran V, Casimiro, L, Banfield, V, et al. Research for Interprofessional Competency-Based Evaluation (RICE). J Interprofessional Care 2009;23(3):297-300.
Previous: Development Methods