Shared Mental Models and Technical Assessment in Video Assisted Thoracoscopy Surgery Lobectomies

BACKGROUND The majority of errors in healthcare in general, and the operating theatre in particular, are caused by human factors1,2. Hindering of these errors can be accomplished by good non-technical skills (NTS), including good teamwork and efficient communication within the operating room (OR) team3,4. NTS are defined as 'the cognitive, social and personal resource skills that complement technical skills and contribute to safe and efficient task performance'5.

Previously, safety studies have analysed and measured adverse events as an indication of patient safety4,6. A new safety theory by Erik Hollnagel, named Safety II, argues that instead of only investigating adverse events as a 'measure' for safety, we should also be investigating what happens when these do not occur or when they are avoided as equally important7. This broader perspective on patient safety has also been advocated by others8-10.

In an explorative, interview-based study, we took a Safety-II perspective when examining which NTS were most important for ensuring safe and efficient video-assisted thoracoscopic surgery (VATS) lobectomy, a high risk minimally invasive surgery with complementary and overlapping scope of practice between the surgical and anaesthesia subteams. The multi-professional teams identified specific important NTS that served as the basis for shared mental models (SMMs) regarding the patient, the current situation or the resources within the team. These mental models were accomplished through communication and coordination, and enabled the team to anticipate problems and stay ahead of the situation11. These teams performing high-risk procedures emphasized the importance of the six identified NTS for them to be able to obtain an accurate SMM to enable them to be ahead which they viewed as most important for a successful and safe operation.

Some of the first to look at SMM as a requirement for effective team performance was Cannon-Bowers and Salas in the 1990s12. They defined SMMs as common or overlapping cognitive representations of task requirements, procedures, and role responsibilities. More recent, SMMs have been specified as a shared understanding of the task (including equipment), the situation and resources of other team members. In turn, this will enable them to understand and interpret the situation and what the other team members know, what they need to know and what they are doing12-14. Moreover, others have found identical or similar constructs to SMMs, i.e. team mental models/shared situation models/team knowledge/team situation awareness, and the importance of these constructs for patient safety is largely agreed upon15-21.

SMMs are more than the sum of individual mental models20,22. They need to be similar, accurate and shared within the team; however, what level of information sharing and team monitoring in the team will result in high team performance is less clear17,18,23.

In a recent methodology publication, different probe questions were used to establish team members' knowledge in order to measure the team's SMM and situation awareness13. The probe questions related to the task, the situation, and future predictions of the situation including possible outcomes. The teams in these experiments were at a second-line emergency centre where team members had distinct responsibilities. These teams differ from OR teams which work closely together in and with the same 'physical environment', namely in the OR with the patient, on a high-risk but at the same time rather standardized task, namely an operation, which in our case is an elective procedure of VATS lobectomy. One can speculate if the SMMs of OR teams are identical or differ to these emergency centre teams. So far, measurement of SMMs in OR teams has to our knowledge only been conducted in simulation settings17. To what extent this shared mental modelling occurs within the OR teams in the clinical setting is not known.

Moreover, the correlation between OR teams' NTS and technical skills (TS) has previously been investigated, however, there is still no clear picture of whether or not a correlation exists4,24-26. Additionally, some studies found a positive correlation27-29, whereas others found a negative correlation30. As the SMMs of the VATS teams seem to be enabled by the NTS perceived as important, it could be hypothesised that this relationship for SMMs and TS in the clinical setting is clearer than for NTS and TS. Yet, it is unknown whether the OR teams' SMMs are correlated to TS or other patient- and procedure-related measures.

The aims of this study are twofold. First, to explore to what extent SMM occur within the VATS teams in a cardiothoracic surgery setting. Second, to establish whether the VATS team's SMM is associated with the surgeons' TS, procedural specific times, per-operative bleeding, or patient characteristics (age, gender, FEV %, comorbidity, tumour size or location).

Research questions for the study

1. To what extent are the individual team members' mental models in agreement within the VATS team?

   1. Pre-operatively?
   2. Post-operatively?
   3. Pre- and post-operative correlations?

2. How is the team's shared mental model associated with

   1. The surgeon's TS assessed by VATS lobectomy assessment tool (VATSAT)?
   2. Procedural specific times (see later for description)?
   3. Amount of bleeding?
   4. Patient characteristics?

METHODS Design This study will be a case series study of VATS lobectomies purposefully sampled at all four cardiothoracic surgery centres in Denmark. Data will be gathered from 60 lobectomies performed by multi-professional VATS teams including both experienced and intermediate level surgeons. Gathering of data is planned to start in December 2016.

Data sampling Team members' mental models will be explored through two questionnaires handed out to all team members before and after the operation (see Appendix to protocol (in Danish)). The questionnaires were constructed using the probe questions from Sætrevik et al., adapted to fit VATS lobectomy settings11,13,14.

Specifically, the pre-operative questions concern team members' perceived technical skills, teamwork skills and familiarity; risk assessment ('global' and 'global in view of the current team'); and risk assessment of specific tasks. The post-operative questions concern team members' perceived technical skills and teamwork skills; risk assessment ('global' and 'global in view of the current team'), leadership and teamwork during the procedure, and knowledge of challenges experienced during the procedure.

Perceived technical and teamwork skills, familiarity, 'global' and 'global in view of the current team' risk assessments are rated on a Likert scale from 1-7, and risk assessment of specific possible challenging situations are rated on a Likert scale from 1-3. The remaining items are either yes/no-questions or open-ended questions.

The questionnaires will be handed out to each team member individually before and after the VATS lobectomy. Endoscopic video-recordings of all the VATS lobectomies will be collected from the thoracoscope using the recording device MedCapture. Using these video-recordings, the TS level of the VATS surgeon will be evaluated by two independent raters using a VATSAT assessment tool31. The VATSAT tool evaluates eight items on a five point Likert scale and produces a pass/fail score.

An observer will be present throughout all the operations to collect data on amount of bleeding (in ml) and procedural specific times. The procedural specific times will be both pre-specified times and a continuous registration of time for tasks. The pre-specified times will be from the time the patient enters the OR to anaesthesia begins, to the surgeon makes the first incision, to the surgeon finishes the last suture, to the patient wakes up from anaesthesia, and to the patient leaves the OR.

The observer will in addition take field notes for qualitative analysis with special focus on the six NTS identified in our previous study as important to safe and successful VATS lobectomies.

Ethics The Regional Ethics Committee of the Capital Region of Denmark has been applied, and a letter of exemption has been given according to Danish Law (H-16041772). The participants will receive oral and written information and will give informed consent on the day of the operation with the continuous option of retraction of consent and participation. The endoscopic recordings will not start until the thoracoscope is inside the patient. All video-recordings, physical examples of questionnaires and master sheets will be stored according to Danish Data Protection Law, and any written description in the final manuscript and article will be presented anonymously. Only the members of the research team will review the endoscopic video-recordings.

Statistical analyses Questionnaire data The preoperative questionnaire will have in all 24 items concerning risk assessment and team resources. The post-operative questionnaire contains 16 items. It will describe the risk, skills and teamwork as perceived by each team member as it unfolded during the procedure. For each item, percentage of answers 'I don't know' will be calculated. For the remaining answers of each specific item, Cronbach's α will be calculated thereby representing the agreement within the team. These two numbers (percentage however revised to a number between 0-1) will be multiplied, thereby giving a number between 0-1. This will be done for all items. For each questionnaire, all item scores can be added to give a total score, between 0-24 or 0-16. For the pre-operative questionnaire, 24 will represent total agreement within the team before the operation, indicating complete overlap in the team's SMM of team resources and risk assessment. For the post-operative questionnaire, 16 will represent total agreement within the team after the operation, indicating complete overlap in the team's view on team resources, teamwork and risk assessment.

The post-operative questionnaire will conclude with an open-ended question concerning challenges faced during the procedure by all the different team members. From these it will be possible to categorize each of these answers as either correct (given a score of 1) or wrong (including 'I don't know'; given a score of 0) when compared to the answer given by the person in question. An example of this could be if the scrub nurse answers that the surgeon did not have any challenges, and the surgeon himself answers that he did have challenges dissecting the lymph nodes, the scrub nurse will get a score of zero.

In all, this will result in the calculations of three scores for SMMs regarding team resources, risk assessment and awareness of challenges within the team.

Correlation calculations The items concerning team members' perceived technical skills, teamwork skills, and risk assessment ('global' and 'global in view of the current team') will be collected both before and after the procedure. This will enable analysis of how well each team member predict these items before the operation as compared to the assessment after. This will be done by calculating Pearson's correlation coefficient r for each team member and for the team.

The scores representing the total SMM in the team pre- and post-operatively can be correlated to the surgeon's technical skills (the VATSAT score), procedural specific times, bleeding and patient characteristics. Pearson's correlation coefficient r will be used if the data are normally distributed or Spearman's correlation coefficient (rho) if the data are non-parametrically distributed. Statistical analyses will be performed using IBM SPSS statistics 22; SPSS Inc., Chicago, IL, USA.

Order of authors Kirsten Gjeraa, Peter Dieckmann, Anna Sofie Mundt, Lene Spanager, René Horsleben Petersen, Henrik Jessen Hansen, Doris Østergaard, Lars Konge.