Objective The objective of the current study was to evaluate the utility and ease of use of a novel non-invasive tablet-based application for quantification of the pivot shift (QPS) in a multicentre setting.
Methods Nine senior orthopaedic surgeons were recruited from academic medical centres and enrolled a total of 90 subjects. All surgeons received video and electronic training on how to perform QPS measurement using a novel tablet-based visual tracking system. Skin markers were used for visual tracking. Each orthopaedic surgeon performed a pivot shift test and obtained a QPS measurement for 10 subjects with known anterior cruciate ligament (ACL) injuries. Anterior translation of the lateral tibial compartment during the reduction event (in millimetre) was recorded. After all subjects were evaluated, the surgeons completed a survey about the ease of use, utility and willingness to use the mobile-based application in clinical practice. All automated visual tracking used to calculate anterior translation of the lateral tibial compartment was manually evaluated to ensure accurate tracking of skin markers. Statistical comparison was made between groups using Student’s t-test with significance defined as p<0.05.
Results The results showed that there is adequate tracking of skin markers to provide a QPS measurement across multiple academic medical centres and surgeons in only 70% of subjects. In 27 of 90 subjects, there were tracking errors which resulted in an invalid QPS measurement. Tracking errors were due to background noise, including objects in the room which were the same colour as the skin markers or use of skin preparation solution similar in colour to the skin markers. The QPS measurement was 3.2±1.8 mm, 2.4±1.3 mm in isolated ACL deficiency, and 3.2±2.0 mm with combined ACL and meniscus injury (not significant). There was significant variation among the QPS measurements of different surgeons (p=0.008).
Conclusion This study showed that mobile-based QPS measurement software can be used intraoperatively to allow surgeons greater objective feedback regarding the rotatory laxity of the knee prior to and following ACL reconstruction. Software upgrades with improved real-time feedback could help prevent errors in measurement and improve utility of QPS for assessment of injury severity in subjects.
Level of evidence Level III.
- anterior cruciate ligament
- diagnosis method
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, an indication of whether changes were made, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
What are the new findings?
Tablet-based software can be used by multiple surgeons for real-time quantification of the pivot shift.
In deployment of mobile applications, hands-on training may be necessary with the staff using the software in addition to the primary surgeon.
Real-time user feedback is necessary to improve accuracy of data acquisition.
Evaluation of knee stability in the setting of anterior cruciate ligament (ACL) deficiency includes assessing anterior tibial translation (anterior drawer test and Lachman test), as well as rotatory knee laxity (pivot shift test). These physical examinations currently depend on subjective assessment of the stability of the knee. The pivot shift test reproduces the subluxation event that occurs at the time of ACL rupture.1 A positive test occurs with rapid reduction of an anteriorly subluxated lateral tibial plateau as the knee is brought from extension into flexion with an internal rotation and valgus stress is applied.2 3 The pivot shift test is most commonly graded on a subjective scale: grade 1 (glide), grade 2 (clunk) and grade 3 (gross).4 5
Rotatory stability of the knee is commonly assessed via the pivot shift test prior to ACL reconstruction, during ACL reconstruction and in following patients longitudinally to assess that no residual rotatory laxity exists after reconstruction.6 The pivot shift test has greater specificity and higher negative predictive value than the anterior drawer or the Lachman test for diagnosing ACL insufficiency.7–9 Objective methods have been developed to standardise and quantify anterior tibial translation via the use of arthrometers, which have been accepted as the objective standard in the clinical research setting,10 11 but there are currently no widely used methods for quantification of rotatory laxity during the pivot shift test.
Initial quantification of the reduction during the pivot shift test was proposed using an invasive electromagnetic system. Subsequently, quantification of the pivot shift (QPS) was also attempted using invasive navigation-assisted surgical systems.12–14 Non-invasive methods for the QPS test have been developed, which include simple image analysis, accelerometer-based devices strapped to the tibia15 16 and electromagnetic trackers.17 18 These methods for quantifying the pivot shift are limited by motion artefact and time-intensive post hoc analysis.19 Recently, a tablet-based application was developed which could quantify the anterior tibial translation seen during the pivot shift test in real-time and provide healthcare providers with instantaneous measurements in the clinical setting without cumbersome setup.20 21 Previously, this tablet-based software was validated in a multicentre study which showed a correlation between the QPS measurement and generalised ligamentous laxity in 103 patients,10 but usability of the application in the setting of ACL deficiency with orthopaedic surgeons is unknown. Additionally, the correlation between the subjective pivot shift and the QPS is not clearly defined.
The objective of the current study was to determine the ease of use of a novel tablet-based application for the QPS test. We hypothesised that the application could be used by multiple surgeons with adequate training to quantify the anterior translation of the lateral tibial compartment during the pivot shift examination in all tested subjects.
Nine sports fellowship-trained, senior orthopaedic surgeons with 9–25 years of experience across eight academic centres within the USA and Canada were enrolled after local institutional review board approval at each site. Each orthopaedic surgeon was trained in the usage of the application through educational videos and a step-by-step instructional guide which accompanied the software (iPad fifth generation, Model A1822, iOS V.9.1; Apple, Cupertino, California, USA). Each orthopaedic surgeon recruited 10 subjects undergoing ACL reconstruction at their centre. QPS was obtained using the tablet-based software during standard examination under anaesthesia prior to ACL reconstruction. Subjective pivot shift grade, as determined by the senior orthopaedic surgeon, was recorded for each examination. Intraoperatively, surgeons recorded additional injury factors for each subject: location of ACL bundle injury, meniscal pathology, and medial or lateral collateral ligament injury (including grade of injury). All data were entered into a central database by the surgeon or support staff from each study site after the subjects were enrolled.
For QPS test, skin markers were attached to bony landmarks on the lateral side of the knee at three locations: Gerdy’s tubercle, the fibular head and the lateral epicondyle (figure 1A). The distance between Gerdy’s tubercle and the fibular head was measured, input into the application and used to calibrate the measurement of the QPS. A video was then recorded of a singular pivot shift test being performed (figure 1B). Anterior translation of the lateral tibial compartment was defined as movement of the relative movement of the skin marker on the lateral epicondyle along the line perpendicular to the line formed between Gerdy’s tubercle and the fibular head as previously described.21 The magnitude of the tibial reduction which occurred during the pivot shift test is calculated in real time with graphical and numerical feedback (figure 1B). Post hoc, all videos were individually inspected to determine if the visual tracking correctly recognised the skin stickers placed on the anatomical landmarks. Resultant videos which did not show tracking of the dots on the skin or which recorded anterior translation of the lateral tibial compartment during the reduction event over 10 mm (supraphysiological) were excluded and considered ‘invalid’, and the reason for erroneous tracking was recorded as either objects in the room (background noise), multiple pivot shift tests being performed during the same recording or inability of the software to determine the anterior translation of the lateral tibial compartment during the pivot shift test.
Following assessment of the 10 subjects with the tablet-based software, each orthopaedic surgeon completed a questionnaire assessing the usability of the tablet-based application in the surgical setting, ease of use of the application and perceived willingness of other orthopaedic surgeons to use the application.
Statistical comparisons were made between groups based on injury pattern or subjective grade of injury using two-tailed Student’s t-test after data normality (D’Agastino-Pearson) confirmed with significance defined as p<0.05. Kruskal-Wallis non-parametric comparisons were made between surgeons for QPS (Prism V.8.0.1, GraphPad, San Diego, California, USA). Data are presented as mean±SD if normally distributed or mean (range) if not normally distributed.
Subject and injury characteristics
Of the 90 subjects, the mean age was 26.9±10.4 years (range 11–55 years). Most subjects had a subjective pivot shift grade of 2 (64.4%), with a smaller number having grade 1 (22.2%) and grade 3 (13.3%) prior to ACL reconstruction. Complete rupture of the anteromedial and posterolateral bundles was seen in 80.0% of subjects, with 15.6% having a partial rupture or stretching injury of one bundle. Four subjects (4.4%) had an intact anteromedial bundle, but no subjects had an intact posterolateral bundle. Injury to the medial or lateral meniscus was visualised in 24.4% and 35.6% of subjects, respectively. Partial or complete disruption (grade 2 or 3 ligament injury) of the lateral collateral ligament and medial collateral ligament was seen on MRI in 10.0% and 5.6% of subjects, respectively.
Intraoperative real-time visual tracking
All videos used for visual tracking and QPS test were inspected for tracking fidelity and visualisation of the anterior translation of the lateral tibial compartment during pivot shift. Sixty-three of 90 subjects showed proper visual tracking and demonstrated a pivot shift reduction. Twenty-seven (30%) of all subjects were excluded due to errors in visual tracking (n=20) or lack of reduction on the pivot shift plot (n=7) (figure 2). Visual tracking errors occurred due to objects in the background with the same colour as the skin markers (figure 2A), usage of antiseptic preparation on the subject’s skin with colour similar to the skin markers (figure 2B) or surgeons performing multiple pivot shift tests during the same video.
Quantitative pivot shift measurements
Anterior translation of the lateral tibial compartment during the pivot shift test was 3.2±1.8 mm for all subjects (figure 3). There was a significant difference between surgeons’ measurements (p=0.008, Kruskal-Wallis). The number of excluded subjects per user varied from zero (surgeons 6 and 9) to seven (surgeon 7). Individual surgeon measurements ranged from 4.9 mm (2.0–6.9 mm) to 1.9 mm (0.6–2.1 mm).
There was no significant difference between QPS in subjects with a partial anteromedial or posterolateral bundle tear (3.4±2.1 mm) and subjects with complete ACL rupture (3.6±1.6 mm, p>0.05). Between subjects with and without a meniscal tear, no significant difference was found (3.2±2.0 mm and 2.4±1.3 mm, respectively; p>0.05). Subjective grade 1 pivot shift resulted in 2.5±1.1 mm QPS, whereas grade 2 or 3 pivot shift resulted in 3.3±1.9 mm QPS. There was no significant difference in QPS between grade 1 pivot shift and grade 2/3 pivot shift (p>0.05).
All surgeons completed the usability questionnaire. Users responded that the software was quick, convenient and user-friendly (table 1). No surgeons had difficulty with placement of skin markers on predefined anatomical locations nor did any note in the survey difficulty identifying the reduction on the screen plot. Most surgeons (63%) had technical difficulties with the software and would also suggest changes to the current software. The majority of surgeons noted a setup time of 5–10 min for the first subject, but all surgeons were able to reduce the setup time to either 0–2 min (40%) or 3–5 min (60%) by the 10th subject. Four surgeons had four or more patients excluded for incorrect visual tracking, and of these surgeons, 3 of 4 (75%) stated that they had technical difficulties with the software and additionally responded that they would not use the software in their clinic.
The most important finding of the current study is that a tablet-based application can be used in the intraoperative setting to provide real-time measurement of rotatory knee laxity, but that lack of adequate support and training led to a large number of visual tracking errors. Thirty per cent of all pivot shift examinations were unsuccessful due to software or user errors.
QPS measurements varied widely between surgeons due to differences in technique and ability to use the software. QPS measurements for subjects with ACL deficiency and concomitant meniscal injury were similar to those found in prior studies.22–24 While previous studies found a significant difference in QPS between subjective grade 1 and subjective grade 2/3 pivot shifts, this study was not powered for determining if differences in QPS existed between subjective pivot shift grades, and no significant differences were found.
Of the 90 tests performed, only 63 showed correct usage of the application, with common errors being poor visual tracking due to objects in the operative field with colour similar to the tracking markers or performance of multiple pivot shift tests during one video. Despite training surgeons to use solid backgrounds to improve visual tracking, clinical support staff were untrained, and objects within the operative suite provided background noise in many cases, invalidating data without the surgeon’s knowledge for 20 (22%) subjects. Three areas for improvement in the application were identified as a result of this small-scale deployment: built-in alerts when objects in the field will interfere with visual tracking, the need for in-person support during initial deployment of the application at each site, and real-time identification of anterior translation of the lateral tibial compartment which fall outside of the physiological range and should be repeated.
In-person deployment training of staff
All surgeons were instructed to perform pivot shift tests and visual tracking with a blank background. Sixty-three of 90 videos showed correct usage of a solid-coloured background to improve real-time visual tracking of the skin markers. For all subjects, the senior surgeon performed the pivot shift test in a sterile field. Therefore, the person using the tablet-based application was not the senior surgeon trained in using the application but ancillary staff members who were not part of the sterile field. In-person support during initial deployment of the application at each site would have allowed for staff training and the ability to train the personnel responsible for using the application. Interestingly, seven patients had no visible tibial reduction on the visual tracking plot, but no surgeons claimed in the usability questionnaire that there was difficulty identifying the reduction plot. In-person training may help support staff to identify the reduction on the tracking plot. In person deployment staff may also provide real-time feedback to surgeons to standardise performance of the pivot shift manoeuvre and improve accuracy.25
Alert user to background interference during visual tracking
An inherent shortcoming to real-time visual tracking is the automated detection of the object being tracked, which is subject to interference from background colours. Twenty of 90 videos had objects in the operative room or on the surgical field which were similar in colour to the tracking markers provided to each surgeon. Preliminary testing had been performed under ideal conditions with solid-coloured background, which did not represent how the application was used by orthopaedic surgeons in real practice.
Alert user to pivot shift outside of physiological range
Automated detection of anterior translation of the lateral tibial compartment during the pivot shift test was able to correctly detect the reduction manoeuvre under ideal conditions. Real-world users of the application were operating room staff members who have not been trained on the use of the application and did not recognise when the software was detecting points which did not represent the reduction. In some cases, the pivot shift manoeuvre was not performed correctly, resulting in ‘no reduction’ of the tibia during knee flexion. Often, the measurements of anterior translation of the lateral tibial compartment were significantly outside of the physiological ranges determined during preliminary testing of the application. Real-time QPS allows for implementation of immediate feedback to users not possible with prior time-intensive post hoc analyses.15
Initial testing among users familiar with the application did not reveal these shortcomings, and deployment to academic centres and surgeons unfamiliar with the intricacies of the software brought forth user issues which need to be addressed prior to widespread distribution. Further studies are needed to evaluate whether implementing the suggested measured will improve the usability of real-time non-invasive quantitative pivot shift in the intraoperative setting.
Limitations of the current study include that this real-time QPS was only performed within the controlled environment of the operating room. There may be additional challenges for use of real-time quantification of pivot shift using the tablet-based software in this study in clinical offices or other practice settings. An additional limitation of the current usability study is that surgeons were not instructed to use a standardised pivot shift manoeuvre, and previously, it has been shown that techniques vary widely between surgeons and standardisation improve consistency.25 Technical improvements, including improved image processing, real-time feedback on the reduction achieved during a pivot shift and the ability to edit the automatically detected tibial reduction on the plot, may increase accuracy. Additionally, many reduction plots did not show a reduction of the skin markers, which may have been due to lack of skin marker motion despite bone movement due to overlying soft tissue. This technology may be limited by body habitus, but this has not been explored previously, and the data from this study are unable to determine if patients with increased soft tissue overlying their tibias may reduce the accuracy of skin markers for quantifying pivot shift. Further studies are needed to determine what a physiologically acceptable range for skin marker reduction is based on body habitus and other injury factors.
This study demonstrated that tablet-based software for QPS test can be used in the intraoperative setting by multiple surgeons. When following the user instructions properly during training, anterior translation during the pivot shift test was comparable to prior studies. However, clinical deployment of this software resulted in a large number of skin marker acquisition errors which could be resolved with hands-on training during initial deployment, availability of clinical support staff and improved software feedback to prevent errors in measurement.
Collaborators Bryson Lesniak, MD; Olufemi Ayeni, MD; James Voos, MD; Asheesh Bedi, MD; John Xerogeanes, MD; Stephen Brockmeier, MD; Ronald Navarro, MD; Gregory Maletis, MD; Seth Sherman, MD.
Contributors RTT, CIM and NKP were involved in the design of the study, data collection, statistical analysis, drafting and review of the manuscript. VM and RD were involved in the design of the study, data collection, drafting and editing of the manuscript. All authors reviewed and approved the manuscript before submission.
Funding We gratefully acknowledge the support of the Wallace H. Coulter Translations Research Foundation, the University of Pittsburgh Clinical and Translational Science Institute (funded in part by National Institutes of Health Clinical and Translational Science Award UL1 TR001857) and the Pittsburgh Innovation Challenge.
Competing interests The authors have the following conflicts of interest: RT, Johnson & Johnson education grant; NKP and CIM, no conflicts of interest; RD and VM, patent holders in pivot shift software; VM, Smith & Nephew consultant, Knee Surgery, Sports Traumatology, Arthroscopy journal Deputy Editor in Chief and International Society of Arthroscopy, Knee Surgery and Orthopaedic Sports Medicine board member.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available upon reasonable request. Deidentified data requests can be sent to the corresponding author and data may be available upon receipt of appropriate request and approval by the local institutional review board.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.