Which patient reported outcomes to collect after anterior cruciate ligament reconstruction

Introduction

In recent years, increasing emphasis has been placed on collecting patient reported outcomes (PROs) in the field of sports medicine in order to assess functional recovery following injury from the perspective of the patient (1-3). In the context of anterior cruciate ligament (ACL) injury, several different PRO measures have been developed and validated for use. Practitioners should be aware of the strengths and limitations of each of these PRO measures in order to determine which score is most relevant and practical to implement into daily clinical practice.

Ideally, an outcome measure for patients with ACL injury would measure post-operative pain, mobility, quality of life, and return to functional activity (4,5). These outcome measures must be validated and responsive, with the ability to detect meaningful, clinically relevant change in patients with different functional capacities (6,7). A balance must be met, however, with survey comprehensiveness and ease of administration; surveys that ask an abundance of questions may be impractical or difficult to administer to patients in a routine fashion.

The goal of this review is to provide an overview of the numerous different available PRO measures for patients with ACL injury. Currently, there is no “gold standard” with regards to which specific measure is “best” to use in these high-demand patients. Unfortunately, this lack of consensus creates inefficiencies when attempting to compare results of different clinical studies that use disparate scores. In the future, consolidation of outcome reporting may be necessary.

Objective outcome measures

Objective measures, which quantitatively assess aspects of recovery independent of patient input, are often used to supplement PROs and can even contribute to final scores, such as in the case of the International Knee Documentation Committee Forms (IKDC) or the Cincinnati Knee Rating System (CKRS) (8,9). The most common objective scores include tests that quantify strength (using a dynamometer), range of motion (using a goniometer), stability (using a knee translation device, such as the KT-1000), and pain medication requirement diaries (10-12).

The primary advantage in measuring objective outcomes is the consistency and reliability of their measurements; that is, the impact of human error or subjective interpretation is largely minimized. Unfortunately, these measures do not incorporate patient perspective and return to functional capabilities. For example, two patients with the same objective knee measurements post-operatively could have different abilities to return to work or athletic participation. While both may “score” similarly according to objective metrics, the patient who cannot regain functional capacity will still be considered a “failed outcome” compared to the patient who returns successfully. This shortcoming forms the compelling foundation for incorporation of PROs into clinical outcomes reporting in patients with ACL injury.

Subjective outcome measures

The use of the visual analog scale (VAS) and single assessment numeric evaluation (SANE) scores can provide a quick and versatile manner in which to assess broad subjective outcomes. With these assessment tools, patients can quickly mark pain levels, functional status, or even satisfaction. The VAS is widely used in research and clinical settings and can be found as a component in numerous outcome measures, or stand as a PRO measure on its own (13). With verbal or pictorial descriptions of the extremes, patients indicate their status on a line of uniform distance. Often this is used for pain evaluation, but this can unfortunately limit a multidimensional subjective feeling of pain to one dimension (e.g., magnitude). Another limitation becomes apparent when quantifying VAS scores, as it is important for all patients to receive a VAS that is uniform in length, which is hard to maintain digitally, or after numerous photocopies of paper versions as well (14).

Similarly, SANE scores are also versatile and brief in nature. Usually on a scale of 0 to 100, the patient can report overall sentiments about function and satisfaction. Shelbourne and colleagues have shown that SANE scores have moderate to strong positive correlations with longer PRO surveys such as the IKDC and CKRS following ACL reconstruction and knee arthroscopy. Thus, in limited fashion, SANE scores may provide a quick alternative to lengthy questionnaires (15). Although caution should be exercised as a SANE score may lack the ability to specify what portion of knee function impacts the overall rating (15). Overall, the VAS and SANE have valuable utility for assessing satisfaction in addition to other aspects of the patient experience. These tools can also gauge satisfaction with regards to care and outcomes, as well as whether the patient would undergo the surgery again and/or recommend the treatment to another patient.

PRO measures

PRO measures are traditionally classified into generic health questionnaires, anatomically-specific or disease-specific (6). The organization of this review mirrors these categorizations.

Generic health questionnaires and other quality of life assessment measures

The most common general health assessments used to report on patients recovering from ACL reconstruction include: The Medical Outcomes Study (MOS) derived Short Form-36 and Short Form-12 (SF-36, SF-12), and European Quality of Life (EQ-5D) (Table 1). It is useful to measure these scores pre- and post-operatively, as patients with higher pre-operative general health scores have been shown to be more satisfied and more likely to return to pre-injury physical activity following ACL reconstruction (31,32). These questionnaires can also form the foundation for cost-utility analysis scores used in cost-effectiveness research. Some studies do this using the SF-6D, a six-dimensional measure derived from SF-36 responses (23,24). One significant weakness in using general health assessments in patients with ACL injury is that the forms may not be sensitive or specific enough to appropriately capture this high-demand patient population, which is often younger and more active than the patients used to establish generic health assessments (33,34). Despite this, quality of life is still important to monitor as it demonstrates the global impact of the injury to the patient.

Table 1 Overview of generic health and quality of life PROMs
Full table

Short form-36 and -12

The SF-36 is one of the most widely used and established generic health surveys and has been validated and psychometrically analyzed extensively in various disease states (16-19,35), including orthopedics (36-38). It contains eight subscales that comprise two summary scores for a total of 36 questions (Table 1). Numerous studies have used it to assess quality of life and cost-effectiveness following ACL reconstruction (39-42). Average time to completion has been reported to be around 5 to 10 minutes (34).

Early studies of the SF-36 have demonstrated strong correlation with traditional ACL PRO scores, such as the IKDC and the Lysholm Knee Score (43). However recent reports have indicated that the SF-36 may not be a relevant score for assessing many musculoskeletal conditions, citing issues of sensitivity, relatively large floor and ceiling effects, and one scoring method that creates a lack of correlation between the two summary scores (20,34,44). Therefore, while useful as a general health assessment tool, its ability to predict function and satisfaction after ACL reconstruction may be limited, and supplementation using knee-specific PRO measures should be considered (33,34).

In light of the challenge in administering a 36-item survey, a 12-item shortened version of the form was created. It has been validated, and psychometrically analyzed in orthopedic populations (21,45). Named the SF-12, it has been shown to be a suitable alternative, displaying strong correlations to its lengthier counterpart, the SF-36 (22). It is our recommendation that the SF-12 be implemented into daily practice when considering a general health assessment form.

EuroQol-5D (EQ-5D)

Another common assessment for general health is the EQ-5D (Table 1). Despite European origin, U.S. normative index scores have been studied across different ages (46). Five questions and a vertically oriented VAS allow the patient to report overall health status in a simplified manner (25). It has been shown in total hip arthroplasty patients that the newer five answer choice version (EQ-5D-5L) can distinguish mores health states than the three-answer variety (EQ-5D-3L), making it a more sensitive instrument with reduced floor and ceiling effects (26,27). Advantages of the EQ-5D include its brief nature (5 questions + a VAS) and standardized population norm based scoring. Furthermore, these scores can be converted to health utility states, thereby facilitating health utility research.

Few of its validation studies focus on orthopedic use (26,47). Of those that do, none has assessed validity on ACL injury or reconstructed patients. Despite this void, the EQ-5D is commonly accepted as a good measure of general health overall, and has nonetheless been used to assess quality of life after ACL reconstruction (48). Use of the EQ-5D over the SF-36 seems to be mostly preference-based, with availability and cost being key factors (Table 1). This questionnaire has been the more preferred choice in European studies, but its brevity, and free access make it a viable option for assessment of ACL quality of life in North America as well.

Knee-specific quality of life assessments

Outside of generic health surveys, there are a handful of other knee-specific quality of life assessments. The ACL-QoL (Table 1) was originally validated and developed by Mohtadi et al. in chronic ACL deficient patients, but has recently been further validated in a broader sample of ACL tears (29,30). The original study showed excellent test-retest reliability with no significant difference between administrations of survey, with average error of the total score around 6%. Additionally, 84% of the survey takers displayed appropriate responsiveness alongside clinical change (29). The more recent investigation revealed adequate internal consistency, as evaluated by Cronbach’s alpha, among meeting other metrics demonstrated by high-quality instruments (30). It is a 32-item, all VAS questionnaire, which may be a substantial addition to the other forms and surveys filled out by patients in the office. Other limitations outlined by some maintain that the VAS style questions do not allow for analysis of symptoms based off of functional limitations (8). Nevertheless, a study comparing knee quality of life questionnaires found that the ACL-QoL (Mohtadi QoL) had the highest percentage of questions endorsed by ACL deficient patients as being important in comparison to other knee specific questionnaires (49). Some propose that the ACL-QoL is the most relevant quality of life assessment and thus most truly “valid” and appropriate to collect for ACL patients (50). Perhaps further scientific investigation might help solidify the role of this undervalued PRO measure.

The Knee Injury and Osteoarthritis Outcome Score (KOOS—discussed more in depth below) has a quality of life component (KOOS-QoL) (Table 1). This subsection consists of four questions and has been used as PRO measure on its own in the literature (33). It has been shown to distinguish patients with an increased risk of ACL reconstruction failure, with a reported clinical failure score of <44, and every 10-point reduction in KOOS-QoL score resulting in 33.6% higher risk for a later revision (51). The KOOS validation study showed that the QoL section showed the largest effect size at 6 months post-op, indicating a highly practical significance as score differences during administration are readily apparent (28). The brevity of this questionnaire may be an enticing choice for the busy provider looking to assess quality of life in his/her ACL patient population.

Few cost-effectiveness studies use knee-specific questionnaires to assess quality of life (52). This is an interesting point since it is known that knee-specific measures produce lower health related quality of life scores than more generic surveys such as the SF-36 (33). Researchers looking to fill this gap could offer an alternative economic analysis using knee specific quality of life measures that is perhaps more accurate and applicable to the decision-making of an ACL/sports surgeon.

Knee and disease-specific PRO measures

The following are well-established and the most commonly used PRO measures for ACL reconstruction reported in recent literature (53): the International Knee Documentation Committee Subjective Knee Form (IKDC-SKF), Lysholm Knee Score, Tegner activity scale, KOOS, and the CKRS (Table 2).

Table 2 Overview of knee-specific PROMs
Full table

International Knee Documentation Committee (IKDC) Subjective Knee Form (SKF)

The IKDC (Table 2) is the most widely used PRO measure in high impact literature to evaluate patient outcomes following ACL reconstruction, appearing in 71% of studies from 2010 through 2014 (53). Furthermore, normative data has been established and easily available for comparison purposes (54). Higgins et al. determined validity with statistically significant correlations to the SF-12 in 534 ACL patients, although higher correlation was expectedly seen with the PCS score rather than the MCS (55). This corroborates the original findings of the validation study by Irrgang et al. done in relation to the SF-36 (9). Furthermore, Higgins et al. found internal consistency coefficients were notably high at 0.87 and 0.88, above the acceptable standard of 0.70 (55). The original validation paper commented on high test-retest reliability coefficients (0.94) although only a small sample size was studied, on average 50 days apart in survey administration (n=33) (9).

Responsiveness of the IKDC-SKF was assessed by Irrgang’s group in 207 patients and displayed adequate responsiveness to clinical change, even in the ACL injury sub-cohort (n=50). However, only 22 of those subjects underwent reconstruction, with no comment on the responsiveness post-operatively (56). Other investigations reveal that the IKDC-SKF outperforms the KOOS in a head to head comparison when monitoring ACL injuries and reconstruction with regards to measurement properties (62) and patient perception (63). The IKDC-SKF is the preferred choice to track PROs in ACL reconstructed patients, with well-established measurement properties and patient relevance.

Lysholm knee score and Tegner activity scale

The Lysholm Knee Score (Table 2) is one of the oldest knee-specific PRO measures (57). It was first described in 1982 in ACL patients and later modified in 1985 to capture meniscal injury outcomes (58). Population norms across several age ranges are available in individuals with normal knees for reference (64,65). In 2009, Briggs et al. studied the Lysholm knee score in over 1,000 ACL reconstructed patients and showed acceptable validity, test-retest reliability, and responsiveness. Although satisfactory, the mentioned Lysholm studies report lower values of internal consistency (Cronbach’s alpha =0.72) than IKDC and CKRS studies outlined in this review. Additionally, there was an inherent and noteworthy lack of correlation to the mental component score of the SF-12 likely due to its anatomic/region-specific design, as expected (58). Interestingly, an ACL reconstruction study found that the Lysholm Knee Score can be accurately administered over the phone with comparable results to a face-to-face interview (66). The brief nature and convenience of administration of this score affords it to be one of the more commonly used validated knee-specific scores (53).

The Tegner activity scale (Table 2) was designed to be a complementary measure to the Lysholm in order to allow clinicians to detect whether decreased activity levels influence the functional scores of the Lysholm. As such, the Tegner Activity Scale was not meant to be used on its own (57). Although not specific to ACL injuries, it is the third most commonly used PRO in ACL studies from 2010 through 2014, behind the Lysholm (53). Average scores in normal knees have been reported for reference (65). The Tegner’s recent and confirmatory validation study in ACL reconstructed patients was intuitively conducted by the same group that studied the Lysholm Knee Score, Briggs et al. In 505 patients, they found similar results with acceptable psychometric parameters such as test-retest reliability, internal consistency, validity, and responsiveness (58). The authors of this recent validation study recommend to use the two scales at 6, 9, and 12 months, in addition to longer follow-up following ACL reconstruction (58). With continued validity in ACL reconstructed patients, these long-standing PRO measures remain appropriate for use in these patients.

KOOS

In three of its five total subscales, the fully patient-reported KOOS (Table 2) includes all questions in original format from the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) to maintain validity and comprehensiveness in a variety of knee conditions. As it pertains to ACL reconstruction, the questionnaire has been shown to be valid, responsive, and reliable since 1998 (28). Largely, this is likely a result of the addition of two subscales to the WOMAC domains, Sport/Recreation and Quality of Life, which allow the instrument to measure function after acute knee injuries in more active and youthful patients and better assess magnitude of change (67). It is recommended that the subscale scores not be aggregated into a total, rather each subscore interpreted separately (28). An updated study of population norms was recently conducted, providing important reference values (68). A recent KOOS systematic review and meta-analysis verifies the original findings of adequate measurement properties, however only a few of the pooled studies were conducted in ACL injured patients (69). The KOOS in its entirety is considered by some as an invalid PRO measure for the ACL reconstructed patient (50). Others agree, with the caveat that only the QoL and Sports/Rec subsections display ACL appropriate unidimensionality, and psychometrics using a Rasch model analysis (67). Overall, it is freely available and is the fourth most commonly used PRO measure in ACL literature in recent years (53). Though, the number of questions and unrelated subscales make it a weaker choice for ACL reconstruction reporting.

CKRS

The CKRS was first referenced in the literature in 1983. Like the IKDC, it contains subjective and objective portions (Table 2). A modified version exists that focuses on the patient-reported input only and is known as the modified CKRS (70). The CKRS in its entirety has shown to be valid, responsive, and statistically reliable in ACL injured and reconstructed patients (8,71). Similarly, adequate measurement properties have been shown in the modified CKRS in a variety of knee disorders (70,72).

In recent years, the CKRS is used less often in ACL reconstruction literature compared to the other measures mentioned in this review (53). Additionally, there has been no statistical evaluation of the internal consistency of the CKRS in any knee condition, to our knowledge. Although it has been evaluated to be a high-quality instrument, the inconsistencies of the various survey versions in the literature, make the CKRS a suboptimal choice for use in ACL reconstructed patients. Furthermore, the briefer and patient-reported modified version has not been psychometrically tested in ACL patients, even though these properties have been shown to be acceptable in other knee disorders. The CKRS strengths rely on its objective outcome components, and therefore better options are available for patient reported input.

Newer ACL-specific PRO measures

In the last three years, other PRO assessments that have been developed for ACL specific use; these include the KNEES-ACL and the JACL-25 (Table 2). The KNEES-ACL is a 41 item PRO measure that was created using a Rasch analysis of a larger pilot questionnaire developed by literature reviews and patient interviews. Specifically designed for ACL deficient and reconstructed patients, it contains seven constructs, including a psychosocial domain, that have been psychometrically satisfactory in 242 patients (59,60).

The JACL-25 is one of the newest ACL-specific PRO measure developed by a group out of Japan. It consists of 25 questions scored on a 5-point scale, totaling 100 points. Strong correlations to the IKDC, Lysholm, and Tegner have been shown along with reliability and responsiveness (61). For both questionnaires, weaknesses are based off its limited use. Use of these newer questionnaires may lead to innovative reports, but further evaluation in large diverse sample populations are needed to solidify their position in ACL reconstruction outcomes reporting.

Milestone PRO measures: return to activity

It is our opinion that, among patients with acute ACL injury and subsequent treatment or reconstruction, the most important clinical outcome is the time required to return to pre-injury level of function, as opposed to a more arbitrary score on any given PRO measure. However, studies rarely report this important statistic (53). Therefore, we propose that clinicians consider a “milestone PRO” when evaluating these high-demand patients.

Milestone reporting would allow clinicians to report metrics that are most important to patients: return to activity. For example, for an athlete, the milestone reporting would include the time needed to return to full competition following ACL reconstruction. For a laborer, that data point would report time to return to full work capacity. One advantage of this outcome is that it is extremely quick and easy to report, thereby providing a potentially significant advantage over traditional PRO.

Evolving role of NIH PROMIS questionnaires

Recently, the NIH developed the Patient-Reported Outcome Measure Information System (PROMIS) in an effort to standardize outcome collection using valid and reliable PRO measures. This system offers many questionnaires, but the one most relevant to ACL reconstruction—the one gaining the most traction in the orthopedic community—is PROMIS Physical Function (PF) (Table 3). This form, which measures a patient’s self-reported physical capability, is available as either a static short form (SF) with a fixed number of questions, or as a dynamic computerized adaptive test (CAT) that uses patient responses to determine subsequent questions.

Table 3 Psychological assessments and NIH PROMIS questionnaires
Full table

Currently, the data on PROMIS PF for use in ACL reconstruction is very limited. A study by Papuga et al. compared IKDC and PROMIS PF CAT score for 106 patients undergoing ACL reconstruction and found a significant relationship between these PROs with a correlation coefficient of 0.8954. Furthermore, PROMIS PF CAT was more sensitive than the IKDC in detecting a decrease in physical function at 3 weeks post-operative and an increase in physical function at 10 weeks post-operative, while also taking significantly less time to complete (77).

Other studies have demonstrated similar findings for PROMIS PF CAT against the SF-36 general health questionnaire for foot and ankle surgery, revealing these measures to be strongly correlated at 0.79 with PROMIS PF CAT having less unexplained variance, floor effects, and ceiling effects (80,81). However, to date, no studies have directly compared these outcome measures in the setting of ACL reconstruction. It is assumed that these CAT versions require electronic administration with computers, which can be an obstacle for some clinics, but advancements in tablet computers are making this obstacle easier and cheaper to hurdle.

Patient reported psychological assessment: important for ACL recovery?

Numerous studies have shown that psychological factors affect the outcomes of ACL reconstruction (82-84). For example, Christino et al. demonstrated significant relationships of self-esteem levels to functional test performance and measures such as the IKDC, SF-36 and the KOOS-QOL (84). Motivation level during rehabilitation is another important psychological factor for ACL reconstruction success and return to pre-injury level (85). With respect to knee injuries, an overarching concept in this field is self-efficacy, which describes thought processes in regards to judgement of capabilities, as it influences efforts expended when facing the difficulties of recovery (73). This summarizes the need for a psychological assessment tool to track and predict recovery from ACL reconstruction.

The Knee Self-Efficacy Scale (K-SES) (Table 3) has been validated in a sample of 104 ACL injured and reconstructed patients with acceptable test-retest reliability as well (73). The K-SES has been proven to be a great predictor of a patient’s return to intensity and frequency of physical activity at 1 year follow-up (86). The established validity and reliability in ACL reconstructed patients make the K-SES one of the strongest choices for assessing psychological factors before surgery and during recovery.

Another option is the ACL return to sport after injury questionnaire (ACL-RSI) (Table 3). It has demonstrated promising preliminary validation data (74), but further psychometric analyses have not been conducted. Some have used the ACL-RSI with success and have displayed that athletes who returned to sport pre-injury level had significantly greater psychological readiness (83). These findings suggest strong links between psychological factors and recovery, and even utility to gauging a patient’s mental state during ACL injury and recovery. While useful, the ACL-RSI needs further analysis of reliability and responsiveness, and thus is a secondary option to the more well-tested K-SES.

The TSK-11 (Table 3) is an 11-item short form designed to measure kinesophobia, or fear of movement. This survey has been validated, and shown appropriate internal consistency, reliability and responsiveness in chronic low back pain patients (75). Unfortunately, no studies have measured the psychometrics of this form in ACL injured and reconstructed patients. Nevertheless, a pertinent study in ACL reconstruction recovery revealed that the TSK-11 was able to exhibit a significant correlation of high levels of kinesophobia to difficulties experienced in daily activities and poorer knee related quality of life (76). Lack of rigorous psychometric evaluation in the ACL injured/reconstructed population supports a smaller role for the TSK-11 in this setting.

PROMIS also aims to standardize reporting of psychological factors with relevant forms for depression, and pain interference (PI) with day-to-day activities. These forms are also available in static short form or dynamic CAT versions. The depression CAT form pulls from a 28-item question bank and focuses on “the last seven days” time span (78). It has shown predictive value in patients undergoing orthopedic procedures (79). PROMIS PI CAT form also consists of an item bank, with 41 questions total to pull from. These questions were studied in patients with acute and chronic knee disorders, and were deemed to be acceptable for use in this population (87). These forms provide a medium for innovation in psychological outcomes reporting with regards to ACL reconstruction and an opportunity for expansion of research in this realm.

Summary

Clinical outcomes reporting serves two vital roles: helping monitor recovery following treatment as well as forming the foundation for patient-centered clinical research. As seen in this review, there are numerous ways by which outcomes can be measured following ACL treatment. These include objective outcomes, subjective outcomes, and PROs. Advantages of objective outcome reporting include the ability to compare outcomes across disparate patients and patient groups. However, this type of reporting may not adequately represent outcomes from the patient’s perspective. As a result, PROs have increased in usage in order to provide more patient-centered input. A number of different PRO scores exist and are available for use, all with respective advantages and disadvantages. Non-traditional types of reporting, such as assessment of patient psychological state and “milestone based reporting” should also be considered for consistent use when monitoring ACL reconstruction recovery.

Acknowledgments

Funding: None.

Footnote

Provenance and Peer Review: This article was commissioned by the Guest Editors (Freddie H. Fu and Jeremy M. Burnham) for the series “Trends in ACL Reconstruction” published in Annals of Joint. The article has undergone external peer review.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/aoj.2017.05.10). The series “Trends in ACL Reconstruction” was commissioned by the editorial office without any funding or sponsorship. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Lavallee DC, Chenok KE, Love RM, et al. Incorporating Patient-Reported Outcomes Into Health Care To Engage Patients And Enhance Care. Health Aff (Millwood) 2016;35:575-82. [Crossref] [PubMed]
2. Services CfMaM. Comprehensive Care for Joint Replacement Model. Baltimore, MD. 2016. Available online: https://innovation.cms.gov/initiatives/cjr, accessed November 26 2016.
3. Services USDoHaH. HHS announces $685 million to support clinicians delivering high quality, patient-centered care. In: HHS News Releases. 2015. Available online: https://www.hhs.gov/about/news/2015/09/29/hhs-announces-685-million-support-clinicians-delivering-high-quality-patient-centered-care.html, accessed November 26 2016.
4. Doward LC, McKenna SP. Defining patient-reported outcomes. Value Health 2004;7:S4-8. [Crossref] [PubMed]
5. Irrgang JJ, Anderson AF. Development and validation of health-related quality of life measures for the knee. Clin Orthop Relat Res 2002;95-109. [Crossref] [PubMed]
6. Hawkins RJ. Recommendations for Evaluating and Selecting Appropriately Valued Outcome Measures. Instr Course Lect 2016;65:587-91. [PubMed]
7. Mohtadi NG. Outcome Measure Development. Instr Course Lect 2016;65:577-82. [PubMed]
8. Barber-Westin SD, Noyes FR, McCloskey JW. Rigorous statistical reliability, validity, and responsiveness testing of the Cincinnati knee rating system in 350 subjects with uninjured, injured, or anterior cruciate ligament-reconstructed knees. Am J Sports Med 1999;27:402-16. [PubMed]
9. Irrgang JJ, Anderson AF, Boland AL, et al. Development and validation of the international knee documentation committee subjective knee form. Am J Sports Med 2001;29:600-13. [PubMed]
10. Collette M, Courville J, Forton M, et al. Objective evaluation of anterior knee laxity; comparison of the KT-1000 and GNRB(R) arthrometers. Knee Surg Sports Traumatol Arthrosc 2012;20:2233-8. [Crossref] [PubMed]
11. Macdonald SA, Heard SM, Hiemstra LA, et al. A comparison of pain scores and medication use in patients undergoing single-bundle or double-bundle anterior cruciate ligament reconstruction. Can J Surg 2014;57:E98-104. [Crossref] [PubMed]
12. Sacramento SN, Magalhaes E, Christel P, et al. A new technique in double-bundle anterior cruciate ligament reconstruction with implant-free tibial fixation. Knee Surg Sports Traumatol Arthrosc 2016;24:2831-7. [Crossref] [PubMed]
13. Zampelis V, Ornstein E, Franzen H, et al. A simple visual analog scale for pain is as responsive as the WOMAC, the SF-36, and the EQ-5D in measuring outcomes of revision hip arthroplasty. Acta Orthop 2014;85:128-32. [Crossref] [PubMed]
14. Flaherty SA. Pain measurement tools for clinical practice and research. AANA J 1996;64:133-40. [PubMed]
15. Shelbourne KD, Barnes AF, Gray T. Correlation of a single assessment numeric evaluation (SANE) rating with modified Cincinnati knee rating system and IKDC subjective total scores for patients after ACL reconstruction or knee arthroscopy. Am J Sports Med 2012;40:2487-91. [Crossref] [PubMed]
16. McHorney CA, Ware JE Jr, Lu JF, et al. The MOS 36-item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care 1994;32:40-66. [Crossref] [PubMed]
17. McHorney CA, Ware JE Jr, Raczek AE. The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care 1993;31:247-63. [Crossref] [PubMed]
18. McHorney CA, Ware JE Jr, Rogers W, et al. The validity and relative precision of MOS short- and long-form health status scales and Dartmouth COOP charts. Results from the Medical Outcomes Study. Med Care 1992;30:MS253-65. [Crossref] [PubMed]
19. Ware JE. User's Manual for the SF-36v2 Health Survey. Second ed. Quality Metric, 2009.
20. Laucis NC, Hays RD, Bhattacharyya T. Scoring the SF-36 in Orthopaedics: A Brief Guide. J Bone Joint Surg Am 2015;97:1628-34. [Crossref] [PubMed]
21. Gandhi SK, Salmon JW, Zhao SZ, et al. Psychometric evaluation of the 12-item short-form health survey (SF-12) in osteoarthritis and rheumatoid arthritis clinical trials. Clin Ther 2001;23:1080-98. [Crossref] [PubMed]
22. Webster KE, Feller JA. Use of the short form health surveys as an outcome measure for anterior cruciate ligament reconstruction. Knee Surg Sports Traumatol Arthrosc 2014;22:1142-8. [Crossref] [PubMed]
23. Mather RC 3rd, Hettrich CM, Dunn WR, et al. Cost-Effectiveness Analysis of Early Reconstruction Versus Rehabilitation and Delayed Reconstruction for Anterior Cruciate Ligament Tears. Am J Sports Med 2014;42:1583-91. [Crossref] [PubMed]
24. Mather RC 3rd, Koenig L, Kocher MS, et al. Societal and economic impact of anterior cruciate ligament tears. J Bone Joint Surg Am 2013;95:1751-9. [Crossref] [PubMed]
25. Rabin R, de Charro F. EQ-5D: a measure of health status from the EuroQol Group. Ann Med 2001;33:337-43. [Crossref] [PubMed]
26. Greene ME, Rader KA, Garellick G, et al. The EQ-5D-5L Improves on the EQ-5D-3L for Health-related Quality-of-life Assessment in Patients Undergoing Total Hip Arthroplasty. Clin Orthop Relat Res 2015;473:3383-90. [Crossref] [PubMed]
27. Feng Y, Devlin N, Herdman M. Assessing the health of the general population in England: how do the three- and five-level versions of EQ-5D compare? Health Qual Life Outcomes 2015;13:171. [Crossref] [PubMed]
28. Roos EM, Roos HP, Lohmander LS, et al. Knee Injury and Osteoarthritis Outcome Score (KOOS)--development of a self-administered outcome measure. J Orthop Sports Phys Ther 1998;28:88-96. [Crossref] [PubMed]
29. Mohtadi N. Development and validation of the quality of life outcome measure (questionnaire) for chronic anterior cruciate ligament deficiency. Am J Sports Med 1998;26:350-9. [PubMed]
30. Lafave MR, Hiemstra L, Kerslake S, et al. Validity, Reliability, and Responsiveness of the Anterior Cruciate Ligament Quality of Life Measure: A Continuation of Its Overall Validation. Clin J Sport Med 2017;27:57-63. [Crossref] [PubMed]
31. Ardern CL, Österberg A, Sonesson S, et al. Satisfaction With Knee Function After Primary Anterior Cruciate Ligament Reconstruction Is Associated With Self-Efficacy, Quality of Life, and Returning to the Preinjury Physical Activity. Arthroscopy 2016;32:1631-1638.e3. [Crossref] [PubMed]
32. Dunn WR, Wolf BR, Harrell FE Jr, et al. Baseline predictors of health-related quality of life after anterior cruciate ligament reconstruction: a longitudinal analysis of a multicenter cohort at two and six years. J Bone Joint Surg Am 2015;97:551-7. [Crossref] [PubMed]
33. Filbay SR, Ackerman IN, Russell TG, et al. Health-related quality of life after anterior cruciate ligament reconstruction: a systematic review. Am J Sports Med 2014;42:1247-55. [Crossref] [PubMed]
34. Patel AA, Donegan D, Albert T. The 36-item short form. J Am Acad Orthop Surg 2007;15:126-34. [Crossref] [PubMed]
35. Perkins AJ, Stump TE, Monahan PO, et al. Assessment of differential item functioning for demographic comparisons in the MOS SF-36 health survey. Qual Life Res 2006;15:331-48. [Crossref] [PubMed]
36. Busija L, Osborne RH, Nilsdotter A, et al. Magnitude and meaningfulness of change in SF-36 scores in four types of orthopedic surgery. Health Qual Life Outcomes 2008;6:55. [Crossref] [PubMed]
37. Kosinski M, Keller SD, Ware JE Jr, et al. The SF-36 Health Survey as a generic outcome measure in clinical trials of patients with osteoarthritis and rheumatoid arthritis: relative validity of scales in relation to clinical measures of arthritis severity. Med Care 1999;37:MS23-39. [PubMed]
38. Ruta DA, Hurst NP, Kind P, et al. Measuring health status in British patients with rheumatoid arthritis: reliability, validity and responsiveness of the short form 36-item health survey (SF-36). Br J Rheumatol 1998;37:425-36. [Crossref] [PubMed]
39. Månsson O, Kartus J, Sernert N. Health-related quality of life after anterior cruciate ligament reconstruction. Knee Surg Sports Traumatol Arthrosc 2011;19:479-87. [Crossref] [PubMed]
40. Möller E, Weidenhielm L, Werner S. Outcome and knee-related quality of life after anterior cruciate ligament reconstruction: a long-term follow-up. Knee Surg Sports Traumatol Arthrosc 2009;17:786-94. [Crossref] [PubMed]
41. Núñez M, Sastre S, Núñez E, Lozano L, et al. Health-related quality of life and direct costs in patients with anterior cruciate ligament injury: single-bundle versus double-bundle reconstruction in a low-demand cohort--a randomized trial with 2 years of follow-up. Arthroscopy 2012;28:929-35. [Crossref] [PubMed]
42. Ochiai S, Hagino T, Tonotsuka H, et al. Prospective analysis of health-related quality of life and clinical evaluations in patients with anterior cruciate ligament injury undergoing reconstruction. Arch Orthop Trauma Surg 2011;131:1091-4. [Crossref] [PubMed]
43. Shapiro ET, Richmond JC, Rockett SE, et al. The use of a generic, patient-based health assessment (SF-36) for evaluation of patients with anterior cruciate ligament injuries. Am J Sports Med 1996;24:196-200. [Crossref] [PubMed]
44. Obremskey WT, Brown O, Driver R, et al. Comparison of SF-36 and Short Musculoskeletal Functional Assessment in recovery from fixation of unstable ankle fractures. Orthopedics 2007;30:145-51. [PubMed]
45. Gosling CM, Gabbe BJ, Williamson OD, et al. Validity of outcome measures used to assess one and six month outcomes in orthopaedic trauma patients. Injury 2011;42:1443-8. [Crossref] [PubMed]
46. Fryback DG, Dunham NC, Palta M, et al. US norms for six generic health-related quality-of-life indexes from the National Health Measurement study. Med Care 2007;45:1162-70. [Crossref] [PubMed]
47. Conner-Spady BL, Marshall DA, Bohm E, et al. Reliability and validity of the EQ-5D-5L compared to the EQ-5D-3L in patients with osteoarthritis referred for hip and knee replacement. Qual Life Res 2015;24:1775-84. [Crossref] [PubMed]
48. Fältström A, Hägglund M, Kvist J. Patient-reported knee function, quality of life, and activity level after bilateral anterior cruciate ligament injuries. Am J Sports Med 2013;41:2805-13. [Crossref] [PubMed]
49. Tanner SM, Dainty KN, Marx RG, et al. Knee-specific quality-of-life instruments: which ones measure symptoms and disabilities most important to patients? Am J Sports Med 2007;35:1450-8. [Crossref] [PubMed]
50. Zarins B. Are validated questionnaires valid? J Bone Joint Surg Am 2005;87:1671-2. [PubMed]
51. Granan LP, Baste V, Engebretsen L, et al. Associations between inadequate knee function detected by KOOS and prospective graft failure in an anterior cruciate ligament-reconstructed knee. Knee Surg Sports Traumatol Arthrosc 2015;23:1135-40. [Crossref] [PubMed]
52. Paxton ES, Kymes SM, Brophy RH. Cost-effectiveness of anterior cruciate ligament reconstruction: a preliminary comparison of single-bundle and double-bundle techniques. Am J Sports Med 2010;38:2417-25. [Crossref] [PubMed]
53. Makhni EC, Padaki AS, Petridis PD, et al. High Variability in Outcome Reporting Patterns in High-Impact ACL Literature. J Bone Joint Surg Am 2015;97:1529-42. [Crossref] [PubMed]
54. Anderson AF, Irrgang JJ, Kocher MS, et al. The International Knee Documentation Committee Subjective Knee Evaluation Form: normative data. Am J Sports Med 2006;34:128-35. [Crossref] [PubMed]
55. Higgins LD, Taylor MK, Park D, et al. Reliability and validity of the International Knee Documentation Committee (IKDC) Subjective Knee Form. Joint Bone Spine 2007;74:594-9. [Crossref] [PubMed]
56. Irrgang JJ, Anderson AF, Boland AL, et al. Responsiveness of the International Knee Documentation Committee Subjective Knee Form. Am J Sports Med 2006;34:1567-73. [Crossref] [PubMed]
57. Tegner Y, Lysholm J. Rating systems in the evaluation of knee ligament injuries. Clin Orthop Relat Res 1985;43-9. [PubMed]
58. Briggs KK, Lysholm J, Tegner Y, et al. The reliability, validity, and responsiveness of the Lysholm score and Tegner activity scale for anterior cruciate ligament injuries of the knee: 25 years later. Am J Sports Med 2009;37:890-7. [Crossref] [PubMed]
59. Comins JD, Krogsgaard MR, Brodersen J. Development of the Knee Numeric-Entity Evaluation Score (KNEES-ACL): a condition-specific questionnaire. Scand J Med Sci Sports 2013;23:e293-301. [PubMed]
60. Comins JD, Krogsgaard MR, Kreiner S, et al. Dimensionality of the Knee Numeric-Entity Evaluation Score (KNEES-ACL): a condition-specific questionnaire. Scand J Med Sci Sports 2013;23:e302-12. [PubMed]
61. Nagao M, Doi T, Saita Y, et al. A novel patient-reported outcome measure for anterior cruciate ligament injury: evaluating the reliability, validity, and responsiveness of Japanese anterior cruciate ligament questionnaire 25. Knee Surg Sports Traumatol Arthrosc 2016;24:2973-82. [Crossref] [PubMed]
62. van Meer BL, Meuffels DE, Vissers MM, et al. Knee injury and Osteoarthritis Outcome Score or International Knee Documentation Committee Subjective Knee Form: which questionnaire is most useful to monitor patients with an anterior cruciate ligament rupture in the short term? Arthroscopy 2013;29:701-15. [Crossref] [PubMed]
63. Hambly K, Griva K. IKDC or KOOS: which one captures symptoms and disabilities most important to patients who have undergone initial anterior cruciate ligament reconstruction? Am J Sports Med 2010;38:1395-404. [Crossref] [PubMed]
64. Demirdjian AM, Petrie SG, Guanche CA, et al. The outcomes of two knee scoring questionnaires in a normal population. Am J Sports Med 1998;26:46-51. [PubMed]
65. Briggs KK, Steadman JR, Hay CJ, et al. Lysholm score and Tegner activity level in individuals with normal knees. Am J Sports Med 2009;37:898-901. [Crossref] [PubMed]
66. Kose O, Deniz G, Ozcan H, et al. A comparison of telephone interview versus on-site completion of Lysholm knee score in patients who underwent arthroscopic ACL reconstruction: are the results equivalent? Eur J Orthop Surg Traumatol 2015;25:1069-72. [Crossref] [PubMed]
67. Comins J, Brodersen J, Krogsgaard M, et al. Rasch analysis of the Knee injury and Osteoarthritis Outcome Score (KOOS): a statistical re-evaluation. Scand J Med Sci Sports 2008;18:336-45. [Crossref] [PubMed]
68. Williamson T, Sikka R, Tompkins M, et al. Use of the Knee Injury and Osteoarthritis Outcome Score in a Healthy United States Population. Am J Sports Med 2016;44:440-6. [Crossref] [PubMed]
69. Collins NJ, Prinsen CA, Christensen R, et al. Knee Injury and Osteoarthritis Outcome Score (KOOS): systematic review and meta-analysis of measurement properties. Osteoarthritis Cartilage 2016;24:1317-29. [Crossref] [PubMed]
70. Agel J, LaPrade RF. Assessment of differences between the modified Cincinnati and International Knee Documentation Committee patient outcome scores: a prospective study. Am J Sports Med 2009;37:2151-7. [Crossref] [PubMed]
71. Hopper DM, Goh SC, Wentworth LA, et al. Test-retest reliability of knee rating scales and functional hop tests one year following anterior cruciate ligament reconstruction. Physical Therapy in Sport 2002;3:10-8. [Crossref]
72. Greco NJ, Anderson AF, Mann BJ, et al. Responsiveness of the International Knee Documentation Committee Subjective Knee Form in comparison to the Western Ontario and McMaster Universities Osteoarthritis Index, modified Cincinnati Knee Rating System, and Short Form 36 in patients with focal articular cartilage defects. Am J Sports Med 2010;38:891-902. [Crossref] [PubMed]
73. Thomeé P, Währborg P, Börjesson M, et al. A new instrument for measuring self-efficacy in patients with an anterior cruciate ligament injury. Scand J Med Sci Sports 2006;16:181-7. [Crossref] [PubMed]
74. Webster KE, Feller JA, Lambros C. Development and preliminary validation of a scale to measure the psychological impact of returning to sport following anterior cruciate ligament reconstruction surgery. Phys Ther Sport 2008;9:9-15. [Crossref] [PubMed]
75. Woby SR, Roach NK, Urmston M, et al. Psychometric properties of the TSK-11: a shortened version of the Tampa Scale for Kinesiophobia. Pain 2005;117:137-44. [Crossref] [PubMed]
76. Tichonova A, Rimdeikiene I, Petruseviciene D, et al. The relationship between pain catastrophizing, kinesiophobia and subjective knee function during rehabilitation following anterior cruciate ligament reconstruction and meniscectomy: A pilot study. Medicina (Kaunas) 2016;52:229-37. [Crossref] [PubMed]
77. Papuga MO, Beck CA, Kates SL, et al. Validation of GAITRite and PROMIS as high-throughput physical function outcome measures following ACL reconstruction. J Orthop Res 2014;32:793-801. [Crossref] [PubMed]
78. Choi SW, Schalet B, Cook KF, et al. Establishing a common metric for depressive symptoms: linking the BDI-II, CES-D, and PHQ-9 to PROMIS depression. Psychol Assess 2014;26:513-27. [Crossref] [PubMed]
79. Ho B, Houck JR, Flemister AS, et al. Preoperative PROMIS Scores Predict Postoperative Success in Foot and Ankle Patients. Foot Ankle Int 2016;37:911-8. [Crossref] [PubMed]
80. Driban JB, Morgan N, Price LL, et al. Patient-Reported Outcomes Measurement Information System (PROMIS) instruments among individuals with symptomatic knee osteoarthritis: a cross-sectional study of floor/ceiling effects and construct validity. BMC Musculoskelet Disord 2015;16:253. [Crossref] [PubMed]
81. Hung M, Franklin JD, Hon SD, et al. Time for a paradigm shift with computerized adaptive testing of general physical function outcomes measurements. Foot Ankle Int 2014;35:1-7. [Crossref] [PubMed]
82. Ardern CL, Taylor NF, Feller JA, et al. Psychological responses matter in returning to preinjury level of sport after anterior cruciate ligament reconstruction surgery. Am J Sports Med 2013;41:1549-58. [Crossref] [PubMed]
83. Ardern CL, Taylor NF, Feller JA, et al. Sports participation 2 years after anterior cruciate ligament reconstruction in athletes who had not returned to sport at 1 year: a prospective follow-up of physical function and psychological factors in 122 athletes. Am J Sports Med 2015;43:848-56. [Crossref] [PubMed]
84. Christino MA, Fleming BC, Machan JT, et al. Psychological Factors Associated With Anterior Cruciate Ligament Reconstruction Recovery. Orthop J Sports Med 2016;4:2325967116638341 [Crossref] [PubMed]
85. Sonesson S, Kvist J, Ardern C, et al. Psychological factors are important to return to pre-injury sport activity after anterior cruciate ligament reconstruction: expect and motivate to satisfy. Knee Surg Sports Traumatol Arthrosc 2017;25:1375-84. [PubMed]
86. Thomeé P, Währborg P, Börjesson M, et al. Self-efficacy of knee function as a pre-operative predictor of outcome 1 year after anterior cruciate ligament reconstruction. Knee Surg Sports Traumatol Arthrosc 2008;16:118-27. [Crossref] [PubMed]
87. Lynch AD, Dodds NE, Yu L, et al. Individuals with knee impairments identify items in need of clarification in the Patient Reported Outcomes Measurement Information System (PROMIS®) pain interference and physical function item banks - a qualitative study. Health Qual Life Outcomes 2016;14:77. [Crossref] [PubMed]