Cognitive Impairment in Patients with Multiple Sclerosis as Assessed by Objective Computerized Testing

*Roberto Bomprezzi
Department Of Neurology, University Of Massachusetts Medical School, Massachusetts, United States

*Corresponding Author:
Roberto Bomprezzi
Department Of Neurology, University Of Massachusetts Medical School, Massachusetts, United States
Email:rbomprez@gmail.com

Published on: 2020-11-27

Abstract

Background: Cognitive impairment (CI) has a substantial impact on quality of life in patients with multiple sclerosis (MS), but testing has been limited. A brief, easy-to-administer neuropsychological test could increase the frequency of routine assessment of CI in patients with MS, leading to a positive impact on disease management.

Methods: The study, conducted at the University of Massachusetts Medical School enrolled consecutive patients who consented to testing and it utilized Cognivue®, which is an FDA-cleared computerized testing tool designed to assess early signs of CI. Participants completed the Expanded Disability Status Scale (EDSS), symbol digit modality test (SDMT), Nine-Hole Peg Test, timed 25-foot walk, and 10-minute Cognivue® testing (basic motor & visual ability, perceptual processing, and memory processing). Statistical analyses using a one-way ANOVA were performed to determine differences between neuropsychological testing methods.

Results: Thirty-six patients (mean age 48.6 y [range 20-74], 78% female [n=28/36]), completed the tests. Based on Cognivue® scores, 50% of patients were categorized as having normal cognitive function (mean 84.7; EDSS 2.64), 33.3% as having low CI (mean 66.0; EDSS 3.38), and 16.7% as having moderate to severe CI (mean 39.2; EDSS 5.17). Overall Cognivue® scores demonstrated statistically significant correlations with EDSS (Pearson correlation coefficient -0.54), SDMT (0.67), and timed 25-foot walk (-0.56). No relationship was seen between patient age and Cognivue® scores. All key cognitive domains were equally affected.

Conclusions: When MS affects cognition, all spheres are impacted and Cognivue® proved helpful in detecting multi-domain CI providing an opportunity for early intervention and potentially improving outcomes.

Keywords

Cognitive Impairment, Multiple Sclerosis, Spheres of Cognition, Computerized Testing

Introduction

Cognitive impairment (CI) appears to be highly prevalent in patients with multiple sclerosis (MS), however rates reported in the literature vary widely (28-91%) [1-6], which may be attributable, in part, to different criteria used to classify CI. A more consistent measure of CI among individual patients with MS, as well as on an interpatient basis, would help provide clinicians with the means to accurately assess, monitor, and manage MS-related cognitive impairments.

In his early descriptions of the neurological characteristics of MS patients, Charcot highlighted slowing of mentation, deficits of memory and information processing, and those are the most commonly reported impairments in MS, with altered information processing being present in all MS phenotypes [2,4, and 7]. These cognitive deficits confer a substantial burden upon patients with MS, negatively affecting both quality of life and productivity [8].

Traditional cognitive assessment methods such as the St. Louis University Mental Status (SLUMS) examination, Mini-Mental Status Examination (MMSE), and Montreal Cognitive Assessment (MoCA) rely on a pen and paper format. These tests can have multiple drawbacks like educational, language, gender, and cultural biases [9,10], as well as a lack of sensitivity [11,12], subjectivity [13], and retest reliability [14], but they are used in clinical practice, and the MMSE in particular has been shown to be insufficient for detecting cognitive deficits in the context of MS [2]. Screening and monitoring tools better suited for an MS patient population, such as the symbol digit modality test (SDMT), Processing Speed Test (PST), Computerized Speed Cognitive Test (CSCT), Multiple Sclerosis Neuropsychological Screening Questionnaire (MSNQ), Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS), Brief Repeatable Neuropsychological Battery (BRNB), and Minimal Assessment of Cognitive Function in MS (MACFIMS), may be more useful [2].

However, tools for the routine evaluation of cognitive decline in patients with MS are rarely utilized in a regular and systematic way in clinical practice, making it imperative that any novel testing method first overcome the various impediments of what is currently available. The lack of regular use may be due to the greater amount time required to administer traditional cognitive assessments than is generally available in the limited duration of office visits [8]. A brief, easy-to-administer neuropsychological test that can be easily integrated into clinical practice is needed to routinely assess cognitive status among patients with MS.

Cognivue®, a US Food and Drug Administration (FDA)-cleared computerized test rooted in adaptive psychophysics, was developed to evaluate cognitive function. In 2015, Cognivue® was granted de novo approval by the FDA for use as an adjunctive tool to aid in evaluating cognitive function [15], although Cognivue® is not intended to be used as the sole means of diagnosing CI.

Cognivue® eliminates the constraints and subjectivity commonly associated with current testing. The 10-minute, self-administered test uses scores from a sequence of tasks to provide clinicians and patients with a simple, easy-to-read two page report with an overall score and a subsequent breakdown into 6 key cognitive domains and 2 speed parameters.

Initial studies of Cognivue® included a comparative trial (using the SLUMS as a reference standard) to determine cut-off scores for classifications of impairment and a subsequent clinical trial to assess the validity, reliability, and psychometric properties of Cognivue® [16]. Participants were stratified according to SLUMS scoring levels of impairment and a Cognivue® cut-off score of 54.5 was found to correspond to the SLUMS cut-off score of <21 (impairment); a Cognivue® cut-off score of 78.5 was found to correspond to the SLUMS cut-off score of >26 (no impairment). Cut-off scores for Cognivue® scores were set at 55?64 for impairment and 74-79 for no impairment [16].

A second clinical trial validated the agreement between Cognivue® and SLUMS classifications of impairment, assessed the test-retest reliability of Cognivue®, and compared the psychometric properties of the Cognivue® sub-tests with other neuropsychological tests in 401 participants. Regression and rank linear regression analyses revealed good agreement between Cognivue® and SLUMS scores as well as similar reliability upon retesting for both tests. Cognivue® showed good overall psychometric validity with the other, traditional neuropsychological tests, correlating most closely with the tests measuring verbal processing, manual dexterity and speed, visual acuity, visuospatial and executive function, and speed and sequencing [16].

Taking advantage of the Cognivue’s ability to assess multiple domains, we asked the question whether any sphere of cognition would be preferentially more affected when CI was present in MS patients and in the current study, we sought to determine the utility of Cognivue® relative to the Expanded Disability Status Scale (EDSS), Symbol Digit Modality Test (SDMT), Nine-Hole Peg Test (NHPT), and a Timed 25-foot Walk (T25W).

Methods

Study Participants

Consecutive adults with MS who agreed to the testing were enrolled at the University of Massachusetts Medical School. The study was approved by the university's Institutional Review Board and all participants provided informed consent.

Study Design

Study subjects completed five tests during the study visit: the EDSS, the SDMT, the NHPT, a T25W, and Cognivue®.

The EDSS is a tool used to evaluate the degree of neurologic impairment in patients with MS. The scale is graded in 0.5 increments from 0 (normal) to 10 (death due to MS) and is based on impairment in eight functional systems (pyramidal, cerebellar, brain stem, sensory, bowel & bladder, visual, cerebral, and other) [17].

The SDMT is a neuropsychological tool commonly used in MS to assess processing speed in which the patient is asked to substitute a number (orally or written) for a random set of geometric figures. A SDMT score, which can range from zero to 110, is generally the total number of correct substitutions made within the 90 seconds. The SDMT has been shown to be sensitive to the slower information processing common among patients with MS [18].

The NHPT assesses manual dexterity in patients with MS. The test involves patients rapidly placing and removing nine pegs into nine holes. Scoring of the NHPT is typically represented by the number of seconds taken to complete the test [19].

The T25W test is used to measure a patient's level of walking disability. It is administered and scored in a standardized fashion with the patient being instructed to walk a 25-foot straight distance as quickly and safely as possible. The score is the average time from two successive tests [20].

Cognivue® has 3 main components: visuomotor and visual salience, perceptual processing, and memory processing. The 10-minute automated sequence includes a total of 10 sub-tests. While the sub-tests are scored separately from one another, those within the visuomotor component do not factor into the overall Cognivue® score but instead are used to ascertain the test-taker's level of adaptive motor control and dynamic visual contrast sensitivity in order to calibrate the remaining sub-tests to the specific individual. This also serves to nullify potential disability bias when assessing CI in patients with visual or motor skill deficits.

The overall Cognivue® score as well as those from the individual sub-tests of the perception and memory sub-batteries are expressed as a percentage of correct responses (0 to 100%). Additional descriptive aspects and sample screens of the individual sub-tests are shown in Table 1.

Statistical Methods

In order to determine the differences between the various neuropsychological tests, statistical analyses using a one-way ANOVA model were conducted with Pearson correlation coefficients being calculated.

Results

Demographic characteristics of the 36 study participants with MS are provided in Table 2. Age, sex, and MS profile were recorded at the initial study visit. The majority of patients were female and had relapsing-remitting MS.

Table 2: Demographic characteristics of study participants.

Characteristic

Patients with MS (n=36)

Mean age, y (range)

48.6 (20-74)

Sex, n (%)

Male

8 (22)

Female

28 (78)

MS profile, n (%)

Relapsing-remitting

30 (83.3)

Secondary progressive

5 (13.9)

Not recorded

1 (2.8)

 

Overall, cognitive function was classified as unimpaired in 50% of patients, while low and moderate to severe CI was found in 33% and 16.7% of patients, respectively. Mean overall Cognivue® scores and corresponding mean EDSS scores according to degree of CI are shown in Table 3.

Table 3: Mean scores according to degree of cognitive impairment in patients with MS (n=36).

Classification

Mean scores

Overall
Cognivue®

EDSS

SDMT

NHPT

T25W

Normal cognitive function (n=18)

84.7

2.64

59.1

14.0

6.0

Low CI (n=12)

66.0

3.38

49.2

13.5

6.7

Moderate to severe CI (n=6)

39.2

5.17

37.5

21.0

11.3

CI: cognitive impairment; EDSS: Expanded Disability Status Scale; NHPT: Nine-Hole Peg Test; SDMT: Symbol Digit Modality Test; T25W: timed 25-foot walk.

Mean overall scores for both the perception and memory sub-batteries of Cognivue®, as well as mean scores on the 10 individual sub-tests, according to impairment levels are shown in Table 4.

Table 4: Cognivue® sub-test scores according to impairment classification.

Cognivue® sub-test

Classification

Normal cognitive function (n=18)

Low CI (n=12)

Moderate to severe CI (n=6)

Overall perception

83.5

62.3

38.5

Overall memory

86.1

70.2

40.3

Visuomotor sub-tests

Adaptive motor control

74.2

67.6

60.0

Visual salience

76.7

69.9

58.3

Perception sub-tests

Letter discrimination

77.9

61.7

63.2

Word discrimination

84.6

60.9

30.2

Shape discrimination

91.2

66.4

37.5

Motion discrimination

81.9

61.6

24.3

Memory sub-tests

Letter memory

85.2

70.6

38.0

Word memory

93.9

71.8

38.8

Shape memory

84.2

66.3

35.2

Motion memory

82.3

73.3

50.5

CI: cognitive impairment

A statistically significant negative correlation (-0.54) was demonstrated between Cognivue® total score and the EDSS (Figure 1), and between Cognivue® total score and the T25W (-0.56) (Table 5). A statistically significant positive correlation (0.67) was demonstrated between Cognivue® total score and SDMT. The degree of correlation between Cognivue® perception and memory sub-batteries and the other neuropsychological tests are shown in Table 5.

EDSS: Expanded Disability Status Scale.

Table 5: Correlation between Cognivue® and other tests.

Test

Overall
Cognivue® score

Overall
perception score

Overall
memory score

EDSS

-0.54*

-0.52*

-0.47

SDMT

0.67*

0.72

0.53*

NHPT

-0.47

-0.40

-0.48

T25W

-0.56*

-0.59*

-0.45

*Moderate level of correlation; High level of correlation; EDSS: Expanded Disability Status Scale;
NHPT: Nine-Hole Peg Test; SDMT: Symbol Digit Modality Test; T25W, Timed 25-foot walk.

The highest positive correlation (0.72) in this study was between overall scores on the Cognivue® perception sub-battery and SDMT scores (Figure 2).

SDMT: Symbol Digit Modality Test.

Correlation coefficients were also calculated for scores on the EDSS and the individual sub-tests of Cognivue® (Table 6). Seven of the 10 sub-tests were significantly correlated with EDSS (p<0.05). No correlations were found between EDSS score and the adaptive motor control (-0.31), motion discrimination (-0.29), and motion memory (-0.25) sub-tests of Cognivue®.

Table 6: Correlation between Cognivue® sub-tests and EDSS.

Cognivue® sub-test

Correlation coefficient, r

Adaptive motor control

-0.31

Visual salience

-0.39*

Letter discrimination

-0.41*

Word discrimination

-0.55*

Shape discrimination

-0.51*

Motion discrimination

-0.29

Letter memory

-0.54*

Word memory

-0.36*

Shape memory

-0.37*

Motion memory

-0.25

*p<0.05; EDSS: Expanded Disability Status Scale.

No significant associations were found between Cognivue® total score and age (Figure 3) or Cognivue® total score and sex (data not shown).

Discussion

Impairments in cognitive functions result in substantial burden for patients with MS overall, with delayed processing speed being the predominant factor influencing both quality of life and participation in activities of daily living [21]. Studies have described the negative impact of impaired cognition on various domains and activities of daily living (eg, work, school, and recreation participation), as well as driving, medical decision-making, medication adherence, and money management [further reviewed in reference 2]). Additionally, a greater level of CI in patients with MS has been found to be predictive of future work disability [22]. There is evidence that early CI is associated with greater decline [23-25]. For this reason, screening guidelines from the National Multiple Sclerosis Society recommend early detection to inform strategies to improve CI and provide patients with compensatory strategies to maximize functioning [2]. Patient education regarding cognitive symptoms and the necessity of ongoing assessment, as well as remedial interventions to address the specific cognitive deficits identified and optimize performance of activities of daily living are recommended [2].

In line with the understanding of the pathological processes in MS, our study demonstrated that the possible impact of the disease on cognition is global, it affects all spheres and its severity correlates with the overall aggressiveness of the disease. Accurately determining and regularly monitoring over time both the number of and the specific subdomains have clinical relevance and Cognivue® provided a picture of the nature of the CI, with its overall and key domain subscores. In our cohort of MS patients, Cognivue® correlated well with established testing methods-the EDSS and SDMT-for the assessment of CI, with all key cognitive domains being affected. The statistically significant positive correlation (0.67) between Cognivue® total score and SDMT score confirms the results of an earlier pilot study by Smith III AD, et al. (2018) [8]. Statistically significant negative correlations between Cognivue® total score and EDSS score (-0.54) as well as between Cognivue® total score and T25W score (-0.56) were also demonstrated in our investigation.

These findings are corroborated by the results of the aforementioned pilot study that examined the ability of Cognivue® to identify CI among patients (n=24) with MS as compared to healthy controls (n=12) [8]. Participants completed the EDSS at the initial study visit, followed by other tests including the Paced Auditory Serial Addition Test (PASAT), the SDMT, NHPT, T25W, and Cognivue® between 1 and 3 times, with 1-2 months between sessions. There were significant differences in average Cognivue® total scores among those with MS compared with healthy controls (80.71 vs. 93.00; p=0.001). Significant differences between study groups in mean scores on the SDMT (45.67 vs. 61.67; p=0.0005), PASAT (38.75 vs. 50.42; p=0.05), NHPT (26.47 vs. 18.17; p=0.001), and the T25W (4.62 vs. 3.50; p=0.01) were also found. Additionally, multiple Cognivue® sub-tests showed significant differences between those with MS and controls [8].

Many traditional testing methods assess multiple cognitive domains, but typically only provide a single score. This limits their ability to capture the heterogeneous nature of CI in patients with MS. For example, a 7-year retrospective follow-up study showed that poorer visual memory and attention/information processing was associated with relapse risk and greater motor disability [26] -such delineation would not be captured if only a single score is generated and recorded in a clinical setting. Furthermore, the actual number of impaired cognitive subdomains is also relevant as it has been shown to be significantly associated with progressive disease [4] and long-term clinical and cognitive deterioration [27]. This underscores the need for tools allowing for the assessment of specific cognitive subdomains, which might be better suited than a single overall measure of CI for predicting clinical outcomes and more effectively tailoring cognitive rehabilitation strategies [28] Thus in recent years, there has been an incentive to supplement conventional neuropsychology testing with computerized assessment devices, whose reliability and validity were reviewed by Wojcik CM, et al. (2019) [29].

Moreover, studies have shown worsening of cognitive function during MS relapses, which may not be apparent in patient-reported outcomes [30] or self-reported cognitive complaints which can be confounded by other subjective symptoms [31]. Therefore, a computerized testing method that is more sensitive to reaction time and response speed, may allow for better detection of MS-related CI [29].

Computerized testing methods can help overcome the time and resource demands-key barriers to the routine testing and monitoring of cognitive function-and solidify consistent neuropsychological assessment as part of the standard of care for patients with MS [28]. However, computer-based tools that require a technician can be labor intensive [32] and in fact, patients with MS have indicated a preference for a self-administered, computerized process [33].

Although the current study is limited by its small sample size, the data generated are in line with previous observations and Cognivue® was shown to reliably detect early stages of multi-domain CI in patients with MS, providing a potential opportunity for the implementation of early intervention strategies to improve patient outcomes. Cognivue® provides a validated cognitive assessment tool [16] that can be practically incorporated into the routine management of patients with MS.

References

  1. Foley FW, Benedict RH, Gromisch ES, Deluca J (2012) The need for screening, assessment, and treatment for cognitive dysfunction in multiple sclerosis: Results of a multidisciplinary CMSC consensus conference. Int J MS Care 14: 58-64. https://doi.org/10.7224/1537-2073-14.2.58
  2. Kalb R, Beier M, Benedict RH, Charvet L, Costello K, et al. (2018) Recommendations for cognitive screening and management in multiple sclerosis care. Mult Scler 24: 1665-1680. https://doi.org/10.1177/1352458518803785
  3. Macias Islas MA, Ciampi E (2019) Assessment and impact of cognitive impairment in multiple sclerosis: An overview. Biomedicines 7: 22. https://doi.org/10.3390/biomedicines7010022
  4. Renner A, Baetge SJ, Filser M, Ullrich S, Lassek C, et al. (2020) Characterizing cognitive deficits and potential predictors in multiple sclerosis: A large nationwide study applying brief international cognitive assessment for multiple sclerosis in standard clinical care. J Neuropsychol 14: 347-369. https://doi.org/10.1111/jnp.12202
  5. Planche V, Gibelin M, Cregut D, Pereira B, Clavelou P (2016) Cognitive impairment in a population-based study of patients with multiple sclerosis: differences between late relapsing-remitting, secondary progressive and primary progressive multiple sclerosis. Eur J Neurol 23: 282-289. https://doi.org/10.1111/ene.12715
  6. Ruano L, Portaccio E, Goretti B, Niccolai C, Severo M, et al. (2017) Age and disability drive cognitive impairment in multiple sclerosis across disease subtypes. Mult Scler 23: 1258-1267. https://doi.org/10.1177/1352458516674367
  7. Branco M, Ruano L, Portaccio E, Goretti B, Niccolai C, et al. (2019) Aging with multiple sclerosis: prevalence and profile of cognitive impairment. Neurol Sci 40: 1651-1657. https://doi.org/10.1007/s10072-019-03875-7
  8. Smith III AD, Duffy C, Goodman AD (2018) Novel computer-based testing shows multi-domain cognitive dysfunction in patients with multiple sclerosis. Mult Scler J Exp Transl Clin 4: 1-9. https://doi.org/10.1177/2055217318767458
  9. Ranson JM, Ku?ma E, Hamilton W, Muniz-Terrera G, Langa KM, et al. (2019) Predictors of dementia misclassification when using brief cognitive assessments. Neurol Clin Pract 9: 1-9. https://doi.org/10.1212/CPJ.0000000000000566
  10. Cordell CB, Borson S, Boustani M, Chodosh J, Reuben D, et al. (2013) Alzheimer's Association recommendations for operationalizing the detection of cognitive impairment during the Medicare Annual Wellness Visit in a primary care setting. Alzheimers Dement 9: 141-150. https://doi.org/10.1016/j.jalz.2012.09.011
  11. Zadikoff C, Fox SH, Tang?Wai DF, Thomsen T, De Bie RM, et al. (2008) A comparison of the mini mental state exam to the Montreal cognitive assessment in identifying cognitive deficits in Parkinson's disease. Mov Disord 23: 297-299. https://doi.org/10.1002/mds.21837
  12. Athilingam P, Visovsky C, Elliott AF, Rogal PJ (2015) Cognitive screening in persons with chronic diseases in primary care: challenges and recommendations for practice. Am J Alzheimers Dis Other Dement 30: 547-558. https://doi.org/10.1177/1533317515577127
  13. Connor DJ, Jenkins CW, Carpenter D, Crean R, Perera P (2018) Detection of rater errors on cognitive instruments in a clinical trial setting. J Prev Alzheimers Dis 5: 188-196. https://doi.org/10.14283/jpad.2018.20
  14. Collie A, Darby D, Maruff P (2001) Computerised cognitive assessment of athletes with sports related head injury. Br J Sports Med 35: 297-302. http://dx.doi.org/10.1136/bjsm.35.5.297
  15. US Food and Drug Administration (2013) De Novo Summary (DEN130033) for Cognivue. United States.
  16. Cahn-Hidalgo D, Estes PW, Benabou R (2020) Validity, reliability, and psychometric properties of a computerized, cognitive assessment test (Cognivue®). World J Psychiatry 10: 1-11. https://dx.doi.org/10.5498/wjp.v10.i1.1
  17. Kurtzke JF (1983) Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology 33: 1444-1452. https://doi.org/10.1212/WNL.33.11.1444
  18. Benedict RH, DeLuca J, Phillips G, LaRocca N, Hudson LD, et al. (2017) Validity of the symbol digit modalities test as a cognition performance outcome measure for multiple sclerosis. Mult Scler 23: 721-733. https://doi.org/10.1177/1352458517690821
  19. Feys P, Lamers I, Francis G, Benedict R, Phillips G, et al. (2017) The nine-hole peg test as a manual dexterity performance measure for multiple sclerosis. Mult Scler 23: 711-720. https://doi.org/10.1177/1352458517690824
  20. Motl RW, Cohen JA, Benedict R, Phillips G, LaRocca N, et al. (2017) Validity of the timed 25-foot walk as an ambulatory performance outcome measure for multiple sclerosis. Mult Scler 23: 704-710. https://doi.org/10.1177/1352458517690823
  21. Goverover Y, Strober L, Chiaravalloti N, DeLuca J (2015) Factors that moderate activity limitation and participation restriction in people with multiple sclerosis. Am J Occup Ther 69: 1-9. https://doi.org/10.5014/ajot.2015.014332
  22. Kavaliunas A, Tinghög P, Friberg E, Olsson T, Alexanderson K, et al. (2019) Cognitive function predicts work disability among multiple sclerosis patients. Mult Scler J Exp Transl Clin 5: 2055217318822134. https://doi.org/10.1177/2055217318822134
  23. Amato MP, Ponziani G, Siracusa G, Sorbi S (2001) Cognitive dysfunction in early-onset multiple sclerosis: a reappraisal after 10 years. Arch Neurol 58: 1602-1606. https://doi.org/10.1001/archneur.58.10.1602
  24. Amato MP, Portaccio E, Goretti B, Zipoli V, Iudice A, et al. (2010) Relevance of cognitive deterioration in early relapsing-remitting MS: a 3-year follow-up study. Mult Scler 16: 1474-1482. https://doi.org/10.1177/1352458510380089
  25. Amato MP, Goretti B, Ghezzi A, Hakiki B, Niccolai C, et al. (2014) Neuropsychological features in childhood and juvenile multiple sclerosis: five-year follow-up. Neurology 83: 1432-1438. https://doi.org/10.1212/WNL.0000000000000885
  26. Carotenuto A, Moccia M, Costabile T, Signoriello E, Paolicelli D, et al. (2019) Associations between cognitive impairment at onset and disability accrual in young people with multiple sclerosis. Sci Rep 9: 18074. https://doi.org/10.1038/s41598-019-54153-7
  27. Damasceno A, Pimentel-Silva LR, Damasceno BP, Cendes F (2019) Cognitive trajectories in relapsing-remitting multiple sclerosis: A longitudinal 6-year study. Mult Scler 26: 1740-1751. https://doi.org/10.1177/1352458519878685
  28. Sumowski JF, Benedict R, Enzinger C, Filippi M, Geurts JJ, et al. (2018) Cognition in multiple sclerosis: State of the field and priorities for the future. Neurology 90: 278-288. https://doi.org/10.1212/WNL.0000000000004977
  29. Wojcik CM, Beier M, Costello K, DeLuca J, Feinstein A, et al. (2019) Computerized neuropsychological assessment devices in multiple sclerosis: A systematic review. Mult Scler 25: 1848-1869. https://doi.org/10.1177/1352458519879094
  30. Benedict RH, Pol J, Yasin F, et al. (2020) Recovery of cognitive function after relapse in multiple sclerosis. Mult Scler 2020: 1352458519898108.https://doi.org/10.1177/1352458519898108
  31. Oreja-Guevara C, Ayuso T, Brieva L, Hernández MÁ, Meca-Lallana V, et al. (2019) Cognitive dysfunctions and assessments in multiple sclerosis. Front Neurol 10: 581. https://doi.org/10.3389/fneur.2019.00581
  32. Lapshin H, Lanctot KL, O'Connor P, Feinstein A (2013) Assessing the validity of a computer-generated cognitive screening instrument for patients with multiple sclerosis. Mult Scler 19: 1905-1912. https://doi.org/10.1177/1352458513488841
  33. Patel VP, Shen L, Rose J, Feinstein A (2019) Taking the tester out of the SDMT: A proof of concept fully automated approach to assessing processing speed in people with MS. Mult Scler 25: 1506-1513. https://doi.org/10.1177/1352458518792772
scroll up