Braz J Psychiatry. 2020 May-Jun;42(3):286-294 doi:10.1590/1516-4446-2019-0539 Brazilian Psychiatric Association 0 0 0 0 -0 02-7316-1 85 ORIGINAL ARTICLE Brazilian version of the European Cross-Cultural Neuropsychological Test Battery (CNTB-BR): diagnostic accuracy across schooling levels Narahyana B. Araujo,10 0 -0 0 -0 0 -0 0 Thomas R. Nielsen,2 Maria L. Barca,3,4 Knut Engedal,3,4 Valeska Marinho,1 Andrea C. Deslandes,1 Evandro S. Coutinho,5 Jerson Laks1,6 1 Instituto de Psiquiatria (IPUB), Universidade Federal do Rio de Janeiro (UFRJ), Rio de Janeiro, RJ, Brazil. 2Danish Dementia Research Centre, University of Copenhagen, Copenhagen, Denmark. 3Norwegian National Unit for Aging and Health, Vestfold County Hospital, Toensberg, Norway. 4Department of Geriatric Medicine, Oslo University Hospital, Norway. 5Escola Nacional de Saúde Pública (ENSP), Fundac¸ão Oswaldo Cruz (FIOCRUZ), Rio de Janeiro, RJ, Brazil. 6Programa de Pós-Graduac¸ão em Biomedicina Translacional (Biotrans), Universidade do Grande Rio (Unigranrio), Duque de Caxias, RJ, Brazil. Objective: To translate, establish the diagnostic accuracy, and standardize the Brazilian Portuguese version of the European Cross-Cultural Neuropsychological Test Battery (CNTB) considering schooling level. Methods: We first completed an English-Brazilian Portuguese translation and back-translation of the CNTB. A total of 135 subjects aged over 60 years – 65 cognitively healthy (mean 72.83, SD = 7.71; mean education 9.42, SD = 7.69; illiterate = 25.8%) and 70 with Alzheimer’s disease (AD) (mean 78.87, SD = 7.09; mean education 7.62, SD = 5.13; illiterate = 10%) – completed an interview and were screened for depression. The receiver operating characteristic (ROC) analysis was used to verify the accuracy of each CNTB test to separate AD from healthy controls in participants with low levels of education (p 4 years of schooling) and high levels of education (X 8 years of schooling). The optimal cutoff score was determined for each test. Results: The Recall of Pictures Test (RPT)-delayed recall and the Enhanced Cued Recall (ECR) had the highest power to separate AD from controls. The tests with the least impact from schooling were the Rowland Universal Dementia Assessment Scale (RUDAS), supermarket fluency, RPT naming, delayed recall and recognition, and ECR. Conclusions: The Brazilian Portuguese version of the CNTB was well comprehended by the participants. The cognitive tests that best discriminated patients with AD from controls in lower and higher schooling participants were RPT delayed recall and ECR, both of which evaluate memory. Keywords: Alzheimer’s disease; education; psychometric tests; diagnostic accuracy Introduction Nearly 17% of the world’s adult population is illiterate.1 Among older adults, illiteracy and low levels of education are more common in lower and middle income countries than in developed countries1 – data from 105 less developed countries for which information is available show that 43% of the population aged 60 or older were illiterate in 2010. In some developed countries, however, the illiteracy rates are also high in old adults.1 Brazil, the country with both the largest territory and population in South America, is no exception – the country faces low levels of education, currently associated with an exponential growth in the population of older adults. According to data from the Brazilian Institute of Geography Correspondence: Narahyana B. Araujo, Rua Conde de Bonfim, 1357/504, Tijuca, CEP 20530-001, Rio de Janeiro, RJ, Brazil. E-mail: narahyana@hotmail.com Submitted May 03 2019, accepted Oct 14 2019, Epub Feb 21 2020. and Statistics (Instituto Brasileiro de Geografia e Estatı́stica [IBGE]), 23% of the nearly 29.9 million people above age 60 years are illiterate in Brazil.2 Nevertheless, few validated instruments are available in Brazilian Portuguese for cognitive assessment of people with less than 9 years of education.3,4 Several publications in Brazil have investigated the impact of education on the subject’s performance in cognitive testing. Cognitive batteries for the detection of mild dementia, such as the Mini Mental State Examination (MMSE),3 Addenbrooke’s Cognitive Examination-Revised (ACE-R),5 and Montreal Cognitive Assessment (MoCA),6 have shown a significant correlation between the total test score and education. The influence of education on performance has also been evaluated in the Cambridge How to cite this article: Araujo NB, Nielsen TR, Barca ML, Engedal K, Marinho V, Deslandes AC, et al. Brazilian version of the European Cross-Cultural Neuropsychological Test Battery (CNTB-BR): diagnostic accuracy across schooling levels. Braz J Psychiatry. 2020; 42:286-294. http://dx.doi.org/10.1590/1516-4446-2019-0539 Diagnostic accuracy of CNTB-BR Cognitive Examination (CAMCOG),7-9 the Alzheimer’s Disease Assessment Scale cognitive subscale (ADASCog),10 and the Consortium to Establish a Registry for Alzheimer’s Disease (CERAD).11 More recently, the Danish Dementia Research Center designed the European Cross-Cultural Neuropsychological Test Battery (CNTB) – a battery of tests to assess cognitive impairment in dementia disorders across ethnic groups and in people with limited or no education.12 The CNTB provides scales that are less influenced by linguistic, cultural, and educational factors12 with crosscultural diagnostic properties that are useful for evaluating dementia in minority populations. Thus, the CNTB, developed to minimize the influence of culture, language, and educational level,13 may represent a valid alternative to other neuropsychological batteries.14 Our group has recently validated a Brazilian Portuguese version of the Rowland Universal Dementia Assessment Scale (RUDAS),15 showing that it is accurate and valid to assess neuropsychological performance regardless of the person’s educational level. However, the availability of methods for accurate assessment of multidomain cognitive functioning in illiterate and older adults with low levels of education is becoming increasingly important in Brazil. In order to bridge this gap, we aimed to translate, verify the diagnostic accuracy, and standardize the Brazilian Portuguese version of the CNTB considering educational level. Methods Participants 287 diseases, who were cognitively healthy according to clinical evaluation and MMSE. The cutoff used was 13 points for illiterate participants, 18 points for those with elementary or middle schooling (o 8 years), and 26 points for those with schooling (X 8 years).19 Neurological or psychiatric diseases were evaluated during the clinical assessment. Imaging and laboratory evaluations were required of all patients. All participants regular patients at the aging research center, private hospital, military hospital or university center, and had been diagnosed with AD or confirmed as cognitively healthy. They were evaluated again before being referred to this study according to the inclusion and exclusion criteria. Because all those referred agreed to participants, all participants were included in the final sample of this study. The low levels of schooling group included illiterate participants with up to 4 years of schooling. The high levels of schooling group included participants with 8 or more years of schooling. Procedures This study was conducted from 2014 to 2017. All participants completed a session lasting approximately 90 minutes, including a structured demographic and medical interview, as well as screening for depression with the Geriatric Depression Scale (GDS-15)20 and the CNTB. Measures 15 Detailed data about participants are provided elsewhere. In summary, participants were selected if they met the inclusion criterion of at least 60 years of age. Participants with and without dementia due to Alzheimer’s disease (AD) were included. Exclusion criteria included neurological comorbidities, psychiatric clinical disorders, and visual or hearing impairment that precluded cognitive testing. Subjects with depressive symptoms who did not fulfill clinical criteria for a depressive disorder were not excluded. Patients with AD were recruited from a university center and a military hospital. Clinical diagnosis of AD according to the DSM-IV16 and the National Institute of Neurological and Communicative Diseases and Stroke/Alzheimer’s Disease and Related Disorders Association (NINCDSADRDA)17 was required, with mild or moderate severity according to the Clinical Dementia Rating (CDR).18 The diagnostic workup for AD patients included complete blood cell count, platelet count, glycemia, triglycerides, total cholesterol and fractions, alkaline phosphatase, glutamic oxaloacetic transaminase and glutamic pyruvic transaminase, bilirubins, urea, creatinine, total proteins, calcium, free T4 levels, thyroid-stimulating hormone (TSH) levels, venereal disease research laboratory (VDRL) test, and dosage of B12 and folates. Cranial computed tomography scan or magnetic resonance imaging scan with or without spectroscopy were also performed. The control group was recruited from a research center on aging, a private hospital, and a military hospital, and included individuals without neurological or psychiatric Demographic and medical interview The interview consisted of questions about demographic data (age, sex, marital status, schooling in years), physical and mental health status, and self-perceived general health and memory. Geriatric Depression Scale (GDS-15) The GDS-15 is a self-report instrument with a dichotomous response format (yes/no), commonly used for clinical work and research. The 15-item version was used in this study. The cutoff used was 6 points.20 European Cross-Cultural Neuropsychological Test Battery (CNTB) The CNTB covers the cognitive domains of global cognitive function: memory, language, executive functions, and visuospatial functions. Only the RUDAS included printed instructions that required translation and back-translation; a Brazilian Portuguese version validated by Araujo et al.15 was used. All other tests were based on oral rather than printed instructions and presented in pictures rather than verbal stimuli.12 The experimental version of the CNTB was tested for comprehensibility in four subjects: two healthy older adults (one with p 4 years of schooling and one with X 8 years of schooling) and two older adults with AD (one with p 4 years of schooling and one with X 8 years of schooling). Braz J Psychiatry. 2020;42(3) 288 NB Araujo et al. Based on this pilot study, it was determined that the CNTB demonstrated adequate comprehensibility and no changes to the original items of the CNTB were necessary. Global cognitive function RUDAS is a brief cognitive instrument that contains six items, testing memory (registration and recall), body orientation, praxis, drawing, judgment, and language, adding up to a total score of 30. Lower scores indicate poorer cognitive function.21 Memory The Recall of Pictures Test (RPT) was developed to assess immediate and delayed recall. The test is similar to a test from the Brief Cognitive Screening Battery (BCSB),22 except that color pictures are used. Subjects are required to learn and recall 10 different pictures. Participants were shown a sheet with 10 pictures of common objects that had to be named, and immediately thereafter recalled by the subjects (incidental recall). The pictures were shown for 30 seconds two additional times and participants were asked to memorize and recall the objects (immediate recall). After a 10-minute interference interval, in which subjects performed other tests, delayed recall of the objects in the pictures was requested, followed by a recognition trial in which the 10 pictures had to be recognized among 10 distracters.12,23,24 In the Enhanced Cued Recall (ECR), 16 pictures are shown, divided into four cards. When the cards are presented, a semantic cue is given for each picture, and the participant is asked to identify the picture on the card that best fits with the cue. Then, an immediate recall of the four pictures is tested with the same semantic cue for each picture. After presentation of all four cards, participants are required to name the month backward or count backward from 20, immediately followed by a free and cued recall trial. Total score of free and cued recall is used with a range of scores of 0-16 points.12 A semicomplex figure is also given, consisting of 11 elements. Participants are required to copy and recall the figure after a 3-minute interference interval.12 This is scored according to the Meyers & Meyers,25 with a score range of 0-22 points. Language Picture naming including the number of pictures correctly named at the initial presentation of the colored pictures of common objects in the RPT was recorded, for a score ranging from 0-10 points.12 Animal verbal fluency (VF) is a 1-minute assessment in which participants are asked to name as many animals as they remember.26,27 Repeated words are not counted towards the final score. Supermarket fluency (SF) is a 1-minute test in which subjects are required to generate as many different ‘‘things you can buy in a supermarket’’ as possible. The score is equivalent to the number of items produced in 1 minute.12,23,24 Braz J Psychiatry. 2020;42(3) Executive functions The Color Trails Test (CTT) is similar to Trail Making Test (TMT).27 The test has two parts: in CTT 1, participants are required to connect numbered circles in ascending order, and in CTT 2, participants are required to switch between the pink and yellow colors while connecting circles in an ascending sequence.12,28 The CTT 2 was applied only to participants with 5 years or more of schooling. The Five Digit Test (FDT)29 is similar to Stroop Color test, but designed to minimize the need for schooling. Participants are exposed to four conditions: 1) naming a series of 50 digits; 2) counting a series of 50 asterisks; 3) counting a series of 50 digits in which the numeric value of the digits is incongruent with the number of digits; and 4) switching between counting and naming incongruent digits. The score is the time in seconds required to complete each condition in the test.30 Serial threes is similar to the serial sevens subtest from the MMSE. Participants are required to count down from 20 by threes.12 Visuospatial functions In copying of simple figures, participants are required to copy a cross and a four-pointed star according to a 0-3 point scale with a total copying score of 0-6 points.12 The Clock Drawing Test (CDT) was administered using a pre-drawn circle. Participants are required to add numbers and indicate the time 11:10. The CDT was scored according to the Shulman criteria (0-5 points).31 In the Clock Reading Test (CRT), the subjects were required to read the time on a series of 12 clocks showing different times. The clocks had no digits around the dial. One point was given for each correct reading and to a reading less than 4 minutes off the correct time. A half point was given to readings that were 4 to 5 minutes or exactly 1 hour off the correct time. The total score was the sum of the 12 clock readings.12,23,24 Statistical analysis Descriptive statistics were calculated for demographic and medical characteristics of the groups. Participants were divided into an AD and a control group according to schooling levels – low schooling (p 4 years) or high schooling (X 8 years). Receiver operating characteristic (ROC) curve analysis was used to verify the accuracy of each CNTB test to detect cognitive impairment in AD. Sensitivity, specificity, positive likelihood ratio (LR+), and negative LR (LR-) were calculated. The LR+ was calculated as the ratio between the proportion of true-positives and the proportion of false-positives. The LR- was obtained by dividing the proportion of false-negatives by the proportion of truenegatives. Values greater than 1 are desirable for LR+, and values lower than 0 are appropriate for LR-. The optimal cutoff score was determined for each test by balancing sensitivity and specificity according to Youden index score. Differences with p-values p 0.05 were considered statistically significant. All analyses were conducted using Stata14. Diagnostic accuracy of CNTB-BR Ethics statement This study was approved by the ethics committee of the Instituto de Psiquiatria, Universidade Federal do Rio de Janeiro, and of the Hospital Naval Marcı́lio Dias, Brazil, and all participants provided written informed consent before any procedure took place. Results The final sample included 135 older adults (age X 60 years), of which 70 had AD. Table 1 shows clinical and sociodemographic data. Patients in the AD group were significantly older, but there was no statistically significant difference in schooling between the groups. In the control group, 25.8% were illiterate, and in the AD group, 10% were illiterate. While most participants in the control group were widowed, most participants in the AD group were married. Also, most participants in both groups considered 289 their overall health as good and considered their memory as ‘‘fairly good.’’ In the AD group, 80% of the sample had mild AD. Partial results from the RUDAS-BR study have been presented in a previous publication.15 For the present study, results appear in Tables 2, 3 and 4. In SF, RPT-picture naming, RPT-delayed recall, and ECR, there is little variation in the optimal cutoff between the different educational levels. The diagnostic accuracy of these tests, i.e., the ability to correctly classify cognitive impairment in AD, did not significantly differ between the low and high schooling groups. The results show lower impact of education on these scales, which is very important for cognitive evaluation in elderly multicultural patient populations with different educational levels. Conversely, the optimal cutoff for diagnostic accuracy using the VF, CTT1, FDT, CDT, CRT, and copying of simple figures, i.e., tests that evaluate executive function and visuoconstruction ability, does substantially vary due Table 1 Sociodemographic and clinical characteristics of the sample Control (n=65) AD (n=70) p-value 72.8367.71 90.77 78.8767.09 56.34 o 0.001 o 0.001* 37 31 32 8 59 33 9.4267.69 25.8 7.6265.13 10 Self-reported health Very bad Bad Fair Good Very good 1.50 9.20 36.90 41.50 10.80 1.40 2.80 33.80 53.50 8.50 Self-reported memory Very bad Bad Fair Good Very good 0 1.30 44.60 35.40 7.70 5.60 22.50 46.50 23.90 1.40 100 0 0 0 80 20 Diabetes Hypertension Depression 20.00 52.31 15.38 30.43 52.17 27.54 Smoking status Never smoked Ex-smoker Current smoker 59.40 37.50 3.10 41.40 52.90 5.70 0 2.5262.56 4.29 2.5862.98 Age in years, mean 6 SD Female sex Marital status Single/divorced Married Widower Schooling in years, mean 6 SD Illiterate CDR 0 1 2 Alcohol use GDS-15 score, mean 6 SD o 0.001* 0.109 o 0.001* 0.45* 0.04* o 0.001* 0.16* 0.98* 0.88* 0.11* 0.97* 0.893 Data presented as %, unless otherwise specified. CDR = Clinical Dementia Rating; D = Alzheimer’s disease; GDS = Geriatric Depression Scale; SD = standard deviation. * w2 test. Braz J Psychiatry. 2020;42(3) 290 NB Araujo et al. Table 2 Measures based on ROC curve analysis in the total sample AUC (95%CI) Optimal cutoff point Sensitivity (%) Specificity (%) Correctly classified (%) LR+ LR- General cognitive function RUDAS (n=135) 0.87 (0.82-0.93) p 23/30 81.54 76.06 78.68 3.41 0.24 Memory RPT-immediate learning (n=135) RPT-delayed recall (n=135) RPT-recognition (n=135) ECR (n=134) 0.93 0.93 0.79 0.93 (0.89-0.97) (0.89-0.98) (0.72-0.86) (0.88-0.97) p 18 p5 p 10 p 12 81.54 90.77 86.15 89.23 87.14 91.43 70.00 84.06 84.44 91.11 77.78 86.57 6.34 10.59 2.87 5.59 0.21 0.10 0.19 0.13 Language Picture naming (n=135) VF (n=135) SF (n=135) 0.59 (0.53-0.66) 0.76 (0.68-0.84) 0.89 (0.85-0.94) p 10 p 13 p 14 90.77 66.15 78.46 27.14 77.46 78.57 57.78 72.00 78.52 1.25 2.9 3.66 0.34 0.43 0.27 Executive functions CTT 1 (n=120) FDT 1 (n=121) FDT 2 (n=119) FDT 3 (n=120) FDT 4 (n=116) Serial threes (n=134) 0.74 0.66 0.61 0.63 0.63 0.61 (0.65-0.83) (0.56-0.76) (0.51-0.71) (0.53-0.73) (0.53-0.73) (0.52-0.70) p 99 p 36 p 38 p 59 p 82 p6 80.73 63.16 60.94 64.29 57.41 59.38 65.50 60.94 59.66 64.06 58.06 62.86 72.50 61.98 58.18 64.17 57.76 61.19 2.31 1.62 1.49 1.79 1.37 1.59 0.29 0.60 0.69 0.56 0.73 0.65 Visuospatial function Copying of simple figures (n=134) Semicomplex figure copy (n=98) CDT (n=133) 0.72 (0.63- 0.80) 0.73 (0.63- 0.83) 0.82 (0.75-0.89) p 05 p 20 p 04 60.94 69.57 71.88 73.61 73.08 86.96 67.65 71.43 79.70 2.30 2.58 5.5 0.53 0.42 0.32 Measure 95%CI = 95% confidence interval; AUC = area under the receiver operating characteristic (ROC) curve; CDT = Clock Drawing Test; CRT = Clock Reading Test; CTT = Color Trails Test; ECR = Enhanced Cued Recall; FDT = Five Digit Test; LR- = negative likelihood ratio; LR+ = positive likelihood ratio; n = sample; RPT = Recall of Pictures Test; RUDAS = Rowland Universal Dementia Assessment Scale; SF = supermarket fluency; VF = verbal fluency (animal). Table 3 Measures based on ROC curve analysis in low schooling participants AUC (95%CI) Optimal cutoff point Sensitivity (%) Specificity (%) Correctly classified (%) LR+ LR- General cognitive function RUDAS (n=69) 0.82 (0.72-0.91) p 23 67.74 79.00 73.91 3.21 0.41 Memory RPT-immediate learning (n=69) RPT-delayed recall (n=69) RPT-recognition (n=6) ECR (n=6) 0.88 0.87 0.76 0.88 (0.81-0.96) (0.78-0.96) (0.64-0.87) (0.79-0.96) p 16 p4 p 10 p 12 80.65 87.10 77.42 80.65 78.95 84.21 73.68 83.78 79.71 85.51 75.36 82.35 3.83 5.51 2.94 4.97 0.25 0.15 0.31 0.23 Language Picture naming (n=69) VF (n=69) SF (n=69) 0.58 (0.49-0.68) 0.67 (0.54-0.80) 0.83 (0.75-0.93) N/S p 12 p 14 54.84 74.19 73.68 78.95 65.22 76.81 2.08 3.52 0.61 0.33 Executive functions CTT 1 (n=58) FDT 1 (n=61) FDT 2 (n=60) FDT 3 (n=60) FDT 4 (n=57) Serial threes (n=68) 0.56 0.55 0.50 0.43 0.48 0.45 (0.40-0.71) (0.40-0.69) (0.35-0.65) (0.29-0.58) (0.32-0.63) (0.32-0.59) N/S N/S N/S N/S N/S N/S Visuospatial function Simple copying test-cross (n=69) Semicomplex figure copy (n=36) CDT (n=67) CRT (N=67) 0.64 0.48 0.75 0.61 (0.51-0.77) (0.28-0.68) (0.64-0.86) (0.48-0.75) p 04 N/S p 03 p4 51.61 73.68 63.77 1.96 0.66 56.67 61.29 83.78 55.56 71.64 58.21 3.49 1.38 0.51 0.69 Measure Data presented as %, unless otherwise specified. 95%CI = 95% confidence interval; AUC = area under the receiver operating characteristic (ROC) curve; CDT = Clock Drawing Test; CRT = Clock Reading Test; CTT = Color Trails Test; ECR = Enhanced Cued Recall; FDT = Five Digit Test; LR- = negative likelihood ratio; LR+ = positive likelihood ratio; n = sample; N/S = not significant; RPT = Recall of Pictures Test; RUDAS = Rowland Universal Dementia Assessment Scale; SF = supermarket fluency; VF = verbal fluency (animal). Braz J Psychiatry. 2020;42(3) Diagnostic accuracy of CNTB-BR 291 Table 4 Measures based on ROC curve analysis in high schooling participants AUC (95%CI) Optimal cutoff point Sensitivity (%) Specificity (%) Correctly classified (%) LR+ LR- General cognitive function RUDAS (n=67) 0.92 (0.86-0.99) p 24 91.18 81.82 86.57 5.00 0.11 Memory RPT-immediate learning (n=66) RPT-delayed recall (n=66) RPT-recognition (n=66) ECR (n=66) Semicomplex figure recall (n=62) 0.98 0.99 0.81 0.97 0.99 (0.94-1.00) (0.99-1.00) (0.72-0.90) (0.95-1.00) (0.97-1.00) p 21 p5 p 10 p 12 p 12 91.18 100.00 94.12 97.06 92.86 100.00 93.75 65.63 84.38 93.33 95.45 96.97 80.30 90.91 93.10 29.18 16.00 2.74 6.21 13.93 0.08 0.00 0.08 0.03 0.08 Language Picture naming (n=66) VF (n=67) SF (n=66) 0.59 (0.52-0.67) 0.82 (0.72-0.92) 0.95 (0.89-0.99) p 10 p 14 p 15 97.06 82.35 82.35 21.88 72.73 81.25 60.61 77.61 81.82 1.24 3.02 4.39 0.13 0.24 0.22 Executive functions CTT 1 (n=62) CTT 2 (n=53) FDT 1 (n=60) FDT 2 (n=59) FDT 3 (n=60) FDT 4 (n=59) Serial threes (n=66) 0.89 0.84 0.73 0.69 0.80 0.73 0.76 (0.81-0.98) (0.71-0.96) (0.59-0.87) (0.54-0.84) (0.69-0.91) (0.59-0.87) (0.66-0.87) p 95 p 141 p 32 p 32 p 57 p 75 p6 78.57 80.00 65.38 72.00 61.54 64.00 85.29 88.24 78.79 79.41 61.76 85.29 64.71 62.50 83.87 79.25 73.33 66.10 75.00 64.41 74.24 6.68 3.77 3.18 1.88 4.18 1.81 2.27 0.24 0.25 0.44 0.45 0.45 0.56 0.24 Visuospatial function Copying of simple figures (n=67) Semicomplex figure copy (n=62) CDT (n=66) CRT (n=63) 0.79 0.86 0.89 0.89 (0.68-0.89) (0.77-0.95) (0.80-0.97) (0.81-0.97) p 06 p 21 p 04 p 9.5 63.64 86.67 94.12 79.41 82.35 71.88 78.13 79.31 73.13 79.03 86.36 79.37 3.61 3.08 4.30 3.84 0.44 0.19 0.08 0.26 Measure Data presented as %, unless otherwise specified. 95%CI = 95% confidence interval; AUC = area under the receiver operating characteristic (ROC) curve; CDT = Clock Drawing Test; CRT = Clock Reading Test; CTT = Color Trails Test; ECR = Enhanced Cued Recall; FDT = Five Digit Test; LR- = negative likelihood ratio; LR+ = positive likelihood ratio; n = sample; RPT = Recall of Pictures Test; RUDAS = Rowland Universal Dementia Assessment Scale; SF = supermarket fluency; VF = verbal fluency (animal). to schooling level. Schooling influenced measures of executive function and visuoconstruction ability. SF scores o14 detected dementia with a sensitivity of 74.19%, a specificity of 78.95%, LR+ of 3.52 and LR- of 0.33 (Table 3, Figure 1) in the low education group. SF scores o15 classified AD with a sensitivity of 82.35% and a specificity of 81.25% in the high schooling group (Table 4, Figure 1). RPT-named scores o10 detected dementia with sensitivity of 83.87% and specificity of 31.58% in the low schooling group (Table 3), and RPT-named scores o10 classified AD with a sensitivity of 97.06% and specificity of 21.88% in the high schooling group (Table 4). RPTdelayed recall scores o4 detected dementia with a sensitivity of 87.10% and specificity of 84.21% in the low schooling group (Table 3, Figure 2), and RPT-delayed recall scores o5 classified AD with a sensitivity of 100% and specificity of 93.75% in the high schooling group (Table 4, Figure 2). RPT-recognition scores o10 detected dementia with a sensitivity of 77.42% and specificity of 73.68% in the low schooling group (Table 3, Figure 3), and RPT-recognition scores o10 classified AD with a sensitivity of 94.12% and specificity of 65.63% in the high schooling group (Table 4, Figure 3). Finally, ECR scores o12 detected dementia with a sensitivity of 80.65% and specificity of 83.78% in the low schooling group (Table 3, Figure 4), and ECR scores o12 classified AD with a sensitivity of 97.06% and specificity of 84.38% in the high schooling group (Table 4, Figure 4). Discussion This article addresses the translation, diagnostic accuracy, and standardized version of the CNTB for use in Brazil, considering the educational level of a sample of older adults with and without AD. To the best of our knowledge, this study is the first to use all subscale battery and to consider the effect of schooling in a Brazilian Portuguese speaking sample with different levels of formal education. The results indicate that the CNTB is useful for speakers of Brazilian Portuguese with different cultural characteristics, as has also been confirmed in a recent study in Europe.12 The CNTB was developed to minimize the influence of culture, language, and educational level.13 In this study, the tests that were least impacted by schooling were RUDAS, SF, RPT Picture naming, delayed recall and recognition, and ECR. Like us, Nielsen et al.12 found that the RPT and ECR were unaffected by schooling. In SF, the optimal cutoff shows little variation between the different schooling levels. Also, in line with our results, other studies have found that the SF is less affected by education, and that it is more ecologically relevant for Braz J Psychiatry. 2020;42(3) 292 NB Araujo et al. Figure 1 Receiver operating characteristic (ROC) curve for the supermarket fluency (SF) test. A) ROC curve for SF in subjects with low education (p 4 years of schooling). B) ROC curve for SF in subjects with high education (X 8 years of schooling). illiterate individuals.32,33 Nielsen et al.23,24 have also investigated the cross-cultural applicability of the RPT in samples of Turkish immigrants and Danish elderly. Despite being a short and simple test, the RPT is useful for specific evaluation of memory, with low impact of schooling. Nitrini et al.22 developed a similar test to the BCSB; the performance of illiterate and literate participants did not differ in the BCSB memory test. The optimal cutoff for diagnostic accuracy using the ECR does not vary for the comparison between the total sample, low, or high schooling. Another study34 evaluated the performance of a Turkish sample in ECR; in that study, again schooling did not affect the ECR results. The present study found significantly poorer performance of the low schooling group compared to the high schooling group in executive function and visuoconstruction. Based on the present findings, the optimal cutoff for diagnostic accuracy using the VF, CTT1, FDT, CDT, Braz J Psychiatry. 2020;42(3) Figure 2 Receiver operating characteristic (ROC) curve for the Recall of Pictures Test (RPT)-delayed recall. A) ROC curve for the RPT-delayed recall in subjects with low education (p 4 years of schooling). B) ROC curve for the RPT-delayed recall in subjects with high education (X 8 years of schooling). CRT, and copying of simple figures does substantially vary with schooling. The high levels of schooling group performed better than the low levels of schooling group. Conversely, RUDAS, RPT, ECR, picture naming, and SF performances were largely unaffected by education. Similarly to the present study, a report by Nielsen et al.35 investigated the performance of illiterate and literate Turkish immigrants on the CNTB. The preliminary findings suggest that illiteracy status affected measures of mental processing speed, executive function, and visuoconstruction. In turn, memory measures relying on free and cued recall, recognition of pictures, naming of colored pictures, and SF were relatively unaffected by illiteracy. This study has some limitations that should be acknowledged. The current sample included only participants at Diagnostic accuracy of CNTB-BR Figure 3 Receiver operating characteristic (ROC) curve for the Recall of Pictures Test (RPT)-delayed recognition. A) ROC curve for the RPT-delayed recognition for subjects with low education (p 4 years of schooling). B) ROC curve for the RPT-delayed recognition for subjects with high education (X 8 years of schooling). Figure 4 Receiver operating characteristic (ROC) curve for the Enhanced Cued Recall (ECR). A) ROC curve for the ECR in subjects with low education (p 4 years of schooling). B) ROC curve for the ECR in subjects with high education (X 8 years of schooling). mild or moderate stages of dementia. Thus, the findings do not apply for evaluation of persons with severe dementia. In addition, the sample was selected from an outpatient setting; and our pilot study only included four participants. Another limitation is that we did not include patients with mild cognitive impairment (MCI), even though this a group will also come to clinics for assessment of dementia. In addition, people with depression were excluded – another limitation, given the high prevalence of comorbidity with mild dementia. Finally, the GDS-15 was only administered to assess depressive symptoms in the AD group. Future research should consider depressive symptoms in dementia reported by caregivers. Also, MCI participants should be included in future studies. Therefore, replication of this study in a community-based sample is important. The CNTB has demonstrated feasibility for application in Brazil, and appears to be appropriate for evaluating different cognitive domains, including global cognitive function, memory, executive functions, visuospatial functions, and language. The cognitive tests less affected by schooling level were RUDAS, SF, RPT-naming, delayed recall, recognition, and ECR. The cognitive tests that best discriminated patients with AD from controls were RPTdelayed recall and ECR, both of which evaluate memory. Based on these features, the use of the CNTB may improve the cognitive assessment of AD and healthy elderly; and CNTB may be useful to discriminate between older adults with and without dementia, with high and low schooling. In Brazil, few validated instruments are available for cognitive assessment of persons with low schooling. 293 Braz J Psychiatry. 2020;42(3) 294 NB Araujo et al. There is a need for reliable methods to detect cognitive deficits that can be used in these populations across cultural scenarios. Thus, the CNTB seems adequate for use in a context with the rich cultural diversity found in Brazil. Acknowledgements Funding was provided by Fundac¸ão Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro (FAPERJ) (edital FAPERJ no. 06/2013, Programa PróIdoso – Apoio ao Estudo de Temas Relacionados à Saúde e Cidadania de Pessoas Idosas – 2013, FAPERJ, E-26/ 110.069/2013). JL is researcher 2 from Conselho Nacional de Desenvolvimento Cientı́fico e Tecnológico (CNPq) and Cientista do Nosso Estado from FAPERJ (2018-2020). The authors thank neuropsychologists Marcos Avellar and Michelle Scipião, psychiatrist Annibal Truzzi, and students Matheus Jardim, Thaı́s Campinho, Renata Naylor, Bruna Alves, and Marcelo Ferreira for the support with data collection. Disclosure The authors report no conflicts of interest. References 1 United Nations (UN), Department of Economic and Social Affairs, Population Division. World population ageing 2015 [Internet]. 2015 [cited 2019 Nov 11]. www.un.org/en/development/desa/population/ publications/pdf/ageing/WPA2015_Report.pdf 2 Instituto Brasileiro de Geografia e Estatı́stica (IBGE). Uma janela para olhar o paı́s. 2017 Mar 8. www.ibge.gov.br/home/presidencia/ noticias/imprensa/ppts/00000024052411102015241013178959.pdf 3 Laks J, Batista EM, Guilherme ER, Contino AL, Faria ME, Figueira I, et al. [Mini-mental state examination in community-dwelling elderly: preliminary data from Santo Antônio de Pádua, Rio de Janeiro, Brazil]. Arq Neuropsiquiatr. 2003;61:782-5. 4 Laks J, Baptista EM, Contino AL, de Paula EO, Engelhardt E. MiniMental State Examination norms in a community-dwelling sample of elderly with low schooling in Brazil. Cad Saude Publica. 2007;23:315-9. 5 Carvalho VA, Caramelli P. Brazilian adaptation of the Addenbrooke’s cognitive examination-revised (ACE-R). Dement Neuropsychol. 2007;1:212-6. 6 Sarmento ALR. Apresentac¸ão e aplicabilidade da versão brasileira da MoCA (Montreal Cognitive Assessment) para rastreio de Comprometimento Cognitivo Leve [MD] [dissertation]. São Paulo: Escola Paulista de Medicina, Universidade Federal de São Paulo; 2009. 7 Bottino CMC, Stoppe A Jr, Scalco AZ, Ferreira RCR, Hototian SR, Scalco MZ. Validade e confiabilidade da versão brasileira do CAMDEX. Arq Neuropsiquiatr. 2001;59(Supl 3):S20. 8 Moreira IFH, Lourenc¸o RA, Soares C, Engelhardt E, Laks J. Cambridge cognitive examination: performance of healthy elderly Brazilians with low education levels. Cad Saude Publica. 2009;25:1774-80. 9 Moreira IFH, Bezerra AB, Sudo FK, Alves GS, Ericeira-Valente L, Tiel C, et al. CAMCOG – valores das subescalas em idosos normais com nı́veis diferentes de escolaridade. Aspectos preliminares. Rev Bras Neurol. 2013;49:32-6. 10 Schultz RR, Siviero MO, Bertolucci PH. The cognitive subscale of the ‘‘Alzheimer’s Disease Assessment Scale’’ in a Brazilian sample. Braz J Med Biol Res. 2001;34:1295-302. 11 Bertolucci PH, Okamoto IH, Brucki SM, Siviero MO, Toniolo Neto J, Ramos LR. Applicability of the CERAD neuropsychological battery to Brazilian elderly. Arq Neuropsiquiatr. 2001;59:532-6. 12 Nielsen TR, Segers K, Vanderaspoilden V, Bekkhus-Wetterberg P, Minthone L, Pissiota A, et al. Performance of middle-aged and elderly European minority and majority populations on a Cross-Cultural Neuropsychological Test Battery (CNTB). Clin Neuropsychol. 2018; 32:1411-30. Braz J Psychiatry. 2020;42(3) 13 Nielsen TR, Segers K, Vanderaspoilden V, Bekkhus-Wetterberg P, Minthon L, Pissiota A, et al. Performance of middle-aged and elderly European minority and majority populations on a Cross-cultural Neuropsychological Test Battery (CNTB). Clin Neuropsychol. 2018; 32:1411-30. 14 Nielsen TR, Segers K, Vanderaspoilden V, Beinhoff U, Minthone L, Pissiota A, et al. Validation of a European Cross-Cultural Neuropsychological Test Battery (CNTB) for evaluation of dementia. Int J Geriatr Psychiatry. 2019;34:144-52. 15 de Araujo NB, Nielsen TR, Engedal K, Barca ML, Coutinho ES, Laks J. Diagnosing dementia in lower educated older persons: validation of a Brazilian Portuguese version of the Rowland Universal Dementia Assessment Scale (RUDAS). Braz J Psychiatry. 2018;40:264-9. 16 American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV). Arlington: American Psychiatric Publishing; 1994. 17 McKhann GM, Knopman DS, Chertkow H, Hyman BT, Jack CR Jr, Kawas CH, et al. The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on AgingAlzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7:263-9. 18 Chaves ML, Camozzato AL, Godinho C, Kochhann R, Schuh A, de Almeida VL, et al. Validity of the clinical dementia rating scale for the detection and staging of dementia in Brazilian patients. Alzheimer Dis Assoc Disord. 2007;21:210-7. 19 Bertolucci PH, Brucki SM, Campacci SR, Juliano Y. [The Mini-Mental State Examination in a general population: impact of educational status]. Arq Neuropsiquiatr. 1994;52:1-7. 20 Almeida OP, Almeida AS. [Reliability of the Brazilian version of the ++abbreviated form of Geriatric Depression Scale (GDS) short form]. Arq Neuropsiquiatr. 1999;57:421-6. 21 Storey JE, Rowland JT, Conforti DA, Dickson HG. The Rowland universal dementia assessment scale (RUDAS): a multicultural cognitive assessment scale. Int Psychogeriatr. 2004;16:13-31. 22 Nitrini R, Caramelli P, Herrera Junior E, Porto CS, Charchat-Fichman H, Carthery MT, et al. Performance of illiterate and literate nondemented elderly subjects in two tests of long-term memory. J Int Neuropsychol Soc. 2004;10:634-8. 23 Nielsen TR, Vogel A, Gade A, Waldemar G. Cognitive testing in nondemented Turkish immigrants comparison of the RUDAS and the MMSE. Scand J Psychol. 2012;53:455-60. 24 Nielsen TR, Vogel A, Waldemar G. Comparison of performance on three neuropsychological tests in healthy Turkish immigrants and Danish elderly. Int Psychogeriatr. 2012;24:1515-21. 25 Meyers JE, Meyers KR. Rey complex figure test and recognition trial: professional manual. Lutz: Psychological Assessment Resources; 1995. 26 Brucki SM, Rocha MS. Category fluency test: effects of age, gender and education on total scores, clustering and switching in Brazilian Portuguese-speaking subjects. Braz J Med Biol Res. 2004;37:1771-7. 27 Strauss E, Sherman EM, Spreen O. A compendium of neuropsychological tests. Administration, norms, and commentary. 3rd ed. New York: Oxford University; 2006. 28 Sant’Ana Rabelo I, Pacanaro SV, Rosseti MO, de Sá Leme IFA, de Castro NR, Güntert CM, et al. Color trails test: a Brazilian normative sample. Psychol. Neurosci. 2010;3:93-9. 29 Sedó MA. Five-digit test: manual. Madrid: TEA; 2007. 30 Campos MC, da Silva ML, Florêncio NC, de Paula JJ. Confiabilidade do Teste dos Cinco Dı́gitos em adultos brasileiros. J Bras Psiquiatr. 2016;65:135-9. 31 Shulman KI. Clock-drawing: Is it the ideal cognitive screening test? Int J Geriatr Psychiatry. 2000;15:548-61. 32 da Silva CG, Petersson KM, Faı́sca L, Ingvar M, Reis A. The effects of literacy and education on the quantitative and qualitative aspects of semantic verbal fluency. J Clin Exp Neuropsychol. 2004;26:266-77. 33 Nielsen TR, Waldemar G. Effects of literacy on semantic verbal fluency in an immigrant population. Neuropsychol Dev Cogn B Aging Neuropsychol Cogn. 2016;23:578-90. 34 Saka E, Mihci E, Topcuoglu MA, Balkan S. Enhanced cued recall has a high utility as a screening test in the diagnosis of Alzheimer’s disease and mild cognitive impairment in Turkish people. Arch Clin Neuropsychol. 2006;21:745-51. 35 Nielsen TR. Effects of illiteracy on the European Cross-Cultural Neuropsychological Test Battery (CNTB). Arch Clin Neuropsychol. 2019;34:713-20.