Introduction

ADHD is a neurodevelopmental disorder characterized by an ongoing pattern of age-inappropriate symptoms of inattention, hyperactivity, and impulsivity, which cause significant impairment in children's personal development, in their relationships with family, and in their school performance (^{American Psychiatric Association, 2013}; ^{Faraone et al., 2015}). The prevalence of this disorder has been be approximately 5 % of the child population (^{Polanczyk et al., 2007}, ²⁰¹⁴). However, this percentage has been controversial, as it varies significantly according to the different diagnostic strategies and procedures used (^{Thomas et al., 2015}).

ADHD is a clinical diagnosis based on several behavioral symptoms and criteria (^{American Psychiatric Association, 2013}). Nevertheless, a strong neuropsychophysiological component is associated with this disorder according to numerous theories (^{Faraone et al., 2015}; ^{Willcutt, 2015}). The exclusive use of interviews and rating scales for parents and teachers can hinder the proper detection of this disorder and may produce highly subjective and biased results (^{Berger et al., 2017}). Therefore, different laboratory tasks have been proposed to support a more objective ADHD clinical diagnosis (^{Fair et al., 2012}; ^{Willcutt, 2015}).

Some of the most commonly used laboratory tasks to diagnose ADHD focus on sustained attention. This mechanism refers to the ability to maintain a tonic alert state over a relatively long period (^{Huang-Pollock et al., 2012}). Sustained attention has been directly or indirectly discussed in the main explanatory theories about ADHD (^{Fair, et al., 2012}; ^{Willcutt, 2015}). The tasks most frequently used to measure sustained attention are generically called continuous performance tests (CPTs), which differ greatly based on multiple parameters (i.e. visual/auditory, type of stimuli, duration of stimuli, inter-stimuli interval, outcome measures). However, the main difference is in the way the paradigms are constructed: Go/No-Go CPTs are tasks related to restraining or inhibiting a response, whereas vigilance CPTs are tasks that focus on sustaining attention.

Despite being highly recommended by many experts in the field, the suitability of CPTs to diagnose and differentiate ADHD in children is not clear. Some studies have shown positive and acceptable results (^{Borgaro et al., 2003}; ^{Epstein et al., 2003}; ^{Fabio et al., 2015}; Rapport et al., 2000; ^{Willcutt, 2015}), whereas others have presented limited outcomes (^{Arble et al., 2014}; ^{Edwards et al., 2007}; ^{Nigg et al., 2005}). ^{Berger et al. (2017)} highlighted that CPTs have low/moderate psychometric validity and a low or moderate relationship with ADHD scales. It is not clear whether this makes CPTs an inadequate or restrictive measure of ADHD or a very specific approach that evaluates unique aspects that go beyond behavioral scales (^{Hall et al., 2016}). Part of this problem was generated by both previous meta-analyses, which mixed different CPTs and other methodological issues (^{Huang-Pollock et al., 2012}). Using vigilance tasks only and random effects models to correct sampling and measurement errors, the results showed large effect sizes in all the usual CPT measures, except reaction times, to differentiate children with ADHD and normal development children. However, these results tend to be variable (as are the parameters that define the different vigilance tasks) and not constant over time.

Moreover, symptoms observed in children with ADHD are also associated with other childhood disorders (^{Faraone et al., 2015}), and interestingly, children with high cognitive abilities are believed to show ADHD-like attentional dysfunctions with a certain frequency (^{Shi et al., 2013}). The relationship between ADHD and high abilities is controversial and infrequently studied. On the one hand, the vast majority of children with ADHD show poor academic performance and a normal IQ (only lower in working memory or processing speed); on the other hand, children with high performance are frequently referred to clinical centers for suspected ADHD. ^{Chae et al. (2003)} found a very high prevalence (9.4 %) of ADHD among children with high performance. Inattention behaviors exhibited by some of these children at school have been considered as suspected ADHD symptoms. Some of the first studies that used CPTs to assess sustained attention in children with high performance concluded that sustained attention may be impaired or ordinary in many children with high performance (^{Chae et al. (2003)}; ^{Webb & Latimer, 1993}).

More recently, some studies have explored electrophysiological correlates and sustained attention performance in talented children and control children. Using a Go/No-Go task, talented children gave significantly more correct answers, made fewer errors of commission, and had, therefore, more attentional capacity, reflected in the d’ index and more neural efficiency (reflected on event-related potential cue-P2 and cue-P3 waves), than their control counterparts (^{Duan et al., 2009}; ^{Liu et al., 2011}; ^{Shi et al., 2013}). These large effect sizes were expected since sustained attention includes concentration and control of impulsivity, which are both essential for the proper functioning of higher cognitive processes. However, these studies have three main limitations: 1) they only assessed children of 12 years of age and older: 2) they did not make strict use of vigilance tasks, and 3) they did not include children with ADHD in the comparisons.

Considering this context, this study aimed to analyze the differences in a vigilance task among ADHD, high cognitive performance, and normal development children in a community setting. It is important to highlight that community-based samples were used since most previous studies featured a more stringent experimental control. Therefore, this study is novel in providing data from a more ecological and applied perspective. We believe that the vigilance component of sustained attention is equally, or more, important than control of impulsivity (measured in the CPTs) for the proper functioning of higher cognitive processes. Indeed, it is not possible to maintain a high level of cognitive efficiency without the ability to hold attention for a long time. Therefore, we hypothesize that children with high performance will score better on vigilance tasks than ADHD and normal development children. Given that these types of tasks have been designed to help identify children with ADHD, we also assume that children in normal development will obtain better scores than children with ADHD. Then, we analyzed the clinical validity of vigilance tasks by comparing their suitability to differentiate the subjects of the three groups studied. In this case, we assume that the vigilance task will present high levels of sensitivity and specificity regarding differentiating normal development children from ADHD children, and high-performance children from ADHD children.

Methods

Participants

The data of this work comes partially from the doctoral dissertation project of the third author. This project had the approval of the research committee of the Doctoral Program in Clinical Psychology and Health of the University of Barcelona (Number: 100-IV), and no further ethical requirements were demanded.

Three samples of children between 7 and 12 years old with different characteristics and from different sources were selected:

1. An ADHD sample was selected and referred by the ADANA Foundation. The ADANA Foundation is a non-profit organization that has been diagnosing and treating children with ADHD for more than 20 years with its own clinical team (psychiatrists and psychologists). This is an ecological clinical sample. The clinicians used the procedures they considered convenient for the diagnosis of the children, although in all cases, at a minimum, interviews and ADHD scales based on the DSM criteria were used. Clinicians were asked to monitor comorbidity so that they could refer participants with learning disorders and oppositional defiant disorder, as well as mild anxious-emotional symptomatology.

2. ADHD high-performance children were selected and referred to our study by the teachers of two private schools from Barcelona: “Oriol Martorell”, a school focused on teaching music and dance, and “European International School”, a multilingual international center that promotes a curriculum of excellence to allow its students direct access to Spanish, United Kingdom, and the United States universities. The teachers of both schools selected children who presented an excellent academic level (in addition, in the case of the "Oriol Martorell" school they had to present high motor skills for dance).

3. Finally, the normal development comparison group was selected from a public school in Barcelona, “Rosa dels Vents”. In this case, the main inclusion criteria for teachers were to select children with an average academic performance.

Exclusion criteria for all groups were the presence of serious neurological or psychiatric disorders, such as intellectual disability, autism spectrum disorder, schizophrenia (or other psychotic disorders), and/or epilepsy. In addition, to be part of the sample, children assessed with the Children Sustained Attention Task (CSAT-R) were required to have a T score in response style C between 37 and 63 (see Measures and Procedures).

The ADHD group initially referred to our study was 51 participants, but 9 were discarded for having an excessive uninhibited response style. Therefore, the final sample for this group consisted of 42 participants (19 % girls): 16 ADHD-combined, 7 ADHD-inattentive, 3 ADHD-hyperactive/impulsive, and 16 who did not belong to any specific subtype. In clinical practice, it is not so unusual for some clinicians to only make use of the diagnosis "ADHD" without specifying the subtype. In general, this is indicative of a "combined" presentation of ADHD, but this was not specified. Moreover, 74 % of these children were pharmacologically treated as follows: 79 % of ADHD-combined, 75 % of ADHD-inattentive, 100 % of ADHD-hyperactive/impulsive, and 65 % of children within an unspecified subtype (evidence that probably most of these children were ADHD-combined although it was not specified in their clinical report). However, all the children were off of their stable ADHD medication at least 48 hours prior to the assessment. Of the total sample, 18 % had some type of comorbidity. Finally, given the lack of differences in CSAT-R attentional indexes depending on the subtype or medication, a single ADHD group was defined. The high-performance group was formed by 11 participants with high academic performance (55 % girls), and 9 with high academic performance and high motor skills (all girls). All of them showed a balanced response style. Given that both groups did not have differences in the other attentional measures used in this study, they were analyzed as a single group: 20 participants (75 % girls). Finally, the normo-typical group was initially formed by 33 children, but 5 were discarded due to their response style (4 for excessive disinhibition and 1 for excessive inhibition). Accordingly, the total sample of this study consisted of 90 participants divided into three groups: ADHD group (n = 42, M_age = 9.50, SD = 1.53), high performance group (n = 20, M_age = 9.50, SD = 1.24), and normo-typical group (n = 28, M_age = 9.54, SD = 1.32). Table 1 compares the average age of the participants by group and sex.

Table 1: Average age of the participants by group and sex. 

The distribution by sex in Table 1 showed statistically significant differences (χ² (2) = 18.08, p < .000), mainly because the percentage of girls in the ADHD group (19%) was much lower than in the high-performance group (75 %). There were no statistically significant differences in the mean age between the three groups or in the group by sex interaction.

Results

A MANOVA with two between-subject factors: group (ADHD, normo-typical development, and high-performance) and sex was performed. Differences in hits, errors, and RT in the CSAT-R were analyzed. The group by sex interaction was not significant, and within the sex factor, there were only statistically significant differences in the RT measure. In fact, boys were 49 milliseconds faster on average than girls (F (1, 94) = 13.73, p < .000, ηp² = .13, medium). Given the small influence of the sex factor in the results, it was removed from the definitive analyses shown in Table 2.

Table 2: Comparison between the three groups in the CSAT measures. 

^Note.RT = Reaction time; C = Response style index; d’ = Attentional capacity parametric index; A’ = Attentional capacity nonparametric index.

The group factor showed statistically significant differences in hits, errors, d’, and A’.

Effect sizes (ES) were small for hits, but medium/large for errors, d’, and A’ (usually from ηp² > .13 is considered a large effect). In addition, note that the standard deviation (SD) showed a clear trend for all variables (except for RT) of much greater variability in the ADHD group than in the high-performance group. Consequently, the Levene's test was significant in all the variables, except for RT and hits. Moreover, normality tests were significant for all measures except RT. For these reasons, it was decided to perform the post hoc comparisons using the nonparametric tests (Mann-Whitney's U) and to provide their appropriate effect size (Rosenthal's r). Table 3 shows the results obtained, both in the direct scores (left side of the table) and the CSAT-R indexes (right side).

Table 3: Group differences analysis in CSAT raw scores and indexes. 

^Note.AD = ADHD group; NO = Normal development group; HP = High-performance group; RT= Reaction time; C = Response style index; d’ = Atten-tional capacity parametric index; A’ = Attentional capacity nonparametric index; U = Mann-Whitney's U; r = Rosenthal's effect size.

The ADHD group compared to normal development group showed significantly more errors, a more disinhibited response style (C), and a lower attention capacity (only for A’ index), but with small ES (all r's values were lower than .30). The ADHD group scored significantly fewer hits and more errors than the high-performance group. Consequently, ADHD showed a lower attention capacity both in d’ and A’. ES were large for errors and both attention indexes (r > .50) and medium for hits. The last comparison showed that the high-performance group got significantly more hits, fewer errors, and more attention capacity than the normal development group with large ES except for hits (medium ES). It was also observed a difference in RT with longer times for high-performance group with a small/medium ES.

Figure 1 shows the distribution of participants with “attention dysfunction”, according to the A’ index in each experimental group. We used the index A' instead of d' since it is a non-parametric index that is not so dependent on normality criteria.

Figure 1. Distribution of participants in the three groups by presence or absence of CSAT detected attentional dysfunction. 

Twenty percent of the total sample showed suspicion of attentional dysfunction, but it was distributed differently among the three groups. Attentional dysfunction was observed at 29 %, 21 %, and 0 % in the ADHD, normo-typical development, and high-performance groups, respectively. Table 4 shows the statistics of the clinical validity of the CSAT-R based on the various comparisons between groups.

Table 4: CSAT clinical validity for group comparisons. 

^Note.AD = ADHD group; NO = normal development group; HP = high-performance group; PNV = Predictive negative value; PPV = Predictive positive value.

The high-performance group obtained a perfect specificity, both in relation to the normo- typical development and to the ADHD group. All high-performance children showed normality in sustained attention. This means that the predictive positive value was also 100 %. Therefore, the probability of belonging to the normo-typical or the ADHD group instead of the high-performance group when the CSAT-R was positive (suspicion of dysfunction) was 100 %. On the other hand, sensitivity was low, which in turn sometimes causes negative predictive values (less than .50). This means that the probability of belonging to the high-performance group instead of the normo-typical development group or to the ADHD group when the CSAT-R was negative (discarding dysfunction) was 48 % and 40 %, respectively.

Besides the comparison among the high-performance group and the others, the most interesting comparison to analyze the clinical validity of the CSAT-R was between the ADHD and the normo-typical groups. In particular, the specificity and the positive predictive value were moderately high: the probability of belonging to the ADHD group when the CSAT-R detects an attentional dysfunction was 67 %. In contrast, the sensitivity and the negative predictive value were low: the probability of belonging to the normo-typical development group when the CSAT- R was negative was only 42 %. This was because the percentage of children who showed attentional dysfunction in the normo-typical development and ADHD groups were very similar (21 % and 29 %, respectively).

Discussion

This study is aimed to address this issue: What kind of differences can we expect in sustained attention in community-based samples of ADHD, high capacity, and normal development children? Currently, ADHD diagnosis is based primarily on interviews and behavioral rating scales, although most explanatory theories consider it a neuropsychophysiological disorder. In this approach, sustained attention has played a fundamental role, but in practice, its measure has not been unique. Traditionally, the CPTs have been preferred, but vigilance tasks could be more relevant (^{Huang-Pollock et al., 2012}).

However, the results have not always been consistent (^{Berger et al., 2017}). In addition, the relationship between ADHD and high capacities is a bit confusing. Although most children with ADHD usually have learning disabilities and moderate IQ, it has been proposed that a significant percentage of them may have high capacities (^{Chae et al., 2003}). Finally, the role of sustained attention in high capacity is little known. There is no doubt that executive functions and even inhibition of response (another key element of ADHD) are areas where talented children stand out, but few studies are evaluating the vigilance function.

Therefore, we aimed to analyze differences among children with ADHD, high academic performance, and normo-typical development in a brief, visual, computerized vigilance task: CSAT-R (^{Servera & Cardo, 2006}; (^{Servera & Llabrés, 2015}). After comparing the three groups, the most striking result was that the high-performance group was significantly differentiated, with large effect sizes, from the other two groups in attention capacity, due more to the influence of errors than hits. This result was favorable to our first hypothesis, which was derived from Shi's works (^{Duan et al., 2009}; ^{Liu et al., 2011}; ^{Shi et al., 2013}). That is, children with high performance are not affected by a possible attentional dysfunction, and their attention capacity was better compared to children with ADHD and normal development children. Therefore, the possible attentional problems sometimes found or attributed to children with high abilities seem to reflect more a behavioral issue (for example, they become bored in class because they know and understand the contents very well), rather than a neuropsychological dysfunction (^{Chae et al., 2003}).

Furthermore, differences between the normo-typical and ADHD groups were not clear. In comparison with previous meta-analysis results (^{Huang-Pollock et al., 2012}), our hypothesis on commission errors can be considered as supported, given that normal development children were significantly superior in this respect to children with ADHD, but with a small effect size.

Nonetheless, our hypothesis has not been proven concerning hits, where the relative best score of normal development children only led to no significant small effect size. As a consequence, our attentional capacity indexes also showed small effect sizes, with only significant differences in A’.

Through observation of the data, children with ADHD showed a significantly more impulsive response style than normal development children. This comparatively riskier style has allowed them to have several hits similar to that of normal development children (only 3% worse), but with a high number of errors (52 % more than normal development children). Besides, it must be considered that our data analysis approach is more conservative and stricter than those used in most of the previous studies since we used adjusted nonparametric procedures and statistics.

The clinical validity of CSAT-R was analyzed through comparisons of sensitivity, specificity, and positive and negative predictive values among the three groups. Comparisons between the high-performance group and the normo-typical and ADHD groups are not very relevant because the main objective of the CSAT-R is not to detect children with high abilities. However, it is worth noting that given that none of the children with high performance showed an attentional dysfunction, the specificity, and thus the positive predictive value, were perfect. The comparison between the ADHD and the normo-typical group is the most interesting to evaluate the clinical validity of CSAT-R. The hypothesis was that the task would have high sensitivity and specificity. The results were mixed: the specificity was moderately high, and the positive predictive value was .67 (i.e. 67 % of children who showed attentional dysfunction belonged to the ADHD group). In contrast, the sensitivity value was low, and the negative predictive value was only .42 (i.e. the probability of belonging to the normo-typical group for a child that showed a normal attentional function was 42 %). These data only partially support the clinical validity of CSAT-R (i.e. the task has acceptable specificity but low sensitivity).

However, it should be noted that our results, unlike most of the preceding ones, come from community settings. This probably implies that the clinical sample and the comparison group are more similar than if they had been selected from cut-off points or stricter procedures. This may explain why 21 % of children without ADHD have attentional dysfunction and 71% with ADHD do not. However, this is the reality in which neuropsychological measures should be applied and interpreted, and the results obtained can be useful, since, in most diagnostic protocols of ADHD, attentional tasks are used to confirm the diagnosis (not to rule it out). From this point of view, the data showed that clinicians can somewhat rely on the CSAT-R, given that when attentional dysfunction is detected, the probability of having an ADHD diagnosis will be close to an acceptable 70 %. Furthermore, if the CSAT-R is negative, it is not possible to hypothesize if the child belongs to a normo-typical or clinical group. In addition, it should be noted that the CSAT-R uses a single global measure of attention. This means that clinical decisions may be more affected by Type 2 than Type 1 error (i.e. the better chances to hit are greater by detecting an attentional dysfunction than by ruling it out).

Beyond these clinical issues, there are three possible explanations for the overall results found in the evaluation of CSAT-R (and other similar attentional tasks), which should be taken into account for future studies. First, as ^{Berger et al. (2017)} pointed out, these tasks have little ecological validity as they perform a pure evaluation of the attentional capacity in a highly controlled environment. Additionally, they do not require a high cognitive load that can cause differences between individuals (^{Fabio et al., 2015}). Our data show that there is little difference between normo-typical children and children with ADHD in the CSAT-R. It is necessary to observe a group of children with the high cognitive capacity to understand how their performance on the CSAT-R truly differs from the other two groups. Second, the CPTs and other attentional tasks could be measuring functional aspects of attention different from the behavioral component measured with parent and teacher's rating scales. The difference between the neuropsychological and behavioral measures of attention has been proposed as a new way of understanding ADHD (^{Fair et al., 2012}). In this line, it has been proposed that at least two neuropsychophysiological dysfunctions could play different roles in ADHD (^{Halperin & Schulz, 2006}): a subcortical dysfunction (more persistent over time and measurable with attentional tasks) and a frontal dysfunction (more sensitive to improvement over time and measurable with behavioral scales).

Finally, it is possible that simple, low-load cognitive tasks, as CSAT-R or most of CPTs, may not be the best measures to assess sustained attention deficit (^{Huang-Pollock et al., 2012}). Moreover, even if poor performance is detected in hits, errors, or attention indices between ADHD and normal groups, it is considered that this is not the result of a sustained attention deficit but rather of the consequence of some secondary effects associated with ADHD, and only decrement performance over time can be linked with attention deficit (^{Tucha et al., 2009}). However, the issue is to know the best way to measure this variability: using standard errors, coefficients of variability, slopes (over blocks, ISIs, etc.), or even using indices from ex-Gaussian distributions (see, for example, ^{Epstein et al., 2011}). While this question is solved, ^{Tucha et al. (2009)} accept that differences between ADHD and healthy subjects on the CPTs performance may be of some clinical utility, as CSAT-R has partially shown.

This study has a series of limitations, most of them derived from working with community samples, and others from technical aspects. First, the group with high academic performance was chosen directly from schools with the help of teachers. In future studies, the high capacity of these children must be confirmed with specific tests of cognitive functioning. Second, the sample of children with ADHD was excessively heterogeneous in terms of subtypes, comorbidity, and treatment received. For example, although no differences were detected between ADHD children who took medication and those who did not, the majority of ADHD children had prescribed medication. This may have influenced the relative lack of differences between them and the normo-typical group. Therefore, in the future, it is necessary to more strictly control the selection of the clinical sample. In the same line, the control sample must also be chosen more carefully, since in our case, there were 21 % children with attentional dysfunction, which seems to be a very high percentage. It is necessary to select this type of sample with interviews and scales that rule out the presence of children with subclinical scores or with other types of problems that make comparing results difficult. Finally, from a technical point of view, it is necessary to highlight some limitations of the CSAT-R: Scoring based on signal detection theory's indexes are used for most of the attentional clinical tasks, but they are not the only and perhaps not the best way to measure performance on these tasks. Diffusion models, although much more widely used in experimental studies, can be a highly recommended alternative (^{Donkin, Averell, Brown, & Heathcote, 2009}; Ratcliff, 1978). Also, the lack of a measure of RT variability in the CSAT-R is an important limitation since this measure seems to be more robust and stable than other measures based on hits, errors, or simply reaction time (^{Kofler et al., 2013}).