Prior Entry for Feature-based Attention: Are Objects of the Attended Color Perceived Earlier?

Keywords

Prior-Entry; Feature-Based Attention; Temporal Order Judgment; Simultaneity Judgment.

Introduction

Attention has been thought to play a major role in selection of input for further process [4]. It is also hypothsized that attention can accelerate sensory processing, thereby causing attended objects to be perceived earlier than unattended objects [30]. This prior-entry effect of attention has been investigated for more than a century [2, 28] and many studies about this hypothesis emerged recently [10, 19, 24, 23, 27, 31, 38, 40]. Temporal order judgment (TOJ) task [21, 23, 34, 35, 38, 39, 5] and simultaneity judgment (SJ) task [21, 39, 40] are the two typical paradigms to study the prior-entry effect. Using these paradigms, many researchers have reported the existence of prior-entry effects across different modalities [28, 27, 40, 1, 33], and within the same modality: visual [12, 23, 34, 35, 36], auditory [11] or somatosensory [38, 39].

Within the visual modality, studies have shown that spatial visual attention can speed up visual sensory processing, such that objects in the attended location would be perceived prior to those in the unattended location [26, 23]. Shoreet al. (2001) used an orthogonal cueing temporal order judgment paradigm (TOJ) to demonstrate this effect. In an orthogonal TOJ paradigm, observer’s attention was directed to a dimension that is orthogonal to the TOJ task dimension, thus reducing the confounding effect of response bias that are argued to be inherent in a traditional TOJ paradigm. In their study, Shoreet al. (2001) presented a peripheral flash cue (exogenous spatial cueing) or a central pointing-arrow cue (endogenous spatial cueing) prior to the TOJ stimuli to orient observers’ attention either to the left or right side of the fixation mark. Observers were asked to indicate whether a vertical or horizontal line was presented first, instead of reporting the location of the first stimulus, thus reducing response bias toward the tobe- attended location. The point of subjective simultaneity (PSS) is then determined to indicate how much time one stimulus has to precede the other in order for them to be perceived as simultaneous. Shifts in the PSS are used as a measure of putative prior-entry effects.PSS shifts were observed from both endogenous and exogenous spatial attention cueing. The PSS shift was approximately 61 ms for exogenous orienting and 17 ms for endogenous orienting; objects in the unattended location had to lead objects in the attended location in order for simultaneity to be perceived for the two objects. Schneider and Bavelier (2003) replicated the same effects using a TOJ task. In both studies, PSS shifts from exogenous spatial attention were larger than those from endogenous spatial attention.

Schneider and Bavelier (2003) also demonstrated a much smaller effect using a simultaneity judgment task (SJ). In the SJ task, observers had to judge whether the stimuli presented on different locations were presented simultaneously or sequentially. The main advantage of the SJ paradigm is that it is affected minimally by response and decision biases, as compared to the TOJ paradigm. This is because observers do not have to pick one stimulus over the other; therefore there is no reason for observers to prefer one stimulus to the other, no matter under what attentional condition. If attention has a sensory acceleration effect on perception, PSS shifts should be observed directly from the SJ task [21, 40]. Schneider and Bavelier (2003) reported 10 - 40 ms shifts of PSS from exogenous spatial visual cue using the SJ task but they attributed the shifts to cue-stimulus sensory interactions. In the same study, Schneider and Bavelier (2003) observed a marginally significant effect from an endogenous spatial visual cue in their SJ task only when the cue-lead-time (the time interval between the cue and the stimuli) was 600 ms1 but not when it was 0, 100, 300, 1000, or 1500 ms. They concluded that the major reason whyparticipated in both condition experiments with exogenous cues exhibited increased PSS shifts than endogenous cues is the sensory interaction between the exogenous cue and the stimulus. Due to their spatial and temporal proximity, the cue facilitated the temporal processing of the stimulus. Overall, they concluded that most of the PSS shift was due to decision bias and sensory interactions, with only a small part due to priorentry effects.

In addition to spatial visual attention, feature-based attention is another mode of visual attention that has shown to have similar behavior consequences as spatial attention [3, 13, 18, 37, 41]. Both modes of attention facilitate human behavioral performance [17, 6, 18, 14]. Spatial attention can be either bottom-up (exogenous) or top-down (endogenous) and both modes of spatial attention have been shown to affect the visual temporal processing and visual temporal resolution [15, 23], whereas feature-based attention is thought to be mainly top-down (endogenous) [9]. Unimodal visual prior-entry effects have only been examined for spatial attention and have never been demonstrated for feature-based attention. Thus, a relevant question is whether feature-based attention has a similar effect on temporal processing as spatial attention. If there exists a prior-entry effect from engaging feature-based attention, would its magnitude be comparable to that resulting from engaging endogenous spatial attention? Studying the prior-entry effect with feature-based attention will provide further empirical insight on how endogenous attention affects visual temporal processing.

Here, we first employed an orthogonal TOJ task to investigate the prior-entry effect based on feature-based attention. Central color cues were used to direct observers’ attention to one of the two colors of the stimuli. A recent study used irrelevant non-informative color cues with TOJ as well as SJ tasks, and obtained evidence in favor of the prior-entry effect [29]. However, it is unknown whether active attention to the color cue would cause stronger prior-entry effect. We addressed this question in the present study by including two major conditions, one without and one with an attentive task that users had to perform on the color of the cue. Our results showed that prior-entry effects were not observed when subjects were not required to perform an attentional task on the central color cues; by contrast, a moderate magnitude of PSS shift was present when a central attentional task on the color of the cues was used to strongly engage observers’ attention. The difference in the PSS shift between engaging and not engaging attention on the cue color in the TOJ task is an important finding that is taken up in the Discussion section. Secondly, an SJ task was employed with the intention of eliminating potential response and decision biases that might potentially be present in a TOJ paradigm. A similar trend of PSS shift was observed, however, the PSS shift was not statistically significantly different from zero in the SJ task. A possible explanation of the different findings within the present study, as well as for the discrepancy with the previous study will be discussed later.

Methods

Before providing details on the specific values of the parameters used in the study, we provide a brief global summary and rationale for the experimental conditions. The different experiment conditions were tested in chronological order: TOJ with no central attentional task (condition 1), TOJ with central attentional task (condition 2,3 and4) and SJ with central attentional task (condition 5). Condition 1 was to test whether a cue of a specific color automatically triggers feature-based attention by its mere presence. Since no PSS shift effect was observed in condition 1, we further examined whether observers have to exert volitional effort to engage feature-based attention in condition 2, 3 and 4. To test the time course of feature-based attention, we used a TOJ task combined with an attentive task on the color of the cue with a short, a medium and a long cue-lead-time (CLT), respectively, for conditions 2, 3 and 4. Since the TOJ task generally exhibits much larger effects than the SJ task, if we did not find a priorentry effect with the TOJ task, we would not have to re-test with the SJ task. A significant PSS shift was, however, observed in TOJ with central attentional task, thus, condition 5 that involved an SJ task with a central attentional task and a medium CLT was tested.

Observers

Four observers participated in the TOJ task without central attentional task (condition 1). We used three values of cue-lead time (CLT) in the TOJ task with central attentional task, as explained in detail below. Six observers participated in the short-CLT case (condition 2), eight in the medium-CLT case (condition 3), and eight in the long-CLT case (condition 4). One observer participated in condition 1, condition 2 and condition 3; one observer participated in both conditions 1 and 4; two observers participated in condition 2, 3 and 4; and one observer participated in both condition 3 and 4. Eight different observers participated in the simultaneity judgment (SJ) task (condition 5). All of them had normal or corrected-to-normal visual acuity. All observers were naïve as to the purpose of the experiments.

Stimuli

The display (see Figure 1a and 1b) consisted of two square boxes, each subtending 2.8 x 2.8 degrees of visual angle. The boxes remained on the screen throughout the block, one on each side (left and right) of a central fixation dot. The fixation dot was 4.2 degrees away from the inner edge of each box. The entire display subtended 2.8 x 14.0 degrees of visual angle. The viewing distance was 60 cm. In each trial there was one oriented bar presented in each box. The bars were 0.2 x 1.5 degrees of visual angle. One of them was tilted to the northeast and the other was to the northwest. Each bar was assigned to be red or green. The color and orientation combinations of the bars were counterbalanced to produce equal numbers of each combination. A flicker photometry method was used to measure the equiluminance values for red and green patches for each observer prior to the main experiment.

×

Figure 1 :

Close

Figure 1. (a) Schematic diagram for the TOJ without attentional task: the cue color could be red (as in the example here), green or gray (neutral cue); trials with the three types of cues were randomly mixed in the same block with equal probability. (b) Schematic diagram for the TOJ and the SJ task with an attentional cue task. Three different CLT conditions were tested in the TOJ task, with the cue frame for the short CLT condition shown on the left side and that for the medium and long CLT conditions shown on the right side. Only the medium CLT condition was tested for the SJ task and its parameters are the same as those for the medium CLT condition for the TOJ task.

Procedure

Three different task conditions were tested in sequence: TOJ without central attentional task, TOJ with central attentional task and SJ with central attentional task. The procedure was similar in all conditions with some variations in each. In general, each trial of the experiment started with a fixation phase, followed by a cue phase, and finally a stimulus phase. The fixation phase, which consisted of only the fixation dot and the boxes, lasted for 500 - 750 ms. During the cue phase, the cue lead time (CLT, the time from the onset of the cue to the onset of the first stimulus), the cue duration, and observers’ tasks on the cue were varied in different conditions. In the TOJ without central attentional task condition, the CLT was 110 ms, with the cue presented for 50 ms and a short cue-to-stimulus interval lasting 60 ms. The cue was a color patch (see Figure 1a) that replaced the fixation dot; its color could be red, green, or gray for a neutral cue. In the TOJ with central attentional task condition, a task on the cues was included to engage observers’ attention to the feature cues (see Figure 1b). Both red and green cues were presented in each trial. Three different cue lead times (CLT) between the cue and the first stimulus were used: 250 (short CLT), 700 (medium CLT) and 1200 (long CLT) ms as shown in Figure 1b. The cues lasted for 150, 600 or 1000 ms and then a 100, 100 or 200 ms cue-to-stimulus interval followed the offset of the cues, respectively.

For the medium and long CLT conditions (see the right panel in Figure 1b), two color patches, one reddish and one greenish, were presented. At the last 150 ms of the cue duration, a color saturation change event occurred on the attended color in half of the trials. There could also be a saturation change event on the unattended color, independently of the occurrence of an event on the attended color. Observers were instructed to attend to the red or green cue in separate blocks and perform a saturation change detection task on the attended color cue. The 80%-correct color saturation threshold for each observer was measured prior to the main experiment and the threshold was then used in the saturation change detectiontask. For the short CLT condition (see the left panel in Figure 1b), the central cue consisted of four color patches, two reddish and two greenish. Observers performed a same-or-different-saturation task on the attended color. In other words, they were asked to discriminate whether the two patches of the attended color had the same saturation or not. The two patches had the same saturation in half of the trials, and different saturation in the rest of the trials. The two patches of the unattended color could also have the same or different saturation, independently of the relationship between the patches of the attended color. Given the brief duration of the cue (150ms) in the short CLT condition, if we would use the same cue saturation change detection task as in other two conditions, the saturation change event would last very briefly (e.g. 50ms) and the event would occur shortly after cue onset (e.g. 100ms after cue onset). This would make it much more demanding to perform both the detection task and the TOJ task. Therefore, a same-or-different-saturation task was employed instead to make the task easier. In the SJ with attentional task condition, the cue lead-time, the cue duration and observers’ task were the same as the medium CLT condition in the TOJ with central attentional task.

The last phase of each trial was the stimulus phase. The first stimulus (the northwest bar or the northeast bar, chosen at random) was presented in the left or right box, followed by the second stimulus (the northeast bar or the northwest bar, respectively) in the other box after a variable stimulus onset asynchrony (SOA). In the TOJ without central attentional task condition, the SOAs were ±15, ±45, ±90, ±135, ±240 ms. In the TOJ with central attentional task condition, the SOAs were ±15, ±45, and ±240 ms. In the SJ with central attentional task condition, nine SOAs values were tested: 0, ± 30, ± 45, ±60, and ± 90 ms. After the second bar was presented, both bars stayed on the screen until observers responded. In the TOJ tasks, observers were instructed to make a temporal order judgment on which orientation of bar came first, the northwest one or the northeast one, by pressing two different keys. In the SJ task, observers were asked to make a simultaneity judgment. That is, they were instructed to determine whether the two bars were presented simultaneously or consecutively, regardless of the order. Observers were told that their judgment accuracy would be recorded. Although there was no emphasis on reaction time in the instruction to the observers, reaction time was measured for the medium CLT condition in the TOJ with attentional task condition as well as in the SJ with attentional task condition. Observers participated in a practice session with SOA of 30 and 90 ms between the two stimuli. During the practice session, an auditory feedback was provided at the end of each trial to signify whether the response was correct or incorrect, using two different tones.

Results

TOJ task (without attentional task on cue)

The average proportion of “red first” responses across observers is plotted as a function of SOA in Figure 2a. Even though the task was to report whether the stimulus with northeast or northwest orientation was presented first, we converted a response of orientation to a corresponding response on color, for data analysis and display purposes. In other words, the response was converted to the appropriate color response, as if observers were asked to report whether the red or greed stimulus was presented first. The purpose of this transformation is to see clearly the effect of prior-entry. According to the prior-entry hypothesis, when observers attend to the red color cue, they would be more likely to report the red bar presented first, and vice versa when they attend to the green color cue. The results show that in the TOJ task without an attentional task on the cue, responses under the three cueing conditions (red, green vs. neutral cue) were essentially the same. There is no shift of PSS observed for this experimental condition. Logit models were fitted to data of each cue condition for each observer using maximum likelihood estimation. PSS values were then estimated from the models. Average PSS across observers are shown in Fig. 3a. A one-way ANOVA test indicated that the PSS of different cue conditions was not significantly different, F (2, 9) = 0.22, p = 0.80.

×

Figure 2 :

Close

Figure 2. Proportion of "red first" responses as a function of SOA between the two stimuli in the TOJ without attentional task experiment (a), and when attending to different color cues for the three different CLT conditions: short (b), medium (c) and long (d) in the TOJ with attentional task experiment. Positive/negative SOAs represent trials in which the Red/ Green bar is presented first, respectively.

×

Figure 3 :

Close

Figure 3. PSS data when different color cues were presented in the TOJ without attentional task experiment (a), and when attending to different color cues for the three different CLT conditions: short (b), medium (c) and long (d) in the TOJ with attentional task experiment.

TOJ task (with attentional task on cue)

For all three CLT conditions, when observers were engaged in an attentional task on the cue, there was a left horizontal shift of responses for the attend-to-red condition relative to the attend-togreen condition (see Figure 2b, c, d). Observers were more likely to report that the oriented bar associated with the red color was presented first when attending to the red color, and less likely when attending to the green color. Logit models were fitted to the data of each cue condition for each observer using maximum likelihood estimation. PSS values were then estimated from the models. Average PSS across observers are shown in Figure 3b, c and d. The PSS values were subjected to a 3 x 2 ANOVA test with a between-subject factor (CLT: short, medium and long) and a within-subject factor (attention condition: red or green). The ANOVA test revealed a significant main effect of attention condition: F (1, 19) =11.80, p = 0.003. When observers attended to the red color cue, the average PSS was -13 ms, indicating that the green bar needed to be presented first for the two bars to be perceived as presented simultaneously, and vice versa when they attended to the green color cue (average PSS was 10 ms). The main effect of CLT and the interaction effect were not significant. Analysis on the reaction time for the medium CLT condition did not show significant difference between attending to red or green color feature for each of the SOA values tested.

SJ task (with attentional task on cue)

The average proportions of “simultaneous” responses are shown as a function of SOA in Figure 4. Observers’ responses under the two different attentional conditions were essentially the same. Gaussian models were fitted to the data of each attentional condition for each observer. PSS values were then estimated from the models. Two-tailed paired t-test showed no significant difference on the PSS values between the two attentional conditions. The average PSS values for the “attend-red” and “attend-green” conditions were -5.5 ms and -3.5 ms, respectively; the difference is in the right direction for the presence of a prior-entry effect, but not statistically significant. Analysis on the reaction time did not show significant difference between attending to the red or green color for each of the SOA tested.

×

Figure 4 :

Close

Figure 4. Simultaneity judgment task data when attending to different color cues. Positive/negative SOAs represent trials in which the Red/Green bar is presented first, respectively.

Discussion

The visual prior-entry effect has been tested extensively for spatial attention. Although most studies have concluded that the object cued by spatial cues (exogenous or endogenous) is perceived to arrive earlier than the uncued object [20, 23, 26], a comprehensive study has compared results between data obtained with TOJ and SJ tasks and attributed such putative prior-entry effects mostly to attentional influences on response and decision biases, as well as to sensory cue-stimulus interactions [21]. The effects observed with TOJ tasks are reported to be generally smaller for endogenous spatial cues than for exogenous spatial cues. In these studies, spatial attention was directed to a location by either an abrupt onset cue for exogenous attention, or a foveal cue (e.g. a central arrow) for endogenous attention.

An exogenous spatial cue can strongly capture observers’ attention automatically [6, 7, 16, 17]. It is very difficult for observers to ignore an abrupt-onset spatial cue. However, an endogenous spatial cue does not attract attention automatically, and observers need to voluntarily shift their attention to the directed location. In most studies that investigated prior-entry effects of endogenous spatial attention, the cue was not directly relevant to any of the observers’ tasks [21, 23]. The cue neither provided additional information about which stimulus would be presented first, nor were observers required to perform any task on the cue. In this regard, it is possible that the irrelevant central cue was not efficient in engaging endogenous attention, thus producing smaller TOJ PSS shifts than those from exogenous attention cues. A larger PSS shift may be observed with a more efficient way of engaging endogenous attention.

Unlike spatial attention, feature-based attention is thought to be mainly an endogenous type of attention [9]. Findings from the current experiments provide further evidence on PSS shifts from endogenous attention in general, as well as afford an interesting comparison of PSS shifts from endogenous spatial attention and (non-spatial) feature-based attention. In the current study, we either had observers view the cued feature passively or we controlled the degree of engagement of their feature-based attention by having an attentional task on the cued feature. The TOJ without attentional task (condition 1) is analogous to an endogenous cue (e.g. a central arrow or a gaze-direction cue) for spatial attention. The absence of a PSS shift in condition 1 (Fig. 2a) indicates that a color cue cannot automatically engage feature-based attention; in contrast, when subjects exerted volitional effort by performing a color saturation task on the cue, we obtained a PSS shift in the TOJ condition. Thus, a colored cue in feature-based attention experiments requires subjects’ attentional engagement to exhibit a PSS shift in the TOJ paradigm. These results are however inconsistent with findings of a recent study, in which PSS shift was observed when no attentional task was required on the central cue [29]. It is possible that there are individual differences of observers’ voluntary attentional engagement in the two studies, stemming mainly from the use of the “go/no go” approach in Theeuwes and Van der Burg (2013).

That observers’ attention was not strongly engaged to the cue in condition 1 of the current study is supported by observers’ subjective reports after the experiment, as well as their reports after an additional experimental TOJ condition that we conducted with the same 4 participants in condition 1, where one of the two locations was cued by an exogenous peripheral spatial cue; the parameters and tasks were identical to those of condition 1 (CLT=110 ms, same SOA range). A large - statistically significant - shift of PSS was observed and the magnitude, approximately 76 ms, was comparable to those reported in earlier studies for exogenous spatial attention [21, 23]. In our (exogenous) spatial cue condition, just as withourno-task color-feature cue (condition 1), our observers were not required to perform a task on the cue. After the experiment, observers reported that they felt the spatial cue distracting, whereas they did not feel the same for the feature cue. These subjective reports may indicate that, because there was no attentional task on the cue, the mere presence of the central color patch might not be strong enough to effectively engage feature-based attention.

Researches have reported that the time course of feature-based attention may be different from that of spatial attention. The effects of feature-based attention may develop more slowly than those of spatial attention [13]. Therefore, the cue lead-time of 110 ms in experimental condition 1 may have been too short for feature-based attention to manifest its effects. Thus, we used longer cue lead times and included an attentional task to more effectively manipulate observers’ attention in the TOJ with attentional task cases in experimental conditions 2, 3 and 4. Observers’ performances on the cue task were at least 70% accurate, which suggests that observers did pay attention to the cued feature. As shown in Figs 2b, c, d and 3b, c, d, a different pattern of results was observed than when there was no attentional task on the cue. A significant left shift of PSS was observed when attending to red color versus when attending to green color. The same observers who did not show an effect for condition 1, did show an effect for condition 2 (or 3 or 4).The average PSS values were -13 ms and +10 ms for the attend-to-red and attend-to-green conditions, respectively. That is, when attention was directed to the red color, the green bar needed to be physically presented about 13 ms prior to the red bar for the two bars to be perceived as simultaneous; when attention was directed to the green color, the red bar had to lead by about 10 ms in order to perceive simultaneity. The magnitude of the effect is relatively small, comparable to that from endogenous spatial attention, but smaller than that observed from exogenous spatial attention [23, 21]. These results offer additional evidence that if there is a PSS shift in the TOJ task from endogenous attention, the effects are smaller than those from exogenous attention.

There is a bias in favor of the green target being reported as perceived first, as seen in Figure 3a (all PSS values are positive). If this baseline bias is taken into account, then it appears that there is an asymmetric attentional effect: the PSS effect when attending to red is sizable, in contrast to the small effect when attending to green. Nevertheless, despite this bias, one can still assess the magnitude of an overall PSS shift as the difference between the “attend-to-red” and “attend-to-green” conditions.

A crucial question then is whether the shift of PSS in the TOJ with attentional task condition can be attributed to an attentional prior-entry effect. One may argue that the shift of PSS could also be explained by possible response and decision biases toward the attended color. Even though observers were required to base their responses on the orientation, instead of the color of the objects (i.e. the response dimension is orthogonal to the cue dimension), it is still possible that observes may tend to bias their judgment toward the object with the attended color. Results from the TOJ paradigm in prior-entry studies have been criticized because of the possible confounding from response and decision biases, although some recent studies have demonstrated that the biases cannot account for the shifts of PSS observed in TOJ tasks [19, 34].

To minimize subjective biases, the SJ task is a favored approach by some researchers to investigate the prior-entry hypothesis [21, 22]. Studies using the SJ task have provided evidence for the existence of prior-entry effects for spatial attention [39,40]. For instance, Zampini et al. (2005) found a prior-entry effect for visualauditory pair of stimuli (a 14 ms shift of PSS) using the SJ task, however this effect is smaller than the effect of 30 ms found in the Sternberg et al. (1971) study using a detection task instead of a judgment task. Schneider and Bavelier (2003) also showed that the shift of PSS is much smaller in an SJ task as compared to the shift in a TOJ task from exogenous spatial attention; their experimental data show no significant PSS shift in an SJ task from endogenous spatial attention. In the current study, we observed a minimal, non-significant, shift of PSS between the attend-red and attend-green conditions (2 ms) in the SJ task. The result from the SJ task does not support the existence of a prior-entry effect, although the trend of the result is consistent with a prior-entry effect, and is consistent with findings of the previous study [29]. The use of "go"/"no go" task in the Theeuwes and Van der Burg study (2013) might have provided a necessary condition to show the prior-entry effect with the SJ paradigm.

Finally, it must be noted that temporal order judgment and simultaneity judgment may not measure the same aspect of temporal processing [31, 32, 35]. Theoretically, the simultaneity judgment task emphasizes perceiving the synchronization of the two stimuli, whereas the temporal order judgment measures the ability to discriminate the successiveness. In this regard, these two different paradigms may be more appropriate for different studies, depending on the research question; and the SJ-related mechanism may be less sensitive than a TOJ-related mechanism for studying the temporal resolution or for discriminating the arrival time of different objects.

In summary, the present study provides valuable data for examining the effect of feature-based attention on the temporal processing of successive visual stimuli, depending on whether or not observers engage attention to the cue; these results with the inherently endogenous feature-based attention afford a comparison to results of previous studies on the effect of endogenous and exogenous spatial attention on the temporal processing of stimuli.