UC Davis UC Davis Previously Published Works Title An eye tracking investigation of color-location binding in infants' visual short-term memory. Permalink https://escholarship.org/uc/item/2pr037z5 Journal Infancy : the official journal of the International Society on Infant Studies, 22(5) ISSN 1525-0008 Authors Oakes, Lisa M Baumgartner, Heidi A Kanjlia, Shipra et al. Publication Date 2017-09-01 DOI 10.1111/infa.12184 Peer reviewed eScholarship.org Powered by the California Digital Library University of California An eye tracking investigation of color-location binding in infants’ visual short-term memory Lisa M. Oakesa,b, Heidi A. Baumgartnera,b, Shipra Kanjliab, Steven J. Lucka,b a Department of Psychology and bCenter for Mind and Brain University of California, Davis RUNNING HEAD: BINDING IN INFANTS’ VISUAL SHORT-TERM MEMORY This research and preparation of this manuscript were made possible by NIH grants R01HD49840 and R01EY022525 awarded to LMO and NIH grant R01MH076226 awarded to SJL. We thank Fred Barrett for his help with the data analysis, and the students and staff in the Infant Cognition Laboratory at the University of California, Davis, for their help with data collection. Heidi Baumgartner is now at Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, and Shipra Kanjlia is now at the Department of Psychological and Brain Sciences, Johns Hopkins University. Correspondences should be addressed to Lisa M. Oakes, Center for Mind and Brain, 267 Cousteau Pl., University of California, Davis, CA 95618 (e-mail: lmoakes@ucdavis.edu) An eye tracking investigation of binding, page 2 Abstract Two experiments examined 8- and 10-month-old infants’ (N = 71) binding of object identity (color) and location information in visual short-term memory (VSTM) using a one-shot change detection task. Building on previous work using the simultaneous streams change detection task, we confirmed that 8- and 10-month-old infants are sensitive to changes in binding between identity and location in VSTM. Further, we demonstrated that infants recognize specifically what changed in these events. Thus, infants’ VSTM for binding is robust and can be observed in different procedures and with different stimuli. An eye tracking investigation of binding, page 3 An eye tracking investigation of binding in visual short-term memory in infancy Binding object features is a central function of human memory systems. We do not simply remember that there was a red color and a round shape, but we remember that there was a red ball. We do not only remember what our coffee cup looked like and where we have seen coffee cups, we remember where we last saw our own coffee cup. Such binding of features allows us to recognize and categorize objects, organize memories, and use stored information to coordinate actions such as making eye movements, reaching, and navigating through an environment. It is therefore not surprising that a number of investigations have examined binding processes in both adults’ memory systems (Cowan, Naveh-Benjamin, Kilb, & Saults, 2006; Luck & Vogel, 1997; Ranganath, 2010; Vogel, Woodman, & Luck, 2001; Wheeler & Treisman, 2002) and infants’ memory systems (Feigenson & Halberda, 2004; Kaldy & Lee, 2003; Richmond, Zhao, & Burns, 2015; Slater, Mattock, Brown, Burnham, & Young, 1991). Despite this interest, we have only begun to understand infants’ ability to store representations of bound objects in visual short-term memory (VSTM). A deep understanding of infants’ ability to bind information in VSTM is important for several reasons. First, the properties of VSTM in infancy will have a large impact on how infants interact with and learn about the world. Storing bound information in VSTM is essential for gaze control (Hollingworth & Luck, 2009) and for recognizing continuity in visual information across saccades and eye blinks (Hollingworth & Henderson, 2002; Irwin, 1991), skills that are critical for learning about the visual world. Infants’ learning depends on such skills; therefore, confirming that infants can bind surface properties such as color with location information is an important goal of the present investigation. A second reason why it is important to examine binding in VSTM in infancy is that there An eye tracking investigation of binding, page 4 is increasing evidence that this ability develops across the lifespan. For example, the ability to store bound representations in VSTM appears to increase between childhood and adulthood (Brockmole & Logie, 2013; Cowan et al., 2006), but we know relatively little about the developmental origins of this ability in infancy. Adding to our understanding of binding in infancy was our second goal. There is some evidence that infants bind information in memory representations. In studies in which bindings were learned over successive trials (therefore tapping longer-term memory systems), infants between 4 and 12 months show evidence of learning bindings—and responding to a violation in binding—although the evidence varies in terms of the kinds of features and the task (Mareschal & Johnson, 2003; Newcombe, Huttenlocher, & Learmonth, 1999; van Hoogmoed, van den Brink, & Janzen, 2013). Indeed, developmental changes in the ability to bind features to objects stored in memory has been suggested as one possible mechanism underlying age-related changes in infants’ performance on tasks that require them to remember information about hidden objects (e.g., Wang & Baillargeon, 2005; Wilcox & Schweinle, 2002). Infants’ increasing ability to remember physical properties of hidden objects (such as height) and their individuation of hidden objects based on featural information may reflect, at least in part, increases in their ability to bind features to object representations. Researchers also have uncovered binding in working memory (WM) tasks in infancy. In these tasks, infants must maintain from trial to trial an active memory of where an object is hidden (Kaldy & Lee, 2003; Kaldy & Leslie, 2005). In such tasks, 6- to 8-month-old infants show evidence of storing information for bound features of visually presented stimuli. Despite this interest in binding in infancy, there is little work examining this process in infants’ VSTM. To be clear, WM and VSTM may be closely related, but these terms do not refer to the An eye tracking investigation of binding, page 5 same memory system (see Astle & Scerif, 2011 for a discussion). VSTM is defined by temporal parameters—information is stored rapidly and is maintained for brief periods of time being replaced when other information is stored (Luck & Hollingworth, 2008). WM, in contrast, is defined by the maintenance of information so it can be used in another task (Baddeley & Hitch, 1974)—for example, storing visual information to make comparisons between two items that cannot be simultaneously foveated. Thus, evidence regarding the development of binding in WM does not necessarily inform us about the development of binding in VSTM. Because cognitive abilities are not used in isolation (Oakes, 2009), some performance on WM tasks likely reflects the rapid storage and relatively brief maintenance of information in VSTM, and some performance on VSTM tasks likely reflects maintenance of information in WM so that it is available for use in other tasks. It is therefore not surprising that similar findings may be observed in studies examining WM and VSTM in infancy. In the present investigation, we focus on VSTM because it has not yet been established that the memory systems used in the tasks described here meet the criteria of a working memory (Baddeley & Hitch, 1974), but they have been designed to be within the parameters of visual short-term memory (Luck & Hollingworth, 2008), regardless of whether that memory is also a WM (Kibbe, 2015). Oakes et al. (2006) reported one study in which binding in VSTM was examined using a simultaneous streams change detection task, illustrated in Figure 1A. Oakes et al. found that both 7.5- and 12.5-month-old infants could store bound color-location representations in VSTM, whereas 6-month-old infants could not. The simultaneous streams task was designed to require VSTM by using very short stimulus presentations (500 ms) and retention intervals (300 ms); infants view arrays of 3 different colored rectangles that repeatedly cycled on (for 500 ms) and off (for 300 ms) for the duration of a trial (20 s). This timing was chosen based on findings with An eye tracking investigation of binding, page 6 adults that representations in VSTM are created very rapidly—as quickly as 50 ms per item (Vogel, Woodman, & Luck, 2006; Zhang & Luck, 2009)—and can be maintained for at least a few seconds. Moreover, the stimuli are constructed to reduce the involvement of long-term memory (e.g., random selection of items from a small set of feature values to maximize proactive interference). Thus, this task better isolates VSTM than other tasks used with infants. Conclusions about infants’ VSTM are drawn from their looking at two simultaneously presented stimulus streams, presented side-by-side: a non-changing stream (in which the array is identical on each cycle) paired with a changing stream (in which some aspect of the array changes on every cycle). Because infants generally prefer novel, more complex visual arrays to familiar, less complex visual arrays (Brennan, Ames, & Moore, 1966; Rose, 1981), infants should prefer the changing stream, but only if they can encode into VSTM the information needed to detect the changes. In the Oakes et al. (2006) study, the changing stream consisted of color-location swaps; each array contained the same colors and the same locations, but the bindings changed from cycle to cycle. Detecting changes in this version of the paradigm requires that color-location bindings in one array are stored in VSTM so that changes in these bindings can be detected in the next array. A preference for the changing stream relative to the nonchanging stream is evidence that infants detected the violation in the bindings. Oakes et al., found such a preference in 7.5- and 12-month-old infants but not in 6-month-old infants (these younger infants looked equally at the changing and non-changing streams). However, this is just one study of infants’ VSTM for bound information, and it is important to determine whether these findings generalize across stimuli, procedures, and tasks. Determining whether the findings are general was a third goal of the present investigation. An eye tracking investigation of binding, page 7 ---------------------------------Figure 1 about here ---------------------------------One limitation of the simultaneous streams task is that it differs in important ways from the tasks typically used with adults. Although the timing of the cycles is similar to that in adult procedures, infants are presented with multiple cycles for a period of many seconds, and their average preference over that entire period is recorded. This raises the possibility that a representation of the non-changing array gradually builds up in a longer-term memory system over a period of many seconds. Older children and adults, in contrast, are typically tested with a procedure in which they see a single cycle—one sample array, followed by a brief retention delay, followed by a single test array—in which they must make a judgment about whether or not a change has occurred (Cowan et al., 2006; Simmering, 2012). This eliminates any opportunity to use longer-term memory systems. Oakes and colleagues (2013) therefore developed a one-shot change detection task to use with infants (see Figure 1B). Like adult versions of the task, this one-shot change detection task involves presenting infants with a single sample array, followed by a brief retention delay, and then finally testing their memory for the items in the initial sample array during a single test array in which one (or more) of the items have changed. Utilizing eye tracking methods, infants’ sensitivity to changes is evaluated by comparing their fixation of changed items relative to unchanged items. Thus, this procedure is a purer measure of VSTM than the simultaneous streams task, and we used this procedure in the present study to provide converging evidence of infants’ ability to represent bound objects in VSTM. The one-shot task also allows us to establish not only that infants detect that a change has An eye tracking investigation of binding, page 8 occurred, but also that they detect precisely what has changed. In the simultaneous streams task, conclusions are drawn from how long infants look at the stream as a whole, and whether they watch changing streams for longer durations than non-changing streams. Therefore, this procedure provides no information about whether infants detected which items changed on a given cycle. Thus, this task cannot determine whether infants actually detected what changed. Adult participants, in contrast, can report not only whether a change has occurred but can also report the precise identity of the object that had appeared at a given location (Zhang & Luck, 2008), which provides very direct evidence of feature-location binding (and occasional misbindings; Bays, Catalao, & Husain, 2009). Because the one-shot task involves examining which items are fixated by the infants on a given trial, it allows us to determine whether infants can detect what has changed. In other words, if the infant looks more at a changed item than at an unchanged item, this indicates that they have detected which item has changed and not just the fact that something has changed. In addition, the high temporal resolution of the eye tracker can be used to track the precise time course of the process by which infants compare their memory for the sample array to the test array, and then orient attention to the changed item. Being able to assess infants’ sensitivity to specific changes opens the possibility to answer many new questions. For example, adults’ performance on tasks that require localizing color changes is strongly predictive of their broader cognitive abilities (Fukuda, Vogel, Mayr, & Awh, 2010; Johnson et al., 2013). Demonstrating that we can evaluate similar abilities in infants is an important first step to looking for similar relations between VSTM and other cognitive abilities. We conducted two experiments examining 8- and 10-month-old infants’ detection of color-location bindings in the one-shot change detection task. The two experiments were aimed An eye tracking investigation of binding, page 9 at demonstrating both that infants can detect changes in color-location bindings and that they recognize which items have changed in the arrays. We conducted two different experiments, with different numbers of items, arrangements of items, and trial types, to verify that the obtained results are general and not specific to one age, sample, or stimulus configuration. In Experiment 1, we assessed 10-month-old infants’ color-location bindings in 3-item arrays, using color swaps to assess binding in a manner that was closely analogous to the Oakes et al. (2006) study. In Experiment 2, we assessed color-location binding in 8- and 10-month-old infants using 2-item arrays and a different method for manipulating binding. Experiment 1 Experiment 1 assessed 10-month-old infants’ color-location bindings using arrays of 3 items in which some of the colors “swapped” locations over the course of the trial. These arrays are similar to those used by Oakes et al. (2006) and would confirm that the evidence of binding observed in that previous study are obtained in this one-shot task, which is a purer assessment of VSTM. On each trial in Experiment 1, there was a brief sample period (533 ms) with an array of 3 different colored items, followed by a brief retention period (300 ms), and then a test period in which the 3 items reappeared. Note that the same color values were present during the sample and test arrays, but two of the items swapped locations. Thus, during the test array, the locations of two of the color values changed (e.g., yellow and green in Figure 2) and the location of the third color value remained the same from sample to test (black in Figure 2). If infants remembered only the specific colors (and not where those colors were located), all three items would be equally familiar. However, if infants remembered the color-location bindings, one item in the test array was unchanged relative to the sample and the other two items were changed. In this case, the two items in which the color-location binding had changed (i.e., the two items that An eye tracking investigation of binding, page 10 swapped locations) would be more novel than the unchanged item. Therefore, these two items were the target items in the test array. Method Participants. We screened potential participants for familial risk for colorblindness (i.e., we excluded male infants with maternal familial colorblindness), prematurity (i.e., we excluded infants born more than 21 days before their due date), history of neurological problems or other chronic illness (i.e., we included only infants without chronic health problems). In both experiments, our final analyses included only data from healthy, full-term infants with no known history of vision problems and who were not at risk of colorblindness. We tested infants who met our criteria until we achieved our final sample of 20 10-month-old infants (M = 305.75 days, SD = 6.08; 14 girls). Fifteen infants were White, 4 were of mixed race, and 1 was Native Hawaiian or Pacific Islander. Two infants (1 mixed race, 1 White) were Hispanic. All mothers had at least some college, and 13 mothers had at least a 4-year degree. We tested an additional 6 infants met our inclusion criteria but were not included in the final analyses due to a failure to calibrate the eye tracker (n = 2), equipment malfunction (n = 2), or a failure to contribute at least four trials with useable data (see data processing section below; n = 2). Infant names were obtained from the State Department of Public Health. Parents who lived within 30 miles of our lab were sent a mailing with information about our research, and were given instructions about how to volunteer if they were interested in participating. Parents who responded were contacted by phone or e-mail when their infant reached the appropriate age for this study, and they were offered the opportunity to volunteer to participate. Infants were given a small toy, book, or t-shirt and a certificate for participating. Infants participated in only one of the two experiments reported here. An eye tracking investigation of binding, page 11 Apparatus. The apparatus was identical to that used by Oakes et al. (2013). We collected infants’ eye movement data using an Applied Science Laboratory (ASL) pan/tilt R6 eye tracker with a sampling rate of 60 Hz, controlled by a Dell computer. An eye camera, centered in front of and below a 37-inch (94 cm) Westinghouse LCD monitor (16:9 aspect ratio) on which the stimuli were presented, was focused on the infant’s right eye and tracked point-of-gaze (POG) using the pupil and corneal reflections of an infrared light source. Maintaining focus on the eye was assisted by a magnetic head tracker (Ascension Flock of Birds) affixed to a headband and positioned above the infant’s right eye, allowing the camera to relocate the eye (if the track was lost) by transmitting the location of the infants’ head in space. During the session, an experimenter could manually adjust the camera as needed. A smaller, wide-angle camera was connected to the eye camera and provided a full view of the infant for the experimenter on a separate monitor. Stimuli were presented on the stimulus monitor via a second Dell computer running custom scripts created in Adobe Director. This computer also sent codes specifying trial information (e.g., start and end of a trial, stimulus information, etc.) to the eye tracking computer that were recorded along with the POG data. These codes allowed for the integration of infants’ gaze data with the stimulus information for later analysis. Stimuli. The stimuli were arrays of three rectangles, each 18-cm wide by 13-cm high (10.2° by 7.4° visual angle at a viewing distance of 100 cm), arranged in a triangular configuration (see Figure 2). One item was 11 cm (6.3˚) directly above the fixation point; the other items were to the left or right and slightly below the fixation point (each approximately 7.5 cm, or 5˚ from the fixation point). The color of each rectangle on each trial was randomly chosen (without replacement) from a set of eight possible colors with the following CIE [x, y, An eye tracking investigation of binding, page 12 luminance] coordinates: red [0.63, 0.34, 28.7], green [0.26, 0.54, 25.4], brown [0.44, 0.43, 28.9], yellow [0.38, 0.48, 209], orange [0.47, 0.43, 89.9], black [0.28, 0.34, 0.9], cyan [0.22, 0.33, 94.1], and violet [0.22, 0.13, 27.6]. Short clips of music by Bach, Mozart, Pachelbel, and Ravel accompanied stimulus arrays to keep infants engaged in the task. ---------------------------------Figure 2 about here --------------------------------In addition to these stimulus arrays, we had a collection of brief animated clips taken from children’s television (e.g. Sesame Street, Blues Clues, Teletubbies), a series of pictures of babies accompanied by music, an animated movie of two animals singing “The Lion Sleeps Tonight,” and an animated clip of shapes moving randomly across the screen accompanied by beeping. We periodically played a series of these clips between trials to reengage infants if they were fussy or distracted (e.g., looking at their feet or at their parent’s face). Procedure and design. The procedures were essentially the same as those reported by Oakes et al. (2013) with the exception of the stimulus arrays. The infant sat on a parent’s lap approximately 100 cm away from the stimulus monitor and 75 cm from the eye camera. Parents were seated on a chair without wheels to minimize movement and wore felt-lined sunglasses to occlude their view of the stimuli during the experiment to prevent bias. Two highly-trained experimenters, one controlling the eye tracking system and the other controlling the stimulus presentation, ran each experimental session. Experimenters sat behind a partition in the testing room and were not visible to the parent or infant during the experiment. The eye tracker was adjusted to the infant’s eye and a standard 2-point calibration was employed (see Oakes et al., 2013 for details). Immediately after calibration, the stimulus An eye tracking investigation of binding, page 13 experimenter initiated the experimental session. Prior to each trial, a fixation cross flashed at the center of the screen to attract the infant’s gaze. Once the infant fixated the cross, the stimulus experimenter pressed a computer key to initiate the trial. A sample array consisting of three distinctly-colored rectangles was presented for 533 ms, followed by a 300-ms delay period, which was followed by the test array in which the rectangles reappeared for a 3000 ms test period. In the test array, two of the three rectangles had “switched” locations (relative to the sample array), and the third item was unchanged from sample to test. Our custom program randomly determined on each trial which three colors to use and which items were to be swapped. We presented up to 64 experimental trials; we stopped the procedure when the infant became too fussy or uninterested to continue or when 64 trials had been completed. Our experimental procedure and equipment enabled flexible interaction with the infant through the use of attention-getting stimuli and a subject-controlled trial pace. This allowed us to sustain infants’ interest and engagement while minimizing fussiness and inattentiveness. Data processing. The ASL eye tracking system recorded the horizontal and vertical coordinates of infants’ gaze position at a sampling rate of 60 Hz. To reduce noise in the fixation data, the point of gaze (POG) at each of these time points was averaged with the POG of the three previous time points to provide a running average of four time points, a process that delayed the recording of gaze data relative to the exact gaze time by approximately 25 ms (or 1.5 time points). In addition, we used a blink filter to discriminate between blinks (defined as pupil loss for less than 12 time points) and loss of gaze position that extended beyond 12 time points. The missing points were interpolated for blinks but not for longer periods of data loss. Due to differences in the rate at which different information is sent and received by An eye tracking investigation of binding, page 14 computers, monitors, and so on, there was a slight discrepancy (33 ms ± 8.33 ms) between when the ASL system recorded the codes pushed by the stimulus presentation computer (indicating the start of a trial or that a change had occurred) and when the corresponding stimuli actually appeared on the monitor. All analyses reported here include timing adjustments to account for these minor discrepancies. We used custom Matlab scripts to determine the amount of time infants spent looking within specified rectangular areas of interest (AOI). Separate AOIs were created for each of the three items in the stimulus arrays as well as the area covered by the fixation cross before the sample display. The AOIs were 23.5-cm w by 17-cm h (13.4˚ by 9.7˚), approximately 30% bigger than the item to account for imprecise calibrations and use of parafoveal vision to perceive color. Inclusion criteria. We used the same criteria for including trials in the current analyses as those used by Oakes et al. (2013): trials were included if (1) the total duration of looking within the AOIs during the sample and delay periods (combined) was at least 100 ms (thus excluding trials in which infants could not have encoded any item during the sample), and (2) the infants directed their gaze toward the items for at least 200 ms over the course of the test period (thus including only trials in which infants made at least one clear fixation to the AOIs during the test period). Only trials that met both criteria were included in the final analyses, and any infant who had fewer than four trials that met these criteria was excluded from the analyses. Infants completed an average of 25.35 trials (SD = 15.21, range 11 to 58) that met our inclusion criteria. Two infants were excluded for failing to contribute at least four trials to the analyses (as described in the Participants section earlier). Analysis window. As observed previously in the one-shot task (Oakes et al., 2013), infants An eye tracking investigation of binding, page 15 tended to lose interest in the stimulus arrays over the 3000-ms course of the test period. Thus, we limited our analyses to the period of time during which infants were most engaged in the task, as indicated by looking toward the AOIs for the three test items rather than looking elsewhere (e.g., at other parts of the screen or away from the screen entirely). To determine an objective analysis window, we evaluated each time point on each trial (across infants) and selected the time interval in which at least 50% of the infants’ looking was directed towards the AOIs for these three objects. First, we identified the proportion of trials (across all subjects) with gaze data recorded to one of the three item AOIs at each time point (i.e., every 16.7 ms). Next, we determined the point at which the percentage of trials that had recorded gaze data to one of the three AOIs dropped below 50%--that is, the time point at which only there was gaze data to one of the AOIs for half of all trials for all infants. In this experiment, this occurred at 1733.33 ms after the onset of the test array. We used this time point as the end of our analysis window. The start of our analysis window was 200 ms after the change occurred. We used this start time to account for the time required to execute an eye movement in response to the onset of the test array. Little is known about the timing of infant eye movements; we therefore used a conservative estimate based on the adult literature (Hyun, Woodman, Vogel, Hollingworth, & Luck, 2009). Infants in Oakes et al. (2013) appeared to take, on average, between 300 and 500 ms to look at the target. Thus, it seems likely that infants take even more than 200 ms to make an eye movement, and our criterion for defining the start of the analysis window is very conservative. Finally, using this analysis window we further inspected the amount of looking on each trial, and excluded from the analyses any trials with less than 200 ms of looking during this truncated analysis window. These criteria are identical to those used by Oakes et al. (2013). Results An eye tracking investigation of binding, page 16 Our data stream indicates at each time point on each trial whether or not the infant was looking at one of our AOIs. We use this data stream to calculate looking time by summing the number of time points for which the gaze was directed at one of the AOIs, and then multiplying the number of time points by 16.667 ms. Total looking time, therefore, is calculated by summing the number of time points directed at any AOI and multiplying that number by 16.667 ms. Looking time to a specific AOI is calculated by summing the number of time points directed to that AOI by 16.667. Note that we can calculate proportion scores (e.g., the proportion of time infants looked at an AOI either by dividing the number of time points directed to that AOI by the total number of time points directed to all AOIs combined, or by dividing the duration of looking to that AOI by the duration of looking directed to all AOIs combined. The point is that in these analyses the number of time points and the duration of looking reflect the same thing and are interchangeable. On average, infants were highly attentive in this task. During the 833.33-ms pre-change period (i.e., the sample array plus the delay period), infants looked for an average of 756.85 ms (SD = 59.78 ms) to the AOIs for all three items plus a fourth AOI in the center region (where the fixation cross was positioned before the start of the trial). During the 1533.33-ms analysis window of the test period (i.e., from 200 ms to 1733.33 ms after the test array onset, as defined earlier), infants looked towards the three object AOIs for an average of 1022.01 ms (SD = 169.82 ms). Thus, infants were engaged in the task, and they had ample time to perceive, process, and encode into VSTM the three colored rectangles presented during the 500 ms sample array (Catherwood, Skoien, Green, & Holt, 1996). Our primary analyses involved evaluating infants’ change preferences averaged across the whole analysis window of the test period. In addition, we conducted a more fine-grained An eye tracking investigation of binding, page 17 analysis of moment-to-moment changes in infants’ looking behavior across this period. Infants’ preference for the changed items during the test period. To compare infants’ preference for the changed items relative to the unchanged item during the test period, on each trial we calculated a change preference score by dividing infants’ looking time (calculated from the number of time points) to the two changed items by their looking time to all three items total. Because the test array had three items, infants should devote 33.33% of their looking to each item during test if they do not have a systematic preference (i.e., for the changed items). However, if infants store color-location bindings, they should look more at the two swapped items (more than 33.3% for each item and thus more than 66.7% for the two combined) and less at the one unchanged item. Therefore, a change preference score greater than 66.7% would indicate that infants preferred looking at the two items that changed from sample to test relative to the item that did not change. We calculated the change preference score for every trial and then obtained the median change preference score across trials for each infant. We used infants’ individual median scores because medians are less influenced by extreme values. Occasionally an infant’s preference score for a particular trial would be 0 or 1. The median minimizes the influence of such extreme scores in the analysis. To ensure that use of the median was appropriate, we calculated the skewness of each infants’ scores across trials, and then averaged the skew across infants. The mean skew was relatively low, skew = -1.03 (SD = .43), and thus it is appropriate to evaluate infants’ median scores. Our main analyses involved comparing to chance the average of individual infants’ median change preference scores. During the analysis window, 10-month-old infants devoted a very large proportion of their looking to the two changed items, M = 0.89, SD = 0.13, and their preference for those changed items was significantly greater than chance, t(19) = 7.99, p < 0.0001, d = 1.79. This An eye tracking investigation of binding, page 18 means that they spent approximately 45% of their time looking at each changed item and only 11% of their time looking at the unchanged item. Thus, infants preferred to look at a given changed item approximately four times as much as the unchanged item, averaged across the analysis window of the test period. This replicates previous results indicating that infants maintain color-location bindings in VSTM (Oakes et al., 2006). Because the infants looked at the changed items more than at the unchanged item, these results also indicate that they not only recognized that there was a change, but they also recognized specifically which items had changed. Moment-by-moment gaze behavior toward the changed and unchanged items during test. Next, we used the analytic approach introduced by Oakes et al. (2013) to determine how preferences for the changed and unchanged items developed over the course of a trial. The location of each infant’s gaze was assessed at each individual 16.667-ms time point (i.e., each sampling period of the eye tracker) on each trial. If the infant was looking at one of the items that changed, that time point was assigned a 1; if the infant was looking at the unchanged item, that sample was assigned a 0; if the infant was not looking at any of the items, that sample was assigned as null (note this is essentially the same as the assignment of the looking time above; on each time points, infants’ looking at the AOIs was indicated and counted; and time points in which infants failed to look at any of the AOIs were discarded from the analysis).. Next, we computed across trials for each infant the change preference score at each time point (i.e., the change preference for the first 16.67 ms time point, the second 16.67 ms time point, and so on across the analysis window) by averaging all non-null values at each time point (sampling period) across trials. We used these individual scores to calculate a group change preference score for each time point (see Figure 3; the curve represents the group change An eye tracking investigation of binding, page 19 preference score, and the vertical lines indicate the 95% confidence interval at each time point). As shown in Figure 3, the preference score was near chance during the period of the sample array and delay (because the change had not yet happened) and during the first 200 ms following the onset of the test array. However, infants’ preference for the changed items systematically increased within 300 ms after the onset of the test array, and they maintained that preference for many time points (the shaded area in Figure 3). As an initial assessment of the statistical significance of the change preferences at each time point, we conducted t tests comparing the average change preference score at each time point to chance (.67). Time points at which the change preference score was significantly different than expected by chance (p < .05) are indicated by dots below the curve in Figure 3. It can be seen that none of the pre-change time points had a change preference score that differed from chance, but the average change preference scores were significantly different from chance at every time point between 250 ms and 1483.33 ms after the onset of the test array. However, these t tests were not corrected for multiple comparisons, so they should be treated with caution. ---------------------------------Figure 3 about here --------------------------------To correct for multiple comparisons, we took advantage of the fact that gaze tends to remain relatively stable for many consecutive time points. Thus, we evaluated whether the number of consecutive time points with significant change preference scores (the run length) was different than expected by chance. To estimate the distribution of maximal run lengths that would be expected by chance, we used a resampling approach in which the actual data are subject to random permutations (which randomly shuffle by trial the particular time points assigned as An eye tracking investigation of binding, page 20 fixated on changed or unchanged items); we conducted 1000 iterations of these permutations and calculated the maximal run length on each iteration (see Oakes et al., 2013 for complete details). The result of these permutations is an empirical estimate of the null distribution of maximal run lengths. Observed run lengths that are beyond the 95th percentile of this null distribution are considered significantly longer than expected by chance (p < 0.05). The null distribution obtained for this experiment is depicted in Figure 3B. This distribution indicates that run lengths of 14 or more consecutive time points with significant change preference scores would occur less than 5% of the time by chance. Thus, an observed run length of 14 time points or longer is very unlikely to occur by chance and is considered statistically significant at the .05 level. The 10-month-old infants in our sample showed a run of 75 consecutive time points (or 1250 ms) beginning 250 ms after the onset of the test array. This is substantially longer than would be expected by chance, indicating that it is a statistically significant preference, and that these infants showed strong and consistent preference for the color-location binding switch items. Discussion Ten-month-old infants showed a strong preference for the locations that swapped colors compared to the location that did not change colors, indicating that they stored color-location bindings in VSTM after a single 533-ms exposure. These results provide a conceptual replication of the results reported by Oakes et al. (2006) in a study using the simultaneous streams task. Thus, one important contribution of these findings is that the evidence of binding in VSTM by infants in the second half of the first postnatal year is replicable and can be obtained in different procedures. However, the present results also go beyond the previous data by demonstrating that infants not only detected that a change occurred, but also detect precisely which items changed. An eye tracking investigation of binding, page 21 That is, because infants looked at the changed items significantly more than at the unchanged item, they must have had specific information about which items changed rather than just a vague sense that something had changed. The high temporal resolution of the eye tracking data also made it possible to assess the temporal dynamics of the change preference. A significant preference for the changed items was present starting 250 ms after the onset of the test display, indicating that the infants rapidly detected that the two items swapped locations. This is only about 50 ms slower than the amount of time required for adults to make a saccade to a completely new color when explicitly instructed to do so in a change detection task (Hyun et al., 2009). Thus, the detection of colorlocation binding changes is quite rapid in 10-month-old infants. In addition, we observed that the significant preference for the changed items lasted for more than 1200 ms. Note that this is based on an average across infants and trials, and it does not imply that every infant maintained gaze on the changed items during this entire period on every trial. However, it does imply that infants were able to detect changes in color-location bindings rapidly on at least of subset of trials and maintained their preference for the changed items for a substantial period of time. One may be concerned that the results reported here may reflect the influence of apparent motion (AM) on infants’ eye movements, rather than the detection of a binding change. Apparent motion is the phenomenon in which perceivers see two successive presentations of an object as that object in motion. Although the many aspects of our stimuli are similar to those used in AM studies, our stimuli differ in important ways. Examples of AM and our stimuli are provided here (https://osf.io/s6nvt/?view_only=cc17ef9ec12948a486a9cedd39a7d0a6), and it can be seen that our stimuli do not give the impression of AM. The fact that in our stimuli, one item moves to An eye tracking investigation of binding, page 22 occupy the location of another item of a different color likely disrupts AM, as dissimilarity in color disrupts AM in adults (PapaThomas, Gorea, & Julesz, 1991). Moreover, long-range apparent motion is weak, even at distances that are a fraction of the distances apparently traveled by our objects (3 degrees versus 15 or more degrees) (Cavanagh, Arguin, & von Grünau, 1989). Finally, although it is possible that AM is observed in a single cycle, as in our stimuli, when AM has been observed in adults subjects are shown repeated cycles of the objects being displaced (Kramer & Yantis, 1997; Pantle & Picciano, 1976; Prazdny, 1986). For these reasons we believe it is unlikely that our findings are due to infants’ detection of apparent motion in these displays. Experiment 2 The use of three-item arrays in Experiment 1 allows us to compare our results to those reported by Oakes et al. (2006) using three-item arrays in a simultaneous streams task. Experiment 2 was another conceptual replication of the work by Oakes et al. To test the generalizability of the findings of Experiment 1, we examined infants’ sensitivity to binding in two-item arrays using a somewhat different approach for assessing color-location binding. It is not at first obvious how one could assess infants’ sensitivity to binding with a set size of two. In Experiment 1, two out of three items swapped locations, and we asked whether infants preferred the two swapped items to the unchanged item. However, at set size two, swapping the color-location bindings would result in a display in which both items had changed, making it essentially impossible to evaluate sensitivity to a change relative to no change. We solved this problem by creating trials in which a binding change was paired with another type of change—in this case, an identity change (see Figure 4B). Thus, on some trials, both items changed from sample to test, but they changed in different ways. One item (the left item in the test array of figure 4B) had a new identity—a color not seen during the sample. The other item (the right item An eye tracking investigation of binding, page 23 in the test array of figure 4B) had a familiar identity, but it was in a new location, thus violating the color-location binding. If infants are sensitive to both types of changes, then they will detect that both items have changed and they should show no preference for one item over the other. That is, if infants show equal looking at a binding change and at an identity change, it would indicate that they have stored the bindings in VSTM. But, infants may remember only the colors (and not the color-location bindings), or they may be less sensitive to a binding change than to an identity change. In either of these cases, infants should prefer the new identity. Note that this design raises another problem—evidence of binding would be a null preference. We addressed this issue by intermixing the binding trials just described with trials in which a change preference is unambiguous. Specifically, we included color change trials identical to those used by Oakes et al. (2013) in their demonstration of VSTM using the one-shot task. In these trials, one item remained unchanged from sample to test, and the other item had a new identity during test (see Figure 4A). Showing a preference in these trials requires only detecting a new identity, and a significant change preference on these trials would provide confirmatory evidence in support of the finding reported by Oakes et al. Moreover, a preference for the changed item in these trials—regardless of how infants respond on the bind change trials —will confirm that these infants show a preference for a changed item. Thus, infants’ performance on these trials will provide a context for evaluating any lack of preference obtained for the binding trials. Specifically, if infants find binding changes to be compelling, we would expect their preference for the new color to differ in the two types of trials; they should exhibit a more robust preference for the color-change item when the other item in the test array is a familiar color in a familiar location than when the other item in the test array is a familiar color in a new location. An eye tracking investigation of binding, page 24 Method Participants. In this experiment we tested both 8- and 10-month-old infants. We included 10-month-old infants to allow a comparison to Experiment 1. We included 8-month-old infants to allow a direct comparison to Oakes et al. (2006), who demonstrated binding in infants between 7 and 8 months of age. Using the same inclusion criteria as in Experiment 1, we tested infants until we had achieved our final sample of 25 8-month-old (M =244.48 days, SD = 7.01, 12 girls) and 24 10-month-old (M = 306.5, days, SD = 6.21, 11 girls) infants. Thirty-five infants were White, 6 were Asian, 2 were American Indian or Alaskan Native, 4 were mixed race, and race was not reported for 2 infants. Seven infants were reported to be Hispanic (4 were White, 1 was American Indian or Alaskan Native, and 2 did not have race reported). Forty-eight mothers had graduated high school and 30 mothers had at least a 4-year college degree. An additional 14 infants who met our inclusion criteria were tested but not included in the final analyses (n = 8 8month-old and n = 6 10-month-old infants) due to the infant becoming too fussy to continue the session (n = 3), inability to calibrate the eye tracker (n = 1), equipment malfunction (n = 2), or a failure of the infant to contribute at least four trials with useable data in each condition (n = 8; note this number is higher than in Experiment 1 because in Experiment 2 we required that infants contribute at least four trials of each type to be included in the analyses). Apparatus, Stimuli, and Procedure. The apparatus, stimuli, and procedure were nearly identical to Experiment 1, with just a few exceptions. First, in Experiment 2, the arrays consisted of two (rather than three) colored 18-cm w by 13-cm h (10.2˚ by 7.4˚ visual angle) rectangles, positioned side-by-side, each centered 7.5 cm (5°) to the left or right of the center of the monitor. Using two items allowed us to include feature-only trials that were an exact replication of Oakes et al. (2013), providing both converging evidence for that previous study and a strong baseline An eye tracking investigation of binding, page 25 for interpreting infants’ responding in this task. In addition, by using arrays of two items, we could directly compare infants’ responding to two items that changed in different ways. Although this would have been possible with a set size of three, by using a set size of two each item was unique in the way it changed, and therefore we could more unambiguously interpret the findings. The second difference was that we included two types of trials: Feature-Only and Feature+Binding trials. On Feature-Only trials (see Figure 4A), one of the two items (randomly selected) changed identity (color) from sample to test; thus, in the test array one of the items was identical in both color and location to one of the items in the sample array (e.g., the right item was orange in both the sample and test arrays), and the other item had a novel color value (e.g., the left item was green in the sample array and yellow in the test array). These trials were identical to those used by Oakes et al. (2013), who reported that 8-month-old infants preferred the changed item in this condition. Note that detecting a change here does not require that one encode bindings between color and location; one need only encode the particular colors and recognize when a new color value is present. By contrast, on Feature+Binding trials both items underwent a change (see Figure 4B). As in the Feature-Only trials, one item had a new color value during test (e.g., there was a brown item in the test array but not the sample array). The other item also changed; it represented a binding change from sample to test. Although the item in this location also changed color, it was the same color as the item in the other location had been in the sample array (e.g., the item on the left changed from green to orange, but the item on the right in the sample array had been orange). For this item, the color value was not new during test, but the color value in that location was changed from sample to test. An eye tracking investigation of binding, page 26 ---------------------------------Figure 4 about here --------------------------------We constructed a custom program in Director that on each trial selected the type of trial (Feature-Only or Feature+Binding), the initial color for each item from the same set of 8 colors used in Experiment 1, the location of the item undergoing a feature change (all trials) and the location of the item undergoing a binding change (Feature+Binding trials only), and the new color of the feature-changed item at test. Infants received up to 32 trials of each of the two types of trials (Feature-Only and Feature+Binding), intermixed throughout the session, for a maximum of 64 trials in total. The pre-change period was identical for the two types of trials— two distinctly colored rectangles were presented for 533 ms, followed by a 300 ms retention interval. The two types of trials differed in the post-change arrays as described earlier. Data processing. All aspects of data processing, exclusion criteria, and AOIs were identical to Experiment 1 with two exceptions. First, because there were only two items, we had only two AOIs. Second, infants needed to contribute at least four Feature-Only and four Feature+Binding trials to be included in the final analyses. Eight- and 10-month-old infants completed similar numbers of trials that met our inclusion criteria; 8 months: M = 28.82, SD = 16.31, and 10 months: M = 29.75 SD = 16.85; t(47) = 0.17, p = 0.86, d = 0.05. Analysis window. Using the procedure described in Experiment 1, we established that the percentage of trials on which infants directed their gaze to one of the two items in the test array dropped below 50% 1966.67 ms after the onset of the test array. We therefore limited our analyses to the window from 200 ms to 1966.67 ms after the onset of the test array. Results An eye tracking investigation of binding, page 27 During the sample and retention period, the overall amount of looking time was similar in the 8-month-old infants (M = 744.23 ms, SD = 61.57 ms) and the 10-month-old infants (M = 761.59 ms, SD = 42.84 ms), t(47) = 1.14, p = .13, d = .33. During the 1766.67-ms analysis window of the test period, 8-month-old infants looked somewhat less (M = 1064.51 ms, SD =220.35 ms) than did the 10-month-old infants (M = 1209.20 ms, SD = 203.62), t(47) = 2.38, p = 0.02, d = 0.68. Preference for the new color during the test period. For both types of trials, we examined infants’ new color preference by dividing their looking at the item that was a new color value by their total looking at the two items. If infants prefer a new color to a familiar color, and fail to respond to the binding change, then they will show a significant new color preference in both types of trials. However, if they respond to both the new color and the binding change, they will show a new color preference only in the Feature-Only trials. Thus, comparison of these two types of trials will allow us to examine differences in infants’ preference for precisely the same kind of change in two types of trials. Once again, we established that the infants’ distribution had low skew; average skew for 8-month-old infants was -.21, SD = .27, and average skew for 10-monthold infants was -.29, SD = .41. Thus it was appropriate to use median scores for our analyses of change preference to minimize the influence of extreme values. The average of infant’s median preference scores for each trial type are presented in Figure 5. Initial analyses revealed no differences between the 8- and 10-month-old infants in their preferences for the novel color. For simplicity of presentation, we therefore collapsed across age groups to compare preference for the new color in the two types of trials. A paired t-test revealed that infants’ preference for the new color was significantly greater in the Feature-Only trials (M = .64, SD = .18) than in the Feature+Binding trials (M = .53, SD = .20), t(48) = 2.77, p An eye tracking investigation of binding, page 28 = .008, d = .40. Thus, infants’ preference for a new color was significantly greater when the familiar color it was paired with did not change location from sample to test than when the familiar color it was paired with had been present in the sample array but in a different location, indicating that infants recognized both a new identity and a violation in color-location bindings. We also compared infants’ new color preference scores to chance (.50) for each type of trial. These comparisons revealed that this score was significantly greater than chance on Feature-Only trials, t(48) = 5.69, p < 0.001, d = .81 (combined across age groups; the scores were also significantly greater than chance for both 8- and 10-month-old infants separately). Infants did not prefer the new color more than expected by chance on Feature+Binding trials, t(48) = .93, p = 0.34, d = 0.13 (the scores did not differ from chance for either 8- or 10-monthold infants). Recall that in these trials both items have changed—they both represent a violation of the familiar color-location binding, and one of the items also has a new color feature. The logic of null hypothesis statistical testing does not allow strong inferences to be drawn about the lack of a significant difference, and this limits the conclusions that can be drawn from the fact that the mean preference score was not significantly different from chance on the Feature+Binding trials. We therefore computed the Bayes Factor using software at http://pcl.missouri.edu/bayesfactor (Rouder, Speckman, Sun, Morey, & Iverson, 2009). The Bayes Factor indicates the relative evidence for the null and alternative hypotheses. For the Feature+Binding trials, the Bayes Factor was 4.28 in favor of the null hypothesis. This means that the observed data were 4.28 times more likely to arise if the infants had an equal mean preference for the two items than if they preferred one over the other. By convention, this is considered “substantial” evidence in favor of the null hypothesis (Jeffreys, 1961). An eye tracking investigation of binding, page 29 ---------------------------------Figure 5 about here --------------------------------Moment-by-moment gaze behavior toward the new color value during test. We used the procedures described for Experiment 1 to examine infants’ looking behavior over the course of each individual trial. In Experiment 2, on each 16.67-ms time point of every trial, we assigned a code of 1 if the infant was looking at the new color value, 0 if the infant was looking at the old color value, or null if the infant was not looking at either square. The time-course data, presented in Figure 6, are consistent with the averaged preference scores. During the Feature-Only trials (Figure 6A) infants showed a robust and sustained preference for the new color, beginning approximately 350 ms after the onset of the test array. Averaged across subjects and trials, the new color preference score is significantly greater than chance for many successive time points, as indicated by independent t-tests comparing infants’ preference for the new color to chance (.50) at each time point (indicated by a dot under each significant time point in Figure 6). During the Feature+Binding trials (Figure 6B), in contrast, infants failed to show a significant preference for the new color at any point during the trial. The independent t-tests for each time point revealed that infants rarely showed a significant preference for the new color and they never significantly preferred the old value in the new location (which would be indicated by a new color preference significantly less than chance). On the Feature-Only trials, infants exhibited a run of 51 independently significant time points, beginning 350 ms after the change and continuing for 850 ms. The permutation analysis for these trials indicated that the 95th percentile of the null distribution was 18 time points, so the observed run length was significantly longer than expected by chance. On the Feature+Binding An eye tracking investigation of binding, page 30 trials, by contrast, the longest run of successive individually significant time points was 4 (beginning 266.67 ms post-change and continuing for only 66.67 ms). The permutation analysis for these trials indicated that the 95th percentile of the null distribution was 19 time points, so the observed run length was not different from chance on the Feature+Binding trials. ---------------------------------Figure 6 about here --------------------------------Discussion The results of Experiment 2 are important for several reasons. First, infants’ behavior on the Feature-Only trials replicate the result reported by Oakes et al. (2013). Thus, infants’ ability to detect a feature change in this task is robust and replicable by 8 months. Second, we observed that although infants showed a strong and robust preference for the changed item on FeatureOnly trials, they did not show a preference for that item in the Feature+Binding trials. In these trials both items changed—one item was a color not present during the sample, and the other item was a color previously seen in a different location during the sample. Thus, this other item represented a binding change. Across these trials, infants failed to show a significant preference for either type of change, suggesting that during the test period both the items were equally compelling. Because these same infants showed a significant preference for the new color on the Feature-Only trials, we can be confident that infants not only detect the change in color, but that they also would prefer that change when the other item present did not represent a competing change. Thus, this experiment provides converging evidence for the basic finding in Experiment 1, demonstrating that infants are sensitive to changes in color-location binding in a task that taps VSTM and therefore represent color-location binding in VSTM. Infants detected these binding An eye tracking investigation of binding, page 31 changes in both three-item arrays (Experiment 1) and two-item arrays (Experiment 2). Finally, because Experiment 2 included 8-month-old infants, these data confirm the findings of Oakes et al. (2006) that color-location binding in VSTM emerges by 8 months. General Discussion This study found that infants store bound representations in a task that provides a relatively pure assessment of VSTM, confirming previous conclusions from work using the simultaneous streams task (Oakes et al., 2006). In Experiment 1, infants preferred to look at test items that reflected a change in color-location binding rather than at an unchanged item, and in Experiment 2 infants found equally compelling an old color value presented in a new location (a binding change) and a new color (a feature change). We therefore confirmed previous findings that infants 8 months and older can store bound representations in VSTM. A number of previous results indicated that infants could bind location to item identity, but most of the work has not focused on VSTM. For example, Mareschal and Johnson (2003) familiarized 4-month-old infants with objects being repeatedly hidden by occluders. When the objects were manipulable toys, 4-month-old infants detected a change in the object-location binding, although they did not detect changes in binding for other kinds of stimuli. Newcombe et al. (1999) found that 5-month-old infants failed to detect a violation between location-object binding in a similar task, except when the objects were hidden in continuous space (e.g., in a sandbox) rather than behind two spatially distinct occluders. Van Hoogmoed and colleagues (2013) used an ERP oddball task and found that 12-month-old infants’ ability to detect changes in object-identity bindings in visual scenes was not fully developed. Importantly, in these previous studies, infants were given the opportunity to learn the location-identity relation over multiple trials. Thus it is not clear whether the memory system tapped in the present one-shot An eye tracking investigation of binding, page 32 task contributed to infants’ performance in these previous tasks. Nonetheless, this collection of studies taken together suggests that identity-location binding develops across infancy. Indeed, other reported results suggest that binding, particularly in VSTM, continues to develop into childhood. Cowan and colleagues (2006), for example, found developmental changes in feature binding in a VSTM task during childhood, suggesting that the system tapped in our task has a protracted developmental time course. The task used here may be an important tool for future research to further uncover the developmental time course of binding in VSTM. An important feature of the one-shot task we used here is that it allows us to determine that infants not only detected that a change occurred, but also determined which item(s) changed from sample to test. In Experiment 1, infants looked at each of the items that swapped locations four times more (on average) than they looked at the item that remained unchanged. In Experiment 2, infants looked at the item or items that changed, exhibiting a significant preference when one item changed and dividing their looking equally between two changed items. Thus, these infants robustly indicated that they detected which item changed by preferring to look at changed items to unchanged items. This is similar to when adults explicitly report which item has changed (Johnson et al., 2013) or make eye movements toward the changed item (Hyun et al., 2009). This general approach of using infants’ looking behavior to evaluate which item has changed is similar to the approach used by Kaldy and Leslie (2003; 2005) to study infants’ representation of items in working memory and by Blaser and Kaldy (2010) to study what infants store in iconic memory. In each of these previous studies, as in the two experiments reported here, the design and procedure allowed assessment of infants’ responding to individual items, rather than the more traditional approach of assessing infants’ overall response to a display in which something has changed. That is, in traditional procedures infants’ looking to events in An eye tracking investigation of binding, page 33 general is measured whereas in these alternative procedures infants’ looking to single objects is measured. The use of the one-shot task in the present investigation allowed us to determine that, like adults, infants recognize which items have changed. This work has implications for our understanding of binding and VSTM more broadly. There have been longstanding debates in cognitive psychology and neuroscience about how nonspatial information and location are represented. This work has not yielded a consistent picture. One issue is whether attention to spatial features, such as a location in space, influences attention to and representation of non-spatial features, such as color or identity. Some have argued that attention to spatial locations boosts attention to non-spatial features presented at those attended locations (Hillyard & Munte, 1984; Leonard, Balestreri, & Luck, 2015), whereas others have argued that attention to non-spatial features is independent of spatial attention (Saenz, Buracas, & Boynton, 2002; Treue & Martinez Trujillo, 1999). Another issue is whether spatial and non-spatial features have equivalent roles and functions in attention. Some researchers have argued that spatial and non-spatial features may play equivalent roles in attention (Bundesen, 1990) and non-spatial attention may operate in the same manner as spatial attention (Valdes-Sosa, Bobes, Rodriguez, & Pinilla, 1998; Zhang & Luck, 2009). However, others argue that location plays a unique role in attention (Nissen, 1985), with non-spatial features being used solely to guide attention to relevant locations (Moore & Egeth, 1998). In her influential feature integration theory, Treisman proposed that non-spatial features can be detected even if their locations are unknown, suggesting independence between spatial and identity features (Treisman, 1980). Indeed, electrophysiological evidence indicates that non-spatial features can be detected without the spatial focusing of attention (Luck & Ford, 1998). However, behavioral evidence has shown that subjects cannot detect a feature unless they An eye tracking investigation of binding, page 34 can also localize it (Johnston & Pashler, 1990). Although the work reported here cannot resolve such debates, a more complete understanding of how binding, spatial attention, and VSTM develop in infancy can have an important role in aiding our understanding of these issues. In summary, the results reported here add to our growing understanding of the development of VSTM in infancy. The findings are consistent with those obtained in other studies using other procedures and stimuli, and they extend the understanding obtained from previous studies. The literature taken together suggests that infants in the second half of the first postnatal year can represent both the identities and the identity-location bindings of items in multiple-item arrays. An eye tracking investigation of binding, page 35 References Astle, D. E., & Scerif, G. (2011). Interactions between attention and visual short-term memory (VSTM): What can be learnt from individual and developmental differences? Neuropsychologia, 49, 1435–1445. http://doi.org/10.1016/j.neuropsychologia.2010.12.001 Baddeley, A. D., & Hitch, G. J. (1974). Working Memory. In G. H. Bower (Ed.), The Psychology of Learning and Motivation, Vol. VIII (pp. 47–90). New York: Academic Press. Bays, P. M., Catalao, R. F., & Husain, M. (2009). The precision of visual working memory is set by allocation of a shared resource. Journal of Vision, 9, 1–11. http://doi.org/10.1167/9.10.7 Blaser, E., & Kaldy, Z. (2010). Infants Get Five Stars on Iconic Memory Tests: A Partial-Report Test of 6-Month-Old Infants’ Iconic Memory Capacity. Psychological Science, 21, 1643– 1645. http://doi.org/10.1177/0956797610385358 Brennan, W. M., Ames, E. W., & Moore, R. W. (1966). Age differences in infants’ attention to patterns of different complexities. Science, 49, 354–356. Brockmole, J. R., & Logie, R. H. (2013). Age-related change in visual working memory: A study of 55,753 participants aged 8-75. Frontiers in Psychology, 4. http://doi.org/10.3389/fpsyg.2013.00012 Bundesen, C. (1990). A theory of visual attention. Psychological Review, 97, 523–547. http://doi.org/10.1037/0033-295X.97.4.523 Catherwood, D., Skoien, P., Green, V., & Holt, C. (1996). Assessing the primary moments of infant encoding of compound visual stimuli. Infant Behavior and Development, 19, 1–11. Cavanagh, P., Arguin, M., & von Grünau, M. (1989). Interattribute apparent motion. Vision Research, 29, 1197–1204. http://doi.org/10.1016/0042-6989(89)90065-5 Cowan, N., Naveh-Benjamin, M., Kilb, A., & Saults, J. S. (2006). Life-span development of An eye tracking investigation of binding, page 36 visual working memory: when is feature binding difficult? Developmental Psychology, 42, 1089. http://doi.org/10.1037/0012-1649.42.6.1089 Feigenson, L., & Halberda, J. (2004). Infants chunk object arrays into sets of individuals. Cognition, 91, 173–190. http://doi.org/10.1016/j.cognition.2003.09.003 Fukuda, K., Vogel, E. K., Mayr, U., & Awh, E. (2010). Quantity, not quality: The relationship between fluid intelligence and working memory capacity. Psychonomic Bulletin & Review, 17, 673–679. http://doi.org/10.3758/17.5.673 Hillyard, S. A., & Munte, T. F. (1984). Selective attention to color and location: an analysis with event-related brain potentials. Perception & Psychophysics, 36, 185–198. Hollingworth, A., & Henderson, J. M. (2002). Accurate visual memory for previously attended objects in natural scenes. Journal of Experimental Psychology: Human Perception & Performance, 28, 113–136. http://doi.org/10.1037/0096-1523.28.1.113 Hollingworth, A., & Luck, S. J. (2009). The role of visual working memory (VWM) in the control of gaze during visual search. Attention, Perception & Psychophysics, 71, 936–949. http://doi.org/10.3758/APP Hyun, J. S., Woodman, G. F., Vogel, E. K., Hollingworth, A., & Luck, S. J. (2009). The Comparison of Visual Working Memory Representations With Perceptual Inputs. Journal of Experimental Psychology-Human Perception and Performance, 35, 1140–1160. http://doi.org/10.1037/a0015019 Irwin, D. E. (1991). Information integration across saccadic eye movements. Cognitive Psychology, 23, 420–456. http://doi.org/10.1016/0010-0285(91)90015-G Jeffreys, H. (1961). Theory of Probability. Oxford, UK: Oxford University Press. Johnson, M. K., McMahon, R. P., Robinson, B. M., Harvey, A. N., Hahn, B., Leonard, C. J., … An eye tracking investigation of binding, page 37 Gold, J. M. (2013). The relationship between working memory capacity and broad measures of cognitive ability in healthy adults and people with schizophrenia. Neuropsychology, 27, 220–9. http://doi.org/10.1037/a0032060 Johnston, J. C., & Pashler, H. (1990). Close binding of identity and location in visual feature perception. Journal of Experimental Psychology. Human Perception and Performance, 16, 843–56. http://doi.org/10.1037/0096-1523.16.4.843 Kaldy, Z., & Lee, P. (2003). Identification of objects in 9-month-old infants: Integrating “what” and “where” information. Developmental Science, 6, 360–373. http://doi.org/10.1111/14677687.00290 Kaldy, Z., & Leslie, A. M. (2005). A memory span of one? object identification in 6.5- month-old infants. Cognition, 97, 153–177. http://doi.org/10.1016/j.cognition.2004.09.009 Kibbe, M. M. (2015). Varieties of Visual Working Memory Representation in Infancy and Beyond. Current Directions in Psychological Science, 24, 433–439. http://doi.org/10.1177/0963721415605831 Kramer, P., & Yantis, S. (1997). Perceptual grouping in space and time: evidence from the Ternus display. Perception & Psychophysics, 59, 87–99. http://doi.org/10.3758/BF03206851 Leonard, C. J., Balestreri, A., & Luck, S. J. (2015). Interactions between space-based and feature-based attention. Journal of Experimental Psychology: Human Perceptoin and Performance, 41, 11–16. http://doi.org/10.1037/xhp0000011 Luck, S. J., & Ford, M. A. (1998). On the role of selective attention in visual perception. Proceedings of the National Academy of Sciences, 95, 825–830. Luck, S. J., & Hollingworth, A. (2008). Visual short-term memory. In S. J. Luck & A. Hollingworth (Eds.), Visual Memory (pp. 43–85). New York: Oxford University Press. An eye tracking investigation of binding, page 38 Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390, 279–281. http://doi.org/10.1038/36846 Mareschal, D., & Johnson, M. H. (2003). The “what” and “where” of object representations in infancy. Cognition, 88, 259–276. http://doi.org/10.1016/S0010-0277(03)00039-8 Moore, C. M., & Egeth, H. (1998). How Does Feature-Based Attention Affect Visual Processing ? Journal of Experimental Psychology: Human Perception and Performance, 24, 1296–1310. Newcombe, N., Huttenlocher, J., & Learmonth, A. (1999). Infants’ coding of location in continuous space. Infant Behavior and Development, 22, 483–510. http://doi.org/10.1016/S0163-6383(00)00011-4 Nissen, M. J. (1985). Accessing features and objects: Is location special? In M. I. Posner & O. S. M. Martin (Eds.), Attention and Performance (vol XI) (Vol. XI). Hillsdale, NJ: Erlbaum. Oakes, L. M. (2009). The “Humpty Dumpty Problem” in the Study of Early Cognitive Development: Putting the Infant Back Together Again. Perspectives on Psychological Science, 4, 352–358. Retrieved from http://dx.doi.org/10.1111/j.1745-6924.2009.01137.x Oakes, L. M., Baumgartner, H. A., Barrett, F. S., Messenger, I. M., & Luck, S. J. (2013). Developmental changes in visual short-term memory in infancy: Evidence from eyetracking. Frontiers in Psychology, 4. http://doi.org/10.3389/fpsyg.2013.00697 Oakes, L. M., Ross-Sheehy, S., & Luck, S. J. (2006). Rapid development of feature binding in visual short-term memory. Psychological Science, 17, 781–787. http://doi.org/10.1111/j.1467-9280.2006.01782.x Pantle, A., & Picciano, L. (1976). A multistable movement display: evidence for two separate motion systems in human vision. Science, 193, 500–502. An eye tracking investigation of binding, page 39 http://doi.org/10.1126/science.941023 PapaThomas, T. V., Gorea, A., & Julesz, B. (1991). Two carriers for motion perception: Color and luminance. Vision Research, 31, 1883–1892. http://doi.org/10.1016/00426989(91)90183-6 Prazdny, K. (1986). What variables control (long-range) apparent motion? Perception, 15, 37– 40. Ranganath, C. (2010). Binding Items and Contexts: The Cognitive Neuroscience of Episodic Memory. Current Directions in Psychological Science, 19, 131–137. http://doi.org/10.1177/0963721410368805 Richmond, J. L., Zhao, J. L., & Burns, M. A. (2015). What goes where? Eye tracking reveals spatial relational memory during infancy. Journal of Experimental Child Psychology, 130, 79–91. http://doi.org/10.1016/j.jecp.2014.09.013 Rose, S. A. (1981). Developmental changes in infants’ retention of visual stimuli. Child Development, 52, 227–233. http://doi.org/10.2307/1129235 Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16, 225–237. http://doi.org/10.3758/PBR.16.2.225 Saenz, M., Buracas, G. T., & Boynton, G. M. (2002). Global effects of feature-based attention in human visual cortex. Nature Neuroscience, 5, 631–632. http://doi.org/10.1038/nn876 Simmering, V. R. (2012). The development of visual working memory capacity during early childhood. Journal of Experimental Child Psychology, 111, 695–707. http://doi.org/10.1016/j.jecp.2011.10.007 Slater, A., Mattock, A., Brown, E., Burnham, D., & Young, A. (1991). Visual processing of An eye tracking investigation of binding, page 40 stimulus compounds in newborn infants. Perception, 20, 29–33. Treisman, A. M. (1980). A Feature-Integration Theory of Attention. Cognitive Psychology, 12, 97–136. http://doi.org/10.1016/0010-0285(80)90005-5 Treue, S., & Martinez Trujillo, J. C. (1999). Feature-based attention influences motion processing gain in macaque visual cortex. Nature, 399, 575–579. http://doi.org/10.1038/21176 Valdes-Sosa, M., Bobes, M. A., Rodriguez, V., & Pinilla, T. (1998). Switching attention without shifting the spotlight object-based attentional modulation of brain potentials. Journal of Cognitive Neuroscience, 10, 137–151. http://doi.org/10.1162/089892998563743 van Hoogmoed, A. H., van den Brink, D., & Janzen, G. (2013). Infants’ object location and identity processing in spatial scenes: An ERP study. Brain and Behavior, 3, 729–737. http://doi.org/10.1002/brb3.184 Vogel, E. K., Woodman, G. F., & Luck, S. J. (2001). Storage of features, conjunctions, and objects in visual working memory. Journal of Experimental Psychology Human Perception and Performance, 27, 92–114. http://doi.org/10.1037//0096-1523.27.1.92 Vogel, E. K., Woodman, G. F., & Luck, S. J. (2006). The time course of consolidation in visual working memory. Journal of Experimental Psychology: Human Perception and Performance, 32, 1436–1451. http://doi.org/10.1037/0096-1523.32.6.1436 Wang, S., & Baillargeon, R. (2005). Inducing infants to detect a physical violation in a single trial. Psychological Science, 16, 542–549. http://doi.org/10.1111/j.0956-7976.2005.01572.x Wheeler, M. E., & Treisman, A. M. (2002). Binding in short-term visual memory. Journal of Experimental Psychology: General, 131, 48–64. http://doi.org/10.1037/0096-3445.131.1.48 Wilcox, T., & Schweinle, A. (2002). Object individuation and event mapping: Developmental An eye tracking investigation of binding, page 41 changes in infants’ use of featural information. Developmental Science, 5, 132–150. Zhang, W. W., & Luck, S. J. (2008). Discrete fixed-resolution representations in visual working memory. Nature, 453, 233-U13. http://doi.org/Doi 10.1038/Nature06860 Zhang, W. W., & Luck, S. J. (2009). Feature-based attention modulates feedforward visual processing. Nature Neuroscience, 12, 24–25. http://doi.org/10.1038/nn.2223 An eye tracking investigation of binding, page 42 Figure 1. (A) Schematic depiction of stimulus streams in the object identity-location binding simultaneous streams change detection task of Oakes et al. (2006). Reprinted from Oakes, L. M., Ross-Sheehy, S., & Luck, S. J. (2006). Rapid development of binding in visual short-term memory. Psychological Science, 17, 781-787. Copyright Sage. (B) Schematic depiction of the one-shot change detection task used by Oakes et al. (2013). Reprinted from Oakes, L. M., Baumgartner, H. A., Barrett, F. S., Messenger, I. M., & Luck, S. J. (2013). Developmental changes in visual short-term memory in infancy: Evidence from eye tracking. Frontiers in Psychology, 4, 697. Copyright © 2013 Oakes, Baumgartner, Barrett, Messenger & Luck. An eye tracking investigation of binding, page 43 Figure 2. Schematic of stimuli for a single trial in Experiment 1. Note that from sample to test, the colors of the items in the lower left and lower right “swapped” locations, violating the colorlocation bindings. The third item was unchanged from sample to test. An eye tracking investigation of binding, page 44 Figure 3. (A) Subject-weighted proportion of trials on which infants were looking toward either of the two changed items at each time point (vertical bars represent 95% confidence intervals). A value of 1.0 for any given time point would indicate an averaged score of 100% looking to the changed items, and a value of 0 would indicate an averaged score of 100% looking to the unchanged item; the horizontal line at .67 represents chance responding (i.e., 2/3 of looks directed toward one of the two changed items and 1/3 of looks directed toward the unchanged item). The vertical line at time 0 indicates the onset of the test array; the portion of the graph to the left of the line represents the pre-change (sample and retention) period (note the proportion of looking toward the changed items is at chance levels), and the portion of the graph to the right of this line represents the post-change test period. The area of the curve shaded in gray corresponds An eye tracking investigation of binding, page 45 to a series of consecutive time points with a length predicted to occur by chance less than 5% of the time. (B) Distribution of maximal run lengths obtained from the Monte Carlo analysis of the permutations, which is the distribution expected if the null hypothesis of no preference is true (see text). The 95% point on the distribution is indicated by a vertical line; any runs longer than this (14 samples) occurred on fewer than 5% of the iterations, so any runs longer than this in the observed data are considered significantly longer than expected by chance. An eye tracking investigation of binding, page 46 Figure 4. Schematic depiction of Feature-Only trials (A) and Feature+Binding trials (B) used in Experiment 2. In each type of trial, one randomly selected item in the test array was a new color; in Feature-Only trials the other item did not change, whereas in Feature+Binding trials the second item was a previously-seen color in a new location, representing a change in colorlocation binding. An eye tracking investigation of binding, page 47 Figure 5. Infants’ average preference for the new color in Experiment 2 by trial type (horizontal line indicates chance responding). Each individual infant is represented by a diamond (8-monthold infants are grey diamonds and 10-month-old infants are open diamonds); each diamond is shifted slightly along the horizontal access to make it easier to see the individual values. The mean preference score (collapsed across age) is the black diamond, and the error bars show the 95% confidence interval for each trial type. An eye tracking investigation of binding, page 48 Figure 6. Subject-weighted proportion of trials on which infants were looking towards the new color at each time point (vertical bars represent 95% confidence intervals) in the Feature-Only trials (A) and the Feature+Binding trials (B). A value of 1.0 for any given time point would indicate that all infants looked towards the new color on every trial at that time point. Dots below the curve indicate time points that are independently different than chance. The area of the curve shaded in gray corresponds to a series of consecutive time points that are independently different than chance whose length is predicted from the Monte Carlo simulation to occur less than 5% of the time.