Human Attention in Digital Environments. Cambridge University Press

Edited by Claudia Roda American University of Paris

Cambridge University Press
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore,
S˜ao Paulo, Delhi, Dubai, Tokyo, Mexico City

The Edinburgh Building, Cambridge CB2 8RU, UK

e-books shop
Human Attention in Digital Environments

This book was originally conceived with the objective of disseminating
the results of the AtGentive project, an international research project
sponsored by the European Commission. As the book evolved, the desire
to provide a wider view of the research and applicative work in the area of
attention-aware systems resulted in the inclusion of many chapters coming
from other applied research projects. I am grateful to the authors who
have contributed not only with their excellent chapters but, very often,
also by providing comments and suggestions to authors of other chapters.
This has resulted in creating those bridges that are often missing between
different disciplines and between specific aspects of inquiry within this
area of research. It has been a pleasure and a rewarding learning experience
to be able to coordinate this work.

I also would like to thank the many reviewers who have commented on
individual chapters. The quality of this book has certainly gained from
their insights. I extend my appreciation to the publisher’s anonymous
reviewers who have provided comments and suggestions about the overall
structure and content of the book.

My gratitude goes to Jan Steyn and Antal Neville whose patient
and thorough work in proofreading and organizing references has been invaluable.
Special thanks to Hetty Reid, Commissioning Editor, and Tom
O’Reilly, Production Editor for Cambridge University Press, who have
supported and guided me during the whole process of creating this book,
together with Joanna Garbutt, Carrie Cheek and Oliver Lown. Finally, I
am sincerely grateful for the professional, careful and timely copy-editing
work of Diane Ilott, who has spent many hot summer days patiently fixing
the small details that make all the difference.

Claudia Roda
In recent years it has been increasingly recognized that the advent of
information and communication technologies has dramatically shifted
the balance between the availability of information and the ability of
humans to process information. During the last century information was
a scarce resource. Now, human attention has become the scarce resource
whereas information (of all types and qualities) abounds. The appropriate
allocation of attention is a key factor determining the success of creative
activities, learning, collaboration and many other human pursuits. A suitable
choice of focus is essential for efficient time organization, sustained
deliberation and, ultimately, goal achievement and personal satisfaction.
Therefore, we must address the problem of how digital systems can
be designed so that, in addition to allowing fast access to information
and people, they also support human attentional processes. With the
aim of responding to this need, this book proposes an interdisciplinary
analysis of the issues related to the design of systems capable of supporting
the limited cognitive abilities of humans by assisting the processes
guiding attention allocation. Systems of this type have been referred to
in the literature as Attention-Aware Systems (Roda and Thomas 2006),
Attentive User Interfaces (Vertegaal 2003) or Notification User Interfaces
(McCrickard, Czerwinski and Bartram 2003) and they engender many
challenging questions (see, for example, Wood, Cox and Cheng 2006).
The design of such systems must obviously rest on a deep understanding
of the mechanisms guiding human attention. Psychologists have studied
attention from many different perspectives. In the nineteenth century,
when attention was mainly studied through introspection, William James
(considered by many the founder of American psychology) devoted a
chapter in his Principles of Psychology to human attention and observed:
Everyone knows what attention is. It is the taking possession by the mind, in clear
and vivid form, of one out of what seem several simultaneously possible objects
or trains of thought . . . It implies withdrawal from some things in order to deal
effectively with others (James 1890: 403–4).

However, as for many other things that ‘everyone knows’, such as rationality,
intelligence, memory and love, attention escapes a precise definition,
and more than a century after James’ writing, its mechanisms still
generate debates and controversy in the scientific community.

Since the mid-twentieth century, attention allocation has been viewed
as the process of selecting stimuli for processing, and research has focused
on the question of when and how this selection takes place. Proponents of
early selection theory (Broadbent 1958) argue that stimuli are filtered early,
at the perceptual level, on the basis of their physical properties so that
irrelevant (unattended) stimuli are not further processed. Proponents of
the modified early selection theory (Treisman 1960) maintain that the early
filter is not just on or off but that some stimuli are just attenuated rather
than completely filtered out, so that some irrelevant stimuli may reach
consciousness. Proponents of late selection theory (Deutsch and Deutsch
1963) argue that all stimuli are analysed (i.e., there is no filter at perceptual
level) but only pertinent stimuli are selected for awareness and
memorization. More recently some of the fundamental assumptions of
the early/late selection dichotomy have been questioned (Awh, Vogel and
Oh 2006; Vogel, Luck and Shapiro 1988) and the debate over early and
late selection has directly or indirectly raised many other related questions:
e.g., does attention modify the manner in which we perceive the
environment, or does it impact on our response to what we perceive?
This is an important question for the design of attention-aware systems.

For example, Posner (1980) suggests that cueing facilitates perception
and that different cues activate brain areas devoted to alerting and to orienting
attention (Posner and Fan 2007). This implies that it is possible
to help the user redirect attention, maintain attention on a certain item,
or simply alert him to possibly relevant stimuli. However, psychological
literature also tells us that certain stimuli may be perceived if uncued and
even if they are actively blocked. For example, in a noisy environment
such as a cocktail party we are able to block out noise and listen to just
one conversation amongst many (Cherry 1953), but why will some of
us very easily and almost necessarily notice our name if mentioned in a
nearby but unattended conversation? In trying to address this question,
Conway and his colleagues showed that ‘subjects who detect their name
in the irrelevant message have relatively low working-memory capacities,
suggesting that they have difficulty blocking out, or inhibiting, distracting
information’ (Conway, Cowan and Bunting 2001: 331). Similar results,
relating working-memory capacity and the ability to block distractors,
have been reported in the visual modality with experiments employing
neurophysiological measures (Fukuda and Vogel 2009). A better understanding
of these mechanisms could help us design systems that help
users who have more difficulties in maintaining focus with obvious applications
in, for example, in-car support systems, technology enhanced
learning applications, control room systems, etc. The study of this very
close relationship between attention and working memory has been a
very active area of research (Awh, Vogel and Oh 2006; Baddeley 2003;
Buehner et al. 2006; Engle 2002; Shelton, Elliott and Cowan 2008).

However, both attention and working memory realize multiple functions
implemented by a variety of processes that physically correspond to multiple
areas in the brain and therefore the interaction between attention
and working memory is difficult to grasp. Some of the chapters in this
book take different stands on this interaction. In chapter 4, Low, Jin
and Sweller base their analysis of the relationship between attention and
learning on an assumption of ‘equivalence between working memory
and attentional processes’; in chapter 5, Bowman and his colleagues see
attention as a mechanism that mediates the encoding and consolidation
of information in working memory; in chapter 9, Stojanov and Kulakov
indicate that activated items in working memory guide the perception processes.

Another area of research in cognitive psychology that has had a significant
impact on the field of human–computer interaction addresses the
question of whether all types of stimuli are treated by a central system or,
instead, several different systems manage different types of input. The
organization of attention over several channels associated with different
modalities was first proposed by Allport, Antonis and Reynolds (1972),
who suggested that a number of independent, parallel channels process
task demands. Users’ responses to messages in different modalities have
consequently been studied in relation to the optimization of interaction in
various applications (see, for example, chapters 4 and 7 of this volume).

The interaction between, and the integration of, these different channels
has not yet been extensively studied. The large majority of the studies of
attention have concentrated on either the sound modality or the visual
modality. Recent research, expecially when related to human–computer
interaction, is for the most part focused on visual attention. This greater
focus on visual attention is reflected in this book, with many chapters (3,
5, 7, 10) reporting results in this modality.

A final important issue, recurrent in this volume, addresses how to
facilitate the user in his perception and understanding of messages coming
from digital devices. It is commonly accepted that two types of
processes, bottom-up and top-down, guide attention and visual attention
in particular. Bottom-up processes, also called exogenous processes,
guide attention to salient elements of the environment; and topdown,
or endogenous, processes guide attention to elements of the
environment that are relevant to the current task. The definition of
what determines the saliency of elements of the environment, and the
creation of models that integrate both bottom-up and top-down processes,
has been a very active area of research (Cave 1999; Itti 2005;
Peters and Itti 2007). These issues are central to chapters 3 and 5 of this book.

A challenge that this book aims to address is the creation of a bridge
(or a set of bridges) between the research work carried out in cognitive
psychology and neuroscience, which reports fundamental results
on specific aspects of attentional processes, and the work carried out
in human–computer interaction that endeavours to apply these results.
The difficulty of this effort is mainly due to the fact that, in the former
work, experiments are carried out in controlled environments where the
conditions under which subjects are working are known, and effects are
observed over periods of time that are often very short (down to the millisecond).
Instead, in real-world situations, such as the ones addressed
by research in human–computer interaction, there is very little or no
control over the conditions under which users are working, and the
time lengths are much longer with effects that may span hours, days
or even months. To make things worse, addressing the problems faced
by human–computer interaction would require a holistic theory of attention,
which is still far from being achieved. As a result, the tools and
systems proposed in the chapters of this book necessarily focus only
on some aspects of attention. For example, chapter 8 focuses on the
effects of contextual information, chapter 10 on the conspicuity of visual
information, and chapter 12 on social aspects of attention. Nevertheless,
attention-aware applications have been shown to be greatly beneficial
in several areas, including the control of appliances and desktop
interfaces (chapter 7), robotics (chapter 9), visualization for decision
making (chapter 10), learning and training (chapters 8 and 11), and
online collaborative environments (chapter 12).

The book is organized in three parts, with chapters that focus mainly on
concepts in part I, chapters that focus mainly on theoretical and software
tools in part II, and chapters describing applications in part III.
Part I (Concepts) introduces the conceptual framework of research
aimed at modelling and supporting human attentional processes. The
chapters in this part analyse human attention in digital environments,
integrating results from several different disciplines, including cognitive
psychology, neuroscience, pedagogy and human–computer interaction.
Chapter 2 sets the scene by providing a broad overview of the main
issues addressed by attention research in cognitive psychology and neuroscience,
and their relevance for the design of digital devices.

In chapter 3, Ronald Rensink reviews one of the prevalent areas
of attention research, vision science. Drawing on his vast experience
in this subject, Rensink guides the reader through an exploration of
visual attention and the many processes involved in scene perception.
Based on this knowledge of scene perception, Rensink proposes that
displays may be designed so that they elicit particularly efficient users’ responses.

John Sweller, who co-authors chapter 4 with Renae Low and Putai
Jin, has developed cognitive load theory, one of the most influential theories
relating attention and learning. Cognitive load theory was originally
designed ‘to provide guidelines intended to assist in the presentation of
information in a manner that encourages learner activities that optimize
intellectual performance’ (Sweller, Merrienboer and Paas 1998: 251). In
chapter 4 the authors discuss the impact of cognitive load theory on the
design of digital tools supporting learning.

Part I closes with a chapter by Howard Bowman, Li Su, Brad Wyble
and Phil J. Barnard. The authors report on the results obtained in the
Salience Project,1 and elegantly analyse some aspects of attention that
have been the focus of recent research, including its temporal organization,
its redirection, and the role of long-term goals and emotional
significance in determining saliency.

Part II (Theoretical and software tools) analyses the theoretical and
computational mechanisms currently available for supporting human
attentional processes. These tools span very different areas of attentionrelated
services to users.

Chapter 6, contributed by Benoˆıt Morel and Laurent Ach, focuses on
the design of artificial characters that adapt to the attentional state of
the user. On the strength of over a decade of practice in creating 3D
embodied agents, the authors explain the role that attention plays in creating
engaging agents ‘that are capable of natural, intuitive, autonomous
and adaptive behaviours that account for variations in emotion, gesture,
mood, voice, culture and personality’.

In chapter 7, Kari-Jouko R¨aih¨a, Aulikki Hyrskykari and P¨aivi
Majaranta discuss eye-tracking technology based on their long experience
of leading some of the most successful research endeavours in
this field, including the European Network of Excellence COGAIN and
the EYE-to-IT project. Eye-tracking technology has historically been
central to the development of attention-aware applications because of
the very close relationship between gaze direction and attention. After
reviewing the psychological foundation of visual attention, the authors

address the question of the relation between attention and the point of
gaze as well as the use of the latter for the implementation of adaptive applications.

Chapter 8, authored by Hans-Christian Schmitz, Martin Wolpers,
Uwe Kirschenmann and Katja Niemann, proposes that metadata about
attention allocation can be captured and exploited to personalize information
and tasks environments. Significantly, on the basis of their extensive
application studies, the authors argue for the important role of attention
metadata for the support of cooperative work.

In chapter 9, Georgi Stojanov and Andrea Kulakov analyse how attention
may be modelled within a complete cognitive architecture. After
reviewing how attentional processes are represented in several known
cognitive architectures, the authors present their own cognitive architecture,
founded on robotics research, and they highlight the role played by
attentional processes.

Part III (Applications) presents several computing applications
designed to support attention in specific environments. The applications
presented in this part cover a wide variety of fields, showing the
relevance of attention-aware systems to fields as different as commandand-
control displays, technology-enhanced learning, and the support of
online communication and collaboration.

The application described by Frank Kooi in chapter 10 is the result
of the author’s very long experience in researching and implementating
visual displays. The objective of the two-depth layer display presented
by the author is to increase the amount of information available to the
user without increasing clutter. Based on knowledge of visual attentional
processes, Kooi proposes that, by using dual layer displays, search may
be made much more efficient in command-and-control displays.
Chapter 11, authored by Inge Molenaar, Carla van Boxtel, Peter
Sleegers and Claudia Roda, reports on a system designed to supply
adaptive and dynamic scaffolding through the analysis and support of
learners’ attentional processes. The experimental results clearly show
the potential of the application of attention management in technologyenhanced
learning environments.

Finally, in chapter 12, Thierry Nabeth and Nicolas Maisonneuve propose
an implementation of the general attention support model originally
proposed by Roda and Nabeth (2009). This model is based on
four levels of support: perception, deliberation, operation and metacognition.
Chapter 12 explains how this model may be implemented to support
social attention and describes the attention-aware social platform AtGentNet.


e-books shop

Purchase Now !
Just with Paypal

Product details
 File Size
 3,026 KB
 361 p
 File Type
 PDF format
 978-0-521-76565-7 Hardback
 Cambridge University Press 2011 

Table of Contents
Acknowledgements page ix
Notes on contributors x
List of illustrations xvii
List of tables xx
1 Introduction 1
Part I Concepts
2 Human attention and its implications for
human–computer interaction 11
3 The management of visual attention in graphic displays 63
4 Cognitive load theory, attentional processes and
optimized learning outcomes in a digital environment 93
5 Salience sensitive control, temporal attention and
stimulus-rich reactive interfaces 114
Part II Theoretical and software tools
6 Attention-aware intelligent embodied agents 147
7 Tracking of visual attention and adaptive applications 166
8 Contextualized attention metadata 186
9 Modelling attention within a complete cognitive architecture 210
Part III Applications
10 A display with two depth layers: attentional
segregation and declutter 245
11 Attention management for self-regulated learning:
AtGentSchool 259
12 Managing attention in the social web: the
AtGentNet approach 281
Index of authors cited 311
Index 321
The colour plates appear between pages 204 and 205

2.1 What makes visual search fast? colour plate (CP)
3.1 Flicker paradigm page 67
3.2 Coherence theory 68
3.3 Inattentional blindness 69
3.4 Triadic architecture 72
3.5 Featural cues 75
3.6 Proto-object structure 75
3.7 Drawing of attention by configural focus 76
3.8 Reduction of clutter via grouping 82
3.9 Organizational structures 82
3.10 Effect of different sets of values 84
4.1 A conventional, split-attention geometry example 101
4.2 A physically integrated geometry example 102
5.1 The basic AB effect for letter stimuli 118
5.2 Task schema for the key-distractor blink (Barnard et al.2004) 119
5.3 Proportion of correct responses from both humans 
and model simulations (Su et al. 2007) 119
5.4 Top-level structure of the ‘glance-look’ model with implicational subsystem attended 121
5.5 A neural network that integrates five LSA cosines to classify words as targets 124
5.6 Target report accuracy by serial position comparing
human data (Barnard et al. 2005) and model
simulations for high state and high trait anxious and low state anxious 125
5.7 The ‘glance-look’ model extended with body-state subsystem 126
5.8 Examples of raw P3s recorded from human participants (Su et al. 2009) 130
5.9 ERPs of a participant for target-seen and target-missed trials 131
5.10 Diagram of a brainwave-based receipt acknowledgement device 132
5.11 Examples of virtual P3s generated from model simulations (Su et al. 2009) 134
5.12 Performance (measured as probability of detecting
targets) of AB-unaware and AB-aware systems by
varying the window sizes of the stimuli (Su et al. 2009) 135
5.13 Top-level structure of the ‘glance-look’ model with
computer interaction (through device) and
implicational subsystem attended (Su et al. 2009) 136
5.14 Performance (measured as probability of detecting the
targets) of the reactive approach using EEG feedback
with variability in the P3 detection criterion (Su et al.2009) 138
6.1 Examples of different types of Cantoche embodied
agents (from realistic to cartoonish style) CP
6.2 The Cantoche Avatar Eva displays a series of
behaviours that highlight the advantages of full-body avatars CP
6.3 The Cantoche Avatar Dominique-Vivant Denon helps
users explore the Louvre website. Reprinted by permission CP
6.4 Living ActorTM technology: the three levels of control CP
7.1 Desk-mounted video-based eye trackers: ASL 4250R
at the top, SMI iViewX in the middle and Tobii T60 at the bottom CP
7.2 Put-That-There (Bolt 1980) C 1980 ACM, Inc. Reprinted by permission CP
7.3 Top: attentive television (Shell, Selker and Vertegaal,
2003) C 2003 ACM, Inc.; bottom: eyebox2 by Xuuk,
Inc., Both images reprinted by permission CP
7.4 Nine instances of PONG, an attentive robot (Koons
and Flickner 2003). C 2003 ACM, Inc. Reprinted by permission CP
7.5 Joint attention and eye contact with a stuffed toy robot
(Yonezawa et al. 2007). Picture reprinted courtesy of Tomoko Yonezawa CP
7.6 Two adaptive attention-aware applications. Top: ship
database (Sibert and Jacob 2000) C 2000 ACM, Inc.
Reprinted by permission; bottom: iDict, a reading aid (Hyrskykari 2006) CP
8.1 Core elements of the CAM schema 193
8.2 CAM infrastructure 197
9.1 Main data structures and parallel processes
incorporated into the Vygo architecture 224
9.2 Expansion of an abstract schema up to concrete schemas 230
9.3 Learning systems 233
9.4 A part of the cognitive architecture, responsible for
video processing, having the general learning system at its core 234
9.5 Schematic of the Attention Window (AW) 236
9.6 Two examples of saccadic movements 237
10.1 Parallax CP
10.2 Two images from dual/single layer experiment CP
10.3 Search times 252
10.4 Schematic drawing of experimental set-up and the design of the target monitor 253
10.5 The data substantiating the claim that accommodation
and motion parallax substantially aid the ease of depth perception 254
10.6 Dual-layer display (Zon and Roerdink 2007). NLR. Reprinted by permission CP
10.7 Navigation display. C NLR. Reprinted by permission CP
11.1 Example of metacognitive planning intervention CP
12.1 A snapshot of the AtGentNet platform CP
12.2 The AtGentNet overall architecture CP
12.3 Who reads me? Who do I read? CP
12.4 Stated and observed competences and interests CP

5.1 Comparison of experimental results across twelve
human participants with model simulations (Su et al.2009) page 137
9.1 A coarse-grained view of the semantics of the attention
values attached to Novamente AGI architecture atoms 220
10.1 The depth cues which are directly relevant to depth-displays 247
11.1 A summary of the intervention categories and types 269
12.1 Supporting attention at different levels 290
12.2 Mechanisms of support at different levels 297
12.3 Agent interventions 299


═════ ═════

Loading... Protection Status