| Home | Research | Publications | Classes | Consulting | My Lab |
| Research : | Issues |
These three areas – theory of interaction, workplace design and information architecture – form a coherent research program. To design highly interactive systems, particularly systems which blend virtual and real elements in the way we expect context aware venues to, we need to understand how to create environments that help make the process of managing our tasks cognitively easier. This research is in the tradition of situated and distributed cognition, but extends or complements both in using a combination of ethnographic, experimental, and computational methods to uncover the dynamics that emerge when people solve problems in specific situations. Among the significant issues raised by this research are the following seven.
- Methodology
- What is the nature of cognitive coupling?
- How do interactive strategies evolve?
- How does a theory of interactivity fit with the principles of evolutionary biology?
- Context Aware Offices
- Information Architecture
- Elearning Environments
1. Methodology
How should we experimentally study natural cognition? Methodology has
become problematic because in natural cognition we freely change the environmental
conditions which become the stimulus conditions we face the next moment.
The classical methods of psychological experimentation rely on the experimentalist
fixing the stimuli, working in what Aaron Cicourel calls ‘white rooms’.
To remedy this, scientists who study situated and distributed cognition
have tended to study natural cognition as it manifests ‘in the wild’ by
using ethnographic techniques. But there is no reason a suitably modified
experimental methodology will not deliver valid results. Accordingly,
one key issue is to design interactive experiments where subjects can
change the conditions that constitute their next stimulus. Some of these
are highly controlled, others are semi-structured, allowing the subject
or group of collaborators to work freely within parameters.
2. What
is the nature of cognitive coupling?
As we focus on briefer phases of interaction a key element to understand
is how we are embedded or coupled with the environment when we solve the
momentary problems that everyday life throws at us. Agents interact with
their environments at different time scales, from milliseconds to days,
making the dynamics of agent-environment interaction especially complex.
Little is known of the way we are embedded or coupled with our environments.
One obstacle is that most experimentation and most computational modeling
of problem solving, remains tied, in spirit, to rational decision theory.
Rational decision theory assumes that once agents are given a goal they
act by selecting an action from a feasible set in light of expected consequences.
Once the action is taken they re-evaluate their situation, the feasible
actions and any impact their action may have had on their understanding
of the world. This has the effect of treating the agent as substantially
out of contact with the world for the bulk of the deliberation or action
formulation process. If we are more tightly coupled to the world than
this, it is likely that even during deliberation we are not `out of touch'
with the world. In fact, we often perform non-deliberative acts to enhance
our deliberation. I have coined the terms, epistemic actions, and complementary
actions, to name the various ways we have of interacting with the environment
to facilitate perception, categorization, reasoning, and decision making.
We have barely begun to scratch the range and diversity of these type
of actions (although I made an effort at classifying these in Adapting
the World Instead of Ourselves and in Interactivity
and Multimedia Interfaces).
3. How
do interactive strategies evolve?
If we accept that people are always doing things to the world in the course
of solving problems – they solve problems interactively – the next question
to ask is how they developed these interactive strategies. A simple example
of an interactive strategy can be seen when a person counts more than
10 coins. Most people either use their fingers to point to coins to help
them keep track of where they are in the process of visually counting,
or they physically move the coins into groups to make the visual problem
of counting simpler. To understand how these processes emerge and change,
we need first to understand why certain interactive strategies are computationally
advantageous -- how can people save time, memory, attentional effort or
program state -- by altering the physical circumstances of their task.
To study this we created experimental contexts where people repeatedly
solve a problem so that we could observe the interactive strategies they
developed.Experimental contexts were `blocks world' tasks, Tetris, coin
counting, card playing, sorting sticks, jigsaw puzzling, and playing scrabble.
One of the forces driving people to develop interactive strategies is
the need to cope with interruptions and cognitive overload. In ecologically
natural environments people develop interactive strategies to cope. No
theory of the organization of human behavior is complete without an account
of how humans develop these robust responses -- interactive strategies
– to handle the real exigencies of life. It is a core problem of cognition.
4. How
does a theory of interactivity fit with the principles of evolutionary
biology?
It is a tenet of current science that any theory of the organization
of human behavior must be consistent with the modern theory of adaptation.
Yet in the mathematical theory of evolution the structure of the
environment is assumed to be relatively static, and only minimally a function
of an individual creature's activity. How can we build a theory about
the importance of changing our habitats if evolutionary theory assumes
that, as individuals, we do not significantly change our environments?
To take an example, although a population of creatures may change
the reward landscape of their habitat by overeating or by polluting, thereby
reducing the payoff of certain eating behaviors in that niche (because
now there are no juicy or healthy bits nearby), individual creatures,
by definition, are assumed to have an epsilon effect on their habitat.
Each creature is assumed to adapt to its environment; it doesn’t reshape
its fundamentals. A parallel principle at a cognitive level is that creatures
who acquire knowledge of structures or patterns in their environment do
not change those structures in the process of acquiring their knowledge.
Such assumptions permit the use of powerful mathematical formalisms and
are useful as a first approximation to our biological and cognitive situation.
But in accepting them uncritically we are in danger of overlooking the
range of adaptations that are possible, and the many time scales at which
these adaptations may be understood. Creatures may create certain dynamics
which themselves become the object of adaptation. In the paper, Adapting
the World Instead Of Oneself, I explored the implication of this way
of thinking for evolutionary biology and psychology. My focus there was
on how we change the world to make it a place that fits us better; a study
of the dynamic cycle that occurs when we change the world, adapt to that
change, and so on.
5. Context
Aware Offices
A theory of interactivity is an important component in our changing
understanding of how behavior is organized.It becomes of practical use
in the field of HCI and cognitive engineering. As digital workspaces move
out of the confines of computer screens and into the environment – on
wall screens, in handheld devices, in videoconference units, in furniture
– the question arises: how should hybrid environments be designed?
A hybrid environment is one where the digital is mingled with the physical;
where the virtual is non-intrusively integrated with the material to make
it easier for us to work and play, especially in small teams. By adding
virtual elements to our environments we expand our collaborative possibilities
because users can do things they cannot do in real life. In the most immersive
environments they can hold meetings in “outer space,” on the “sea floor”
simulated to show the consequences of pollution, inside the “combustion
chamber” of an automobile engine, or in their Asian factory in front of
the production line. At a more prosaic level these environments can be
our ordinary offices. We can project ‘personal metadata’ onto documents
so that we can have processors help us find relevant papers such as ‘all
the docs Roger gave me this week’ or the papers I was working on when
Bryan came in. Hybrid environments should adapt to our specific needs,
evolving computational attributes which fit the way we work and think.
Since the field is in its infancy we have little experience in creating
adaptive rooms -- rooms which dynamically adapt to the workflow
needs of participants. Hence there is as yet no clear ontology of objects
necessary for designing such rooms and offices. There is a golden opportunity
for cognitive scientists interested in interactivity and distributed cognition
to shape the basic research agenda. New methodologies for studying workflow
in normal activities are required, as are new conceptions of collaboration,
coordination and interactivity. I discussed such issues in a 1997 article
on Adaptive
Rooms, I have been researching the nature of distributed collaborative
workflow in a major ONR funded project on Distributed Hybrid Environments
for Collaboration, in another project last year we did an ethnographic
study of several cogsci staff in their offices. I have begun writing on
these issues in the articles entitled Changing
the Rules: Architecture in the New Millennium, The
Context of Work, and Metacognition,
Distributed Cognition and Visual Design.
6. Information
Architecture
In designing large scale information spaces, such as are currently
found in corporate intranets or large scale knowledge management systems,
the biggest challenge is to discover a set of organizing principles that
is at once powerful enough to assign all content a meaningful location
and yet simple enough for users to understand. People have to know how
to navigate through a system of labels and visual cues to find the information
they want. Little science has been done in this area. One problem is that
the information a person wants depends substantially on their goals. In
general, there will be no single organizational scheme that classifies
everything appropriately, since the most meaningful classification will
vary with task. This way of looking at the problem of information architecture
puts it squarely in the camp of cognitive science. For the best way to
structure information environments now depends on understanding the cognitive
workflow of users as they undertake their different jobs. As we graduate
from computer screen environments to 3D environments where information
may be made available in new ways the problem only intensifies. We need
principles, guidelines, and experimental and other methodologies to explore
the problem of information design. I made a small start on this problem
with my long standing post doctoral student Thomas Rebotier by testing
to see if people differ in their judgments of classification. We built
an interactive exercise (which I use in my 187A class) in which students
are asked to classify 50 words into 5 or 6 categories. They do this online
in a web based experiment. We then run several clustering algorithms over
the proposed categories to see if the students cluster into a few groups
who share similar categorization schemes. The results so far support the
view that students vary in meaningful ways. One application of this study
might be a method for predicting which category scheme a given user prefers.
Another may be to study cultural biases in classification. If this can
be inferred quickly it may be possible to adapt or customize the information
architecture of a site to meet the biases of each user. This latter idea
was explored in a research contract from France Telecom.
7. Elearning environments
How should we put to use our improved understanding of interactivity and
coordination to design learning environments where students can be creative
but still master the core knowledge requirements necessary for informed
reasoning? For the last 5 years while teaching my UCSD classes from Washington
DC I worked hard at designing a learning environment that would allow
me to teach more effectively in a remote but synchronous manner. Most
research on elearning has focused on distributed asynchronous learning.
This pushes the teacher into the role of a talking head who exists to
help students on issues that are anticipated.My objective has been to
improve the quality of teaching in synchronous contexts providing we can
assume students are sitting in front of computers. The simplest method
for doing this, once we get past the need to have a learning shell which
can house content in a dynamic database driven manner, is to provide students
with interactive exercises for them to do during class time. This increases
their motivation to learn from the teacher, since now they have real concerns.
It also allows for a more tutorial like style of teaching, which is by
definition is highly interactive and social. My classes can be found online:
cogs187A,
cogs187B,
cogs25. Be sure to look
at the collaborative environments that student
teams use in 187B. To login to any of these you may use the login
name of visitor and password test. See the brief discussion
later for a description of the functionality of the major learning environments
I have designed and implemented.
[ top ]
