Home Research Publications Classes Consulting My Lab
 
 



Research :
Issues

These three areas – theory of interaction, workplace design and information architecture – form a coherent research program. To design highly interactive systems, particularly systems which blend virtual and real elements in the way we expect context aware venues to, we need to understand how to create environments that help make the process of managing our tasks cognitively easier. This research is in the tradition of situated and distributed cognition, but extends or complements both in using a combination of ethnographic, experimental, and computational methods to uncover the dynamics that emerge when people solve problems in specific situations. Among the significant issues raised by this research are the following seven.

  1. Methodology
  2. What is the nature of cognitive coupling?
  3. How do interactive strategies evolve?
  4. How does a theory of interactivity fit with the principles of evolutionary biology?
  5. Context Aware Offices
  6. Information Architecture
  7. Elearning Environments

1. Methodology
How should we experimentally study natural cognition? Methodology has become problematic because in natural cognition we freely change the environmental conditions which become the stimulus conditions we face the next moment. The classical methods of psychological experimentation rely on the experimentalist fixing the stimuli, working in what Aaron Cicourel calls ‘white rooms’. To remedy this, scientists who study situated and distributed cognition have tended to study natural cognition as it manifests ‘in the wild’ by using ethnographic techniques. But there is no reason a suitably modified experimental methodology will not deliver valid results. Accordingly, one key issue is to design interactive experiments where subjects can change the conditions that constitute their next stimulus. Some of these are highly controlled, others are semi-structured, allowing the subject or group of collaborators to work freely within parameters.

^Topic List    

2. What is the nature of cognitive coupling?
As we focus on briefer phases of interaction a key element to understand is how we are embedded or coupled with the environment when we solve the momentary problems that everyday life throws at us. Agents interact with their environments at different time scales, from milliseconds to days, making the dynamics of agent-environment interaction especially complex. Little is known of the way we are embedded or coupled with our environments. One obstacle is that most experimentation and most computational modeling of problem solving, remains tied, in spirit, to rational decision theory. Rational decision theory assumes that once agents are given a goal they act by selecting an action from a feasible set in light of expected consequences. Once the action is taken they re-evaluate their situation, the feasible actions and any impact their action may have had on their understanding of the world. This has the effect of treating the agent as substantially out of contact with the world for the bulk of the deliberation or action formulation process. If we are more tightly coupled to the world than this, it is likely that even during deliberation we are not `out of touch' with the world. In fact, we often perform non-deliberative acts to enhance our deliberation. I have coined the terms, epistemic actions, and complementary actions, to name the various ways we have of interacting with the environment to facilitate perception, categorization, reasoning, and decision making. We have barely begun to scratch the range and diversity of these type of actions (although I made an effort at classifying these in Adapting the World Instead of Ourselves and in Interactivity and Multimedia Interfaces).

^Topic List    

3. How do interactive strategies evolve?
If we accept that people are always doing things to the world in the course of solving problems – they solve problems interactively – the next question to ask is how they developed these interactive strategies. A simple example of an interactive strategy can be seen when a person counts more than 10 coins. Most people either use their fingers to point to coins to help them keep track of where they are in the process of visually counting, or they physically move the coins into groups to make the visual problem of counting simpler. To understand how these processes emerge and change, we need first to understand why certain interactive strategies are computationally advantageous -- how can people save time, memory, attentional effort or program state -- by altering the physical circumstances of their task. To study this we created experimental contexts where people repeatedly solve a problem so that we could observe the interactive strategies they developed.Experimental contexts were `blocks world' tasks, Tetris, coin counting, card playing, sorting sticks, jigsaw puzzling, and playing scrabble. One of the forces driving people to develop interactive strategies is the need to cope with interruptions and cognitive overload. In ecologically natural environments people develop interactive strategies to cope. No theory of the organization of human behavior is complete without an account of how humans develop these robust responses -- interactive strategies – to handle the real exigencies of life. It is a core problem of cognition.

^Topic List    

4. How does a theory of interactivity fit with the principles of evolutionary biology?
It is a tenet of current science that any theory of the organization of human behavior must be consistent with the modern theory of adaptation. Yet in the mathematical theory of evolution the structure of the environment is assumed to be relatively static, and only minimally a function of an individual creature's activity. How can we build a theory about the importance of changing our habitats if evolutionary theory assumes that, as individuals, we do not significantly change our environments? To take an example, although a population of creatures may change the reward landscape of their habitat by overeating or by polluting, thereby reducing the payoff of certain eating behaviors in that niche (because now there are no juicy or healthy bits nearby), individual creatures, by definition, are assumed to have an epsilon effect on their habitat. Each creature is assumed to adapt to its environment; it doesn’t reshape its fundamentals. A parallel principle at a cognitive level is that creatures who acquire knowledge of structures or patterns in their environment do not change those structures in the process of acquiring their knowledge. Such assumptions permit the use of powerful mathematical formalisms and are useful as a first approximation to our biological and cognitive situation. But in accepting them uncritically we are in danger of overlooking the range of adaptations that are possible, and the many time scales at which these adaptations may be understood. Creatures may create certain dynamics which themselves become the object of adaptation. In the paper, Adapting the World Instead Of Oneself, I explored the implication of this way of thinking for evolutionary biology and psychology. My focus there was on how we change the world to make it a place that fits us better; a study of the dynamic cycle that occurs when we change the world, adapt to that change, and so on.

^Topic List    

5. Context Aware Offices
A theory of interactivity is an important component in our changing understanding of how behavior is organized.It becomes of practical use in the field of HCI and cognitive engineering. As digital workspaces move out of the confines of computer screens and into the environment – on wall screens, in handheld devices, in videoconference units, in furniture – the question arises: how should hybrid environments be designed? A hybrid environment is one where the digital is mingled with the physical; where the virtual is non-intrusively integrated with the material to make it easier for us to work and play, especially in small teams. By adding virtual elements to our environments we expand our collaborative possibilities because users can do things they cannot do in real life. In the most immersive environments they can hold meetings in “outer space,” on the “sea floor” simulated to show the consequences of pollution, inside the “combustion chamber” of an automobile engine, or in their Asian factory in front of the production line. At a more prosaic level these environments can be our ordinary offices. We can project ‘personal metadata’ onto documents so that we can have processors help us find relevant papers such as ‘all the docs Roger gave me this week’ or the papers I was working on when Bryan came in. Hybrid environments should adapt to our specific needs, evolving computational attributes which fit the way we work and think. Since the field is in its infancy we have little experience in creating adaptive rooms -- rooms which dynamically adapt to the workflow needs of participants. Hence there is as yet no clear ontology of objects necessary for designing such rooms and offices. There is a golden opportunity for cognitive scientists interested in interactivity and distributed cognition to shape the basic research agenda. New methodologies for studying workflow in normal activities are required, as are new conceptions of collaboration, coordination and interactivity. I discussed such issues in a 1997 article on Adaptive Rooms, I have been researching the nature of distributed collaborative workflow in a major ONR funded project on Distributed Hybrid Environments for Collaboration, in another project last year we did an ethnographic study of several cogsci staff in their offices. I have begun writing on these issues in the articles entitled Changing the Rules: Architecture in the New Millennium, The Context of Work, and Metacognition, Distributed Cognition and Visual Design.

^Topic List    

6. Information Architecture
In designing large scale information spaces, such as are currently found in corporate intranets or large scale knowledge management systems, the biggest challenge is to discover a set of organizing principles that is at once powerful enough to assign all content a meaningful location and yet simple enough for users to understand. People have to know how to navigate through a system of labels and visual cues to find the information they want. Little science has been done in this area. One problem is that the information a person wants depends substantially on their goals. In general, there will be no single organizational scheme that classifies everything appropriately, since the most meaningful classification will vary with task. This way of looking at the problem of information architecture puts it squarely in the camp of cognitive science. For the best way to structure information environments now depends on understanding the cognitive workflow of users as they undertake their different jobs. As we graduate from computer screen environments to 3D environments where information may be made available in new ways the problem only intensifies. We need principles, guidelines, and experimental and other methodologies to explore the problem of information design. I made a small start on this problem with my long standing post doctoral student Thomas Rebotier by testing to see if people differ in their judgments of classification. We built an interactive exercise (which I use in my 187A class) in which students are asked to classify 50 words into 5 or 6 categories. They do this online in a web based experiment. We then run several clustering algorithms over the proposed categories to see if the students cluster into a few groups who share similar categorization schemes. The results so far support the view that students vary in meaningful ways. One application of this study might be a method for predicting which category scheme a given user prefers. Another may be to study cultural biases in classification. If this can be inferred quickly it may be possible to adapt or customize the information architecture of a site to meet the biases of each user. This latter idea was explored in a research contract from France Telecom.

^Topic List    

7. Elearning environments
How should we put to use our improved understanding of interactivity and coordination to design learning environments where students can be creative but still master the core knowledge requirements necessary for informed reasoning? For the last 5 years while teaching my UCSD classes from Washington DC I worked hard at designing a learning environment that would allow me to teach more effectively in a remote but synchronous manner. Most research on elearning has focused on distributed asynchronous learning. This pushes the teacher into the role of a talking head who exists to help students on issues that are anticipated.My objective has been to improve the quality of teaching in synchronous contexts providing we can assume students are sitting in front of computers. The simplest method for doing this, once we get past the need to have a learning shell which can house content in a dynamic database driven manner, is to provide students with interactive exercises for them to do during class time. This increases their motivation to learn from the teacher, since now they have real concerns. It also allows for a more tutorial like style of teaching, which is by definition is highly interactive and social. My classes can be found online: cogs187A, cogs187B, cogs25. Be sure to look at the collaborative environments that student teams use in 187B. To login to any of these you may use the login name of visitor and password test. See the brief discussion later for a description of the functionality of the major learning environments I have designed and implemented.

[ top ]