My contributions to date have
been in each of these seven areas, though my greatest effort has been
on the first four, leaving questions and analyses about the office of
the future and information architecture for my future work.
Methodology
Several years ago I constructed a computational environment for studying
Tetris which allows non-intrusive tracking of a subject's interactions
with a Tetris game at millisecond levels. We recorded the state of the
game at each moment and the type of action undertaken. Massive amounts
of data were gathered and analyzed on people playing Tetris for tens of
hours. This gave us a virtually unique data set of how people move from
novices in an interactive game to becoming experts 10 to 20 hours later.
Several companies expressed interest in using this environment in their
research labs and Interval may still be using it. The method of recording
large amounts of data and then data mining on the basis of logical queries
is now used in several research projects around the country. I have also
grappled with the challenges of developing a reliable methodology for
experimentally studying how people play and improve at jigsaw, counting
coins, solving block problems interactively. I made solid advances in
studying selected areas of activity and have published most of my results
(On
Distinguishing Epistemic From Pragmatic Action, Interactive Skill
in Scrabble , Complementary strategies: why we use our hands when we think
– the work on comparing strategy evolution between young and old is still
in preparation). However, I regard the general problem of how experimentally
to study the natural evolution of cognitive strategies as inadequately
solved and an important topic for future research. A closely related problem
arises in Human Computer Interaction where experimental approaches seem
to leech out most of the real life aspects of our way of interacting and
adapting to computational environments, and so requires new methodological
principles.
One of these new methods which I am developing involves creating special
digital capture tools for recording the activity of teams in what may
be called semi-structured contexts. In this approach, subjects are put
in a reasonably natural context then assigned specific tasks and asked
to perform those tasks under varying conditions. A huge amount of data
is collected this way: video via camcorders, video of desktop activity,
separate audio recordings of each person. To make sense of all this data
we have designed an activity review tool which takes 4 tracks of synchronized
audio-video recordings and allows us to machine review the 4 tracks looking
for onsets of conversation, and dozens of other machine identifiable events.
The system has been designed further to allow investigators to review
the tapes and add new lines of commentary. This allows us to lay down
dozens of tracks of ethnographic coding. When these annotations are examined
in conjunction with the machine annotations we are able to increase the
complexity of our activity analysis. The design and implementation of
this activity review tool has been undertaken in conjunction with Dave
Nadeau at the San Diego SuperComputer Center and is supported by the ONR.
The nature of cognitive coupling
The general problem of cognitive coupling, and coordination more
particularly, is a field rich in possibilities. So far I have undertaken
three sets of studies that have contributed to our understanding of the
time course of adaptations we make to our environment, and the computational
advantage of making such adaptations.
a. The first set examines the ways humans use the space around them
strategically to help them in their tasks (The
Intelligent Use of Space, Complementary strategies: why we use our
hands when we think, and in Adapting the World Instead of Ourselves).
I showed how the idea of amortized complexity could be used as an explanation
of why people prepare their environments and I classified the major ways
both humans and, at times, animals have of adapting the world to improve
their overall adaptive advantage.
b. In the second set I have begun to distinguish forms of coordinative
mechanisms and am now pursuing this in the context of command centers
for disaster management. I reported on this in my conference opening lecture
at the European Cognitive Science Society (Distributed
Cognition, Coordination and Environment Design). This work builds
on the concepts of epistemic and complementary actions I introduced a
few years ago (On
Distinguishing Epistemic from Pragmatic Actions, Complementary
strategies: why we use our hands when we think, Adapting
the World Instead of Ourselves, and Are All Actions Performative or
Exploratory).
c. In some new studies on the nature of interruption and mechanisms
for recovery, (part of which is discussed in A
Few Thoughts on Cognitive Overload) I am exploring how interruption
is related to frames of activity. People can multitask if they are prepared,
but stress and cognitive overload occur the more we have to rely on ourselves
alone to re-establish a tight coupling with our environments.
d. Finally, in examining the problem of metacognition in elearning
environments I have considered how visual design can affect coordination.
An initial report of this can be found in Metacognition,
Distributed Cognition and Visual Design.
How interactive strategies evolve
I am currently completing a paper with Larraine MacDonald on how old
and young develop strategies for improving visual search and memory on
a pick and place task. We conducted this experiment over a two year period
and will be submitting the paper soon. It required administering fairly
extensive neuropsychological pretests, a recall post test. Coordination
with elderly subjects proved more time consuming than we expected. Another
paper that is in progress is a write up of experiments on interruption,
how people initially react and then adapt to an environment in which interruption
is common.
How a theory of interactivity fits with the
principles of evolutionary biology
Although I have been interested in the nature of adaptation at
different time scales in much of my experimental and analytical work and
reported on it in many papers, the only paper in which I explored the
implications of the theory for evolutionary biology was in the paper Adapting
the World Instead of Ourselves. I believe this paper was well received
in the zoological world as immediately after its publication I received
letters from several groups of evolutionary biologists at Oxford, Harvard,
Princeton and Stanford who, apparently, are now turning to this topic
at a non-cognitive level.
Context Aware venues/Office of the future
In Adaptive
Rooms, Virtual Collaboration and Cognitive Workflow, I articulated
an ontology of objects necessary for designing adaptive rooms. Virtual
entities can be classified as passive, reactive, active and information
tools and spaces. Self transforming objects are a special type of active
object. Using this ontology it is possible to define an adaptive room
as a collection of self-transforming objects situated in an architecturally
active room containing information spaces and tools. Well designed adaptive
rooms combine the insights of ubiquitous computing – that computerization
should be everywhere, transparently incorporated – with the insights of
augmented reality – that everyday objects can be digitally enhanced to
carry more information about their use. The result is an integrated virtual
environment in which people can cooperate and collaborate on projects
in a way that leverages the simulation strengths of computers. This work
was picked up by Ted Krueger, Director of Information Technology in the
School of Architecture, University of Arkansas and as a result I was invited
to contribute to the Architecture journal Convergence. The paper I wrote
“Changing
the Rules: Architecture in the Millennium is an examination of how
changes in technology are making irrelevant some of the architectural
principles discussed by Alexander.
The theme of how context aware technology may change our conceptions of
design is explored further in the article The
Context of Work. In that article I introduce the idea of an ‘entry
point’ to characterize the attractors in a ‘space’ where users can jump
off into another ‘space’.
On a practical side, I have made a digital glass wall to join the two
sections of my lab which are physically divided. I will soon be increasing
the complexity of that design in a joint project with Calit2.
Information architecture
I have yet to publish on this topic in journals. I have written several
technical documents for the Worldbank on the design principles of their
intranet but none of these are in the public domain. My contributions
on this topic, therefore, should be seen as practical. I designed and
oversaw the implementation of a large intranet (with more than 350,000
pages). This design is of interest, not just because of its scale, but
because it focuses on the knowledge management needs of a major knowledge
producer: the Worldbank. It is still the design which the WorldBank uses
after 4 years. I also designed the information architecture for the Worldbank’s
external website in 1998-9. (It has more than 100,000 pages), and most
recently I designed the architecture of the Global Development Gateway
with several hundreds of thousands of pages. I believe these products
count as intellectual contributions, much as patents do.
Elearning Environments
I spent a significant part of the last 5 years building large web based
software systems for elearning. My systems have gone through several redesigns
and will no doubt go through more. The biggest efforts though were spent
on designing a large course and learning management system and a team
collaboration environment. Both were written in asp and sql.
a. To review the learning management system visit cogsci 187A – Cognitive Aspects of Multimedia
Design, and Cogsci 25 – Intro to Web Programming
To see how to create a course visit our test site. Try
this out login as visitor (password =
test).
This entire system, containing hundreds of asp files and tens of thousands
of lines of asp code is dynamic, driven by an SQL database and completely
customizable. By storing all data in a database we are able to create
and manage an arbitrary number of courses, and to make it simple for teachers
to build new courses quickly. This shell is currently being tested by
several faculty throughout UCSD campus and will soon be in use for many
courses in 6th college, and some at UC Irvine. It has been
designed with ease of use in mind, and the needs of professors as well
as students. On the teacher’s side I have incorporated principles of instructional
design to help professors design better lectures and structure their courses
more powerfully. I studied the workflow of teachers and introduced features
to make it easier to manage TA’s and simplify grading. Students have a
secure portfolio to collect their work and to have private discussions
with their teachers. To reduce the burden of grading, the system has been
designed to support portfolio based assessment, student grading of other
students, quiz taking and auto-grading, and team based work.
b. To review the collaborative shell for team based projects look are some of
the ways students have used the shell. Login as visitor (password
= test)
This team collaborative system was designed to allow small to medium sized
teams of students to work together remotely with each other and a client.
I interviewed students taking my project based class that the hardest
thing they found was getting together to work on their project together.
This application grew from a simple shared document management system
into a dynamic tool that allows team members to define their own navigation,
to comment on gallery images they uploaded, and to comment on files and
announcements. Task assignment and tracking is another major feature of
the system. Students can assign each other tasks, dues dates, they can
comment on their tasks, accept, reject and reassign them. Again the design
of this tool was based on workflow analysis of small teams.
[
top ]
|