Situating Instructions

Abstract: A videographic study of origami is presented in which subjects were observed making four different origami objects under five modes of instruction: photos + captions, illustrations-only, illustrations with small captions, illustrations with large captions, and text-only as control. The objective of the study was to explore the gestures and other actions that subjects produce as they try to follow instructions rather than to determine the most effective style of instruction per se. We found that the task of situating instructions to the context at hand is error prone and that to facilitate it subjects gesture, point, re-orient illustrations, and generally do things that have no function other than to change the epistemic and interactive landscape of activity so they can more easily understand what is to be done. These studies bear on the new questions designers are asking about the placement, timing, and pace of instructions that digital aids now provide and on the fundamental question of how humans embed themselves in an activity by framing their task in a situation specific manner.

Design and evaluation of a wireless electronic health records system for field care in mass casualty settings

Abstract:
Background There is growing interest in the use of technology to enhance the tracking and quality of clinical information available for patients in disaster settings. This paper describes the design and evaluation of the Wireless Internet Information System for Medical Response in Disasters (WIISARD).
Materials and Methods WIISARD combined advanced networking technology with electronic triage tags that reported victims' position and recorded medical information, with wireless pulse-oximeters that monitored patient vital signs, and a wireless electronic medical record (EMR) for disaster care. The EMR system included WiFi handheld devices with barcode scanners (used by front-line responders) and computer tablets with role-tailored software (used by managers of the triage, treatment, transport and medical communications teams). An additional software system provided situational awareness for the incident commander. The WIISARD system was evaluated in a large-scale simulation exercise designed for training first responders. A randomized trial was overlaid on this exercise with 100 simulated victims, 50 in a control pathway (paper-based), and 50 in completely electronic WIISARD pathway. All patients in the electronic pathway were cared for within the WIISARD system without paper-based workarounds.
Results WIISARD reduced the rate of the missing and/or duplicated patient identifiers (0% vs 47%, p<0.001). The total time of the field was nearly identical (38:20 vs 38:23, IQR 26:53–1:05:32 vs 18:55–57:22).
Conclusion Overall, the results of WIISARD show that wireless EMR systems for care of the victims of disasters would be complex to develop but potentially feasible to build and deploy, and likely to improve the quality of information available for the delivery of care during disasters.

Creative Cognition in Choreography

Abstract: Contemporary choreography offers a window onto creative processes that rely on harnessing the power of sensory systems. Dancers use their body as a thing to think with and their sensory systems as engines to simulate ideas non-propositionally. We report here on an initial analysis of data collected in a lengthy ethnographic study of the making of a dance by a major choreographer and show how translating between different sensory modalities can help dancers and choreographer to be more creative.

Comparing tangible and virtual exploration of archaeological objects

Abstract: Can virtual engagement enable the sort of interactive coupling with objects enjoyed by archaeologists who are physically present at a site? To explore this question I consider three points: 1) Tangible interaction: What role does encounter by muscle and sinew playin experiencing and understanding objects? 2) Thinking with things. What sorts of interactions are involved when we manipulate things to facilitate thought? 3) Projection and imagination. Archaeological inquiry involves processes beyond perception. Material engagement of things stimulates these processes. What must be present in a virtual environment to recreate the feel of material engagement? I conclude that nothing, in principle, prevents future virtual environments from supporting this material engagement of digital versions of artifacts. But, there is much that remains to be understood about how to realize this material engagement, both at a technological and a cognitive level.

Thinking With The Body

Abstract: To explore the question of physical thinking -- using the body as an instrument of cognition -- we collected extensive video and interview data on the creative process of a noted choreographer and his company as they made a new dance. A striking case of physical thinking is found in the phenomenon of marking. Marking refers to dancing a phrase in a less than complete manner. Dancers mark to save energy. But they also mark to explore the tempo of a phrase, or its movement sequence, or the intention behind it. Because of its representational nature, marking can serve as a vehicle for thought. Importantly, this vehicle is less complex than the version of the same phrase danced "full-out". After providing evidence for distinguishing different types of marking, three ways of understanding marking as a form of thought are considered: marking as a gestural language for encoding aspects of a target movement, marking as a method of priming neural systems involved in the target movement, and marking as a method for improving the precision of mentally projecting aspects of the target.

Thinking with External Representations

Abstract: Why do people create extra representations to help them make sense of situations, diagrams, illustrations, instructions and problems? The obvious explanation - external representations save internal memory and computation is only part of the story. I discuss seven ways external representations enhance cognitive power: they change the cost structure of the inferential landscape; they provide a structure that can serve as a shareable object of thought; they create persistent referents; they facilitate re-representation; they are often a more natural representation of structure than mental representations; they facilitate the computation of more explicit encoding of information; they enable the construction of arbitrarily complex structure; and they lower the cost of controlling thought - they help coordinate thought. Jointly, these functions allow people to think more powerfully with external representations than without. They allow us to think the previously unthinkable.

Choreographic Methods for Creating Novel, High Quality Dance

Abstract: We undertook a detailed ethnographic study of the dance creation process of a noted choreographer and his distinguished troupe. All choreographer dancer interactions were video-ed, the choreographer and dancers were interviewed extensively each day, as well as other observations and tests performed. The choreographer used three main methods to produce high quality and novel content: showing, making-on, and tasking. We present, analyze and evaluate these methods, and show how these approaches allow the choreographer to increase the creative output of the dancers and himself. His methods, although designed for dance, apply more generally to other creative endeavors, especially where brainstorming is involved, and where the creative process is distributed over many individuals. His approach is also a case study in multi-modal direction, owing to the range of mechanisms he uses to communicate and direct.

Interaction, External Representation and Sense Making

Abstract: Why do people create extra representations to help them make sense of situations, diagrams, illustrations, instructions and problems? The obvious explanation–external representations save internal memory and computation–is only part of the story. I discuss eight ways external representations enhance cognitive power: they provide a structure that can serve as a shareable object of thought; they create persistent referents; they change the cost structure of the inferential landscape; they facilitate re-representation; they are often a more natural representation of structure than mental representations; they facilitate the computation of more explicit encoding of information; they enable the construction of arbitrarily complex structure; and they lower the cost of controlling thought – they help coordinate thought.

Projection, Problem Solving and Anchoring

Abstract: When people make sense of situations, illustrations, instructions and problems they do more than just think with their heads. They gesture, talk, point, annotate, make notes and so on. What extra do they get from interacting with their environment in this way? To study this fundamental problem, I looked at how people project structure onto geometric drawings, visual proofs, and games like tic tac toe. Two experiments were run to learn more about projection. Projection is a special capacity, similar to perception, but less tied to what is in the environment. Projection, unlike pure imagery, requires external structure to anchor it, but it adds ‘mental’ structure to the external scene much like an augmented reality system adds structure to an outside scene. A person projects when they look at a chessboard and can see where a knight may be moved. Because of the cognitive costs of sustaining and extending projection, humans make some of their projections real. They create structure externally. They move the piece, they talk, point, notate, represent. Much of our interactivity during sense making and problem solving involves a cycle of projecting then creating structure.

Knowledge, Implicit vs Explicit

Abstract: In the scientific study of mind a distinction is drawn between explicit knowledge–knowledge that can be elicited from a subject by suitable inquiry or prompting, can be brought to consciousness, and externally expressed in words–and implicit knowledge–knowledge that cannot be elicited, cannot be made directly conscious, and cannot be articulated. Michael Polanyi (1967) argued that we usually ‘know more than we can say’. The part we can articulate is explicitly known; the part we cannot is implicit.

Problem Solving and Situated Cognition

Abstract: In the course of daily life we solve problems often enough that there is a special term to characterize the activity and the right to expect a scientific theory to explain its dynamics. The classical view in psychology is that to solve a problem a subject must frame it by creating an internal representation of the problem‘s structure, usually called a problem space. This space is an internally generable representation that is mathematically identical to a graph structure with nodes and links. The nodes can be annotated with useful information, and the whole representation can be distributed over internal and external structures such as symbolic notations on paper or diagrams. If the representation is distributed across internal and external structures the subject must be able to keep track of activity in the distributed structure. Problem solving proceeds as the subject works from an initial state in this mentally supported space, actively construction possible solution paths, evaluating them and heuristically choosing the best. Control of this exploratory process is not well understood, as it is not always systematic, but various heuristic search algorithms have been proposed and some experimental support has been provided for them.

Explaining Artifact Evolution

Abstract: Much of a culture’s history–its knowledge, capacity, style, and mode of material engagement–is encoded and transmitted in its artifacts. Artifacts crystallize practice; they are a type of meme reservoir that people interpret though interaction. So, in a sense, artifacts transmit cognition; they help to transmit practice across generations, shaping the ways people engage and encounter their world. So runs one argument.

 

Distributed Cognition, A Methodological Note

Abstract: Humans are closely coupled with their environments. They rely on being ‘embedded’ to help coordinate the use of their internal cognitive resources with external tools and resources. Consequently, everyday cognition, even cognition in the absence, may be viewed as partially distributed. As cognitive scientists our job is to discover and explain the principles governing this distribution: principles of coordination, externalization, and interaction. As designers our job is to use these principles, especially if they can be converted to metrics, in order to invent and evaluate candidate designs. After discussing a few principles of interaction and embedding I discuss the usefulness of a range of metrics derived from economics, computational complexity and psychology.

Multi-tasking and Cost Structure: Implications for Design,

Abstract: I argue that it is not possible to accurately represent our task settings as closed environments with a single well defined cost structure. Natural environments are places where many things are done, often at the same time, and often by many people. To appreciate the way such invariants of everyday life affect design I present a case study, a micro-analysis of espresso making at Starbucks to show the challenges facing a cost structure approach.

Metacognition, Distributed Cognition and Visual Design

Abstract: Metacognition is associated with planning, monitoring, evaluating and repairing performance Designers of elearning systems can improve the quality of their environments by explicitly structuring the visual and interactive display of learning contexts to facilitate metacognition. Typically page layout, navigational appearance, visual and interactivity design are not viewed as major factors in metacognition. This is because metacognition tends to be interpreted as a process in the head, rather than an interactive one. It is argued here, that cognition and metacognition are part of a continuum and that both are highly interactive. The tenets of this view are explained by reviewing some of the core assumptions of the situated and distribute approach to cognition and then further elaborated by exploring the notions of active vision, visual complexity, affordance landscape and cue structure. The way visual cues are structured and the way interaction is designed can make an important difference in the ease and effectiveness of cognition and metacognition. Documents that make effective use of markers such as headings, callouts, italics can improve students' ability to comprehend documents and 'plan' the way they review and process content. Interaction can be designed to improve 'the proximal zone of planning' - the look ahead and apprehension of what is nearby in activity space that facilitates decisions. This final concept is elaborated in a discussion of how e-newspapers combine effective visual and interactive design to enhance user control over their reading experience.

Implicit and Explicit Representation

Abstract: The degree to which information is encoded explicitly in a representation is related to the computational cost of recovering or using the information. Knowledge that is implicit in a system need not be represented at all, even implicitly, if the cost of recovering it is prohibitive.

When is Information Explicitly Represented?

Abstract: Computation is a process of making explicit, information that was implicit. In computing 5 as the solution to , for example, we move from a description that is not explicitly about 5 to one that is. We are drawing out numerical consequences of the description . We are extracting information implicit in the problem statement. Can we precisely state the difference between information that is implicit in a state, structure or process and information that is explicit?

The Context of Work

Abstract: The question of how to conceive and represent the context of work is explored from the theoretical perspective of distributed cognition. It is argued that to understand the office work context we need to go beyond tracking superficial physical attributes such as who or what is where when, and consider the state of digital resources, people’s concepts, task state, social relations and the local work culture, to name a few. In analyzing an office more deeply three concepts are especially helpful: entry points, action landscapes, and coordinating mechanisms. An entry point is a structure or cue that represents an invitation to enter an information space or office task. An activity landscape is part mental construct and part physical; it is the space users interactively construct out of the resources they find when trying to accomplish a task. A coordinating mechanism is an artifact, such as a schedule, or clock, or an environmental structure such as the layout of papers to be signed, which helps a user manage the complexity of his task. Using these three concepts we can abstract away from many of the surface attributes of work context and define the deep structure of a setting—the invariant structure that many office settings share. A long term challenge for context-aware computing is to operationalize these analytic concepts.

Changing the Rules: Architecture in the New Millenium

Abstract: Walls, work surfaces and furniture can now all be digitally enhanced. The position presented is that these emerging technologies are radically changing the design constraints of architecture. The article has four parts. First, the cognitive conception of an activity space is critically evaluated to show that the idea of an activity space must be broadened to accommodate actions that are not normally viewed as task relevant. Second, the different properties of physical and digital objects are explained and morals drawn about the way digital objects can be used to enhance physical environments. Three specific examples of digital enhancement are then discussed: telepresence, 3 dimensional intranets, and intelligent furniture. In the fourth section the architectural significance of these types of digital enhancements is shown by discussing how Christopher Alexander’s nine tenets of workspace design must be reconceptualized. The conclusion drawn is that the customary rules of architectural composition are changing.

 

A Few Thoughts on Cognitive Overload

Abstract: This article addresses three main questions: What causes cognitive overload in the workplace? What analytical framework should be used to understand how agents interact with their work environments? How can environments be restructured to improve the cognitive workflow of agents? Four primary causes of overload are identified: too much information supply, too much information demand, constant multitasking and interruptions, and inadequate workplace infrastructure to help reduce the need for planning, monitoring, reminding, reclassifying information, etc... The first step in reducing the cognitive impact of these causes is to enrich classical frameworks for understanding work environments, such as Newell and Simon’s notion of a task environment, by recognizing that our actual workplace is a superposition of many specific environments—activity spaces—which we slip between. Each has its own cost structure arising from the tools and resources available, including the cognitive strategies and interpretational frameworks of individual agents. These cognitive factors are significant, affecting how easy of difficult it is to perform an action, such as finding a specific paper in a ‘messy’ desk. A few simple examples show how work environments can be redesigned and how restructuring can alter the cost structure of activity spaces.

Distributed Cognition: Towards a New Foundation for Human-Computer Interaction Research

Abstract: We are quickly passing through the historical moment when people work in a front of a single computer, dominated by a small CRT and focused on tasks involving only local information. Networked computers are becoming ubiquitous and are playing increasingly significant roles in our lives and in the basic infrastructures of science, business, and social interaction. For human-computer interaction to advance in the new millennium we need to better understand the emerging dynamic of interaction in which the focus task is no longer confined to the desktop but reaches into a complex networked world of information and computer-mediated interaction. We think the theory of distributed cognition has a special role to play in understanding interactions between people and technologies, for its focus has always been on whole environments: what we really do in them and how we coordinate our activity in them. Distributed cognition provides a radical reorientation of how to think about designing and supporting human-computer interaction. As a theory it is specifically tailored to understanding the interactions among people and technologies. In this article we propose distributed cognition as a new foundation for human-computer interaction, sketch an integrated research framework, and use selections from our earlier work to suggest how this framework can provide new opportunities in the design of digital work materials.

Distributed Cognition, Coordination and Evironmental Design

Abstract: The type of principles which cognitive engineers need to design better work environments are principles which explain interactivity and distributed cognition: how human agents interact with themselves and others, their work spaces, and the resources and constraints that populate those spaces. A first step in developing these principles is to clarify the fundamental concepts of environment, coordination, and behavioural function. Using simple examples, I review changes the distributed perspective forces on these basic notions.

Interactive skill in Scrabble

Abstract: An experiment was performed to test the hypothesis that people sometimes take physical actions to make themselves more effective problem solvers. The task was to generate all possible words that could be formed from seven Scrabble letters. In one condition, participants could use their hands to manipulate the letters, and in another condition, they could not. Results show that more words were generated with physical manipulation than without. However, an interaction was obtained between the physical manipulation conditions and the specific letter sets chosen, indicating that physical manipulation helps more for generating words in some circumstances than in others. Overall, our findings can be explained in terms of an interactive search process in which external, physical activity effectively complements internal, cognitive activity. Within this framework, the interaction can be explained in terms of the relative difficulty of generating words from the letters given in the different sets.

Utility Curves: Mean Opinion scores considered biased

Abstract: Mechanisms for QoS provisioning in communication networks range from flow‒based resource reservation schemes, proving QoS guarantees, though QoS differentiation based on reservation aggregation techniques to adaptation of applications, compensating for incomplete reservations. Scalabe, aggregation⁶based rervations can also be combined with adaptation for a more flexible and robust overall QoS provisioning. Adaptation is particularly important in wireless networks, where reservations schemes are more difficult to realize. It is widely accepted that usability of Cellular or Mobile IP can be largely improved if adaptation is incorporated. Adaptation, in turn, depends on coding techniques involved and on user requirements. Those impose certain preferences and ordering relations on QoS hierarchies or degradation path is to be followed along when adaptation is performed. With the advent of MPEG4 coding techniques, more sophisticated adaptation schemes are more likely to be supported in the future. The problem still remains as how adaptation can be effectively controlled. Utility curves (UCs) have been suggested as such a mechanism. But there is a notable lack of knowledge on how utility curves can effectively be determined in a way sensible reflecting users’ needs. UCs provide a formal technique to directly relate network state, such as available bandwidth, to end‒user perceived QoS. The relation is established by tests incorporating questionnaires to find out about users’ opinions on certain aspects of the media’s qualities presented. The quantitative result of such an assessment is called a Mean Opinion Score, MOS for short. An example would be a subjective MOS about when a skew between different media like audio and video becomes visible and notable to some subjects.

Adaptable Rooms and Cognitive Workflow

Abstract: This paper introduces the concept of Adaptive Rooms, which are virtual environments able to dynamically adapt to users needs, including physical and cognitive workflow requirements, number of users, differing cognitive abilities and skills. Adaptive rooms are collections of virtual objects, many of them self‒transforming objects, housed in an architecturally active room with information spaces and tools. An ontology of objects used in adaptive rooms is presented. Virtual entities are classified as passive, reactive, active, and information entities, and their sub‒categories. Only active objects can be self-transforming. Adaptive Rooms are meant to combine the insights of ubiquitous computing—that computerization should be everywhere, transparently incorporated—with the insights of augmented reality—that everyday objects can be digitally enhanced to carry more information about their use. To display the special potential of adaptive rooms, concrete examples are given to show how the demands of cognitive workflow can be reduced.

Worldlets: 3D Thumbnails for 3D Browsing

Abstract: Dramatic advances in 3D Web technologies have recently led to widespread development of virtual world Web browsers and 3D content. A natural question is whether 3D thumbnails can be used to find ones way about such 3D content the way that text and 2D thumbnail images are used to navigate 2D Web content. We have conducted an empirical experiment that shows 3D thumbnails, which we call worldlets, improve travelers landmark knowledge and expedite wayfinding in virtual environments.

Interactivity and Multimedia Interfaces

Abstract: Multimedia technology offers instructional designers an unprecedented opportunity to create richly interactive learning environments. With greater design freedom comes complexity. The standard answer to the problems of too much choice, disorientation, and complex navigation is thought to lie in the way we design the interactivity in a system. Unfortunately, the theory of interactivity is at an early stage of development. After critiquing the decision cycle model of interaction – the received theory in human computer interaction – I present arguments and observational data to show that humans have several ways of interacting with their environments which resist accommodation in the decision cycle model. These additional ways of interacting include: preparing the environment, maintaining the environment, and reshaping the cognitive congeniality of the environment. Understanding how these actions simplify the computational complexity of our mental processes is the first step in designing the right sort of resources and scaffolding necessary for tractable learner controlled learning environments.

Worldlets: 3D Thumbnails for Wayfinding in Virtual Environments

Abstract: Virtual environment landmarks are essential in wayfinding: they anchor routes through a region and provide memorable destinations to return to later. Current virtual environment browsers provide user interface menus that characterize available travel destinations via landmark textual descriptions or thumbnail images. Such characterizations lack the depth cues and context needed to reliably recognize 3D landmarks. This paper introduces a new user interface affordance that captures a 3D representation of a virtual environment landmark into a 3D thumbnail, called a worldlet. Each worldlet is a miniature virtual world fragment that may be interactively viewed in 3D, enabling a traveler to gain first-person experience with a travel destination. In a pilot study conducted to compare textual, image, and worldlet landmark representations within a wayfinding task, worldlet use significantly reduced the overall travel time and distance traversed, virtually eliminating unnecessary backtracking.

Adapting the Environment instead of Oneself

Abstract: This paper examines some of the methods animals and humans have of adapting their environment. Because there are limits on how many different tasks a creature can be designed to do well in, creatures with the capacity to redesign their environments have an adaptive advantage over those who can only passively adapt to existing environmental structures. To clarify environmental redesign I rely on the formal notion of a task environment as a directed graph where the nodes are states and the links are actions. One natural form of redesign is to change the topology of this graph structure so as to increase the likelihood of task success or to reduce its expected cost, measured in physical terms. This may be done by eliminating initial states hence eliminating choice points; by changing the action repertoire; by changing the consequence function; and lastly, by adding choice points. Another major method for adapting the environment is to change its cognitive congeniality. Such changes leave the state space formally intact but reduce the number and cost of mental operations needed for task success; they reliably increase the speed, accuracy or robustness of performance. The last section of the paper describes several of these epistemic or complementary actions found in human performance.

Complementary strategies: Why we use our hands when we think

Abstract: A complementary strategy can be defined as any organizing activity which recruits external elements to reduce cognitive loads. Typical organizing activities include pointing, arranging the position and orientation of nearby objects, writing things down, manipulating counters, rulers or other artifacts that can encode the state of a process or simplify perception. To illustrate the idea of a complementary strategy, a simple experiment was performed in which subjects were asked to determine the dollar value of collections of coins. In the no-hands condition, subjects were not allowed to touch the coin images or to move their hands in any way. In the hands condition, they were allowed to use their hands and fingers however they liked. Significant improvements in time and number of errors were observed when S's used their hands over when they did not. To explain these facts, a brief account of some commonly observed complementary strategies is presented, and an account of their potential benefits to perception, memory and attention.

The Intelligent Use of Space

Abstract: The objective of this essay is to provide the beginning of a principled classification of some of the ways space is intelligently used. Studies of planning have typically focused on the temporal ordering of action, leaving as unaddressed questions of where to lay down instruments, ingredients, work-in-progress, and the like. But, in having a body, we are spatially located creatures: we must always be facing some direction, have only certain objects in view, be within reach of certain others. How we manage the spatial arrangement of items around us is not an afterthought: it is an integral part of the way we think, plan, and behave. The proposed classification has three main categories: spatial arrangements that simplify choice; spatial arrangements that simplify perception; and spatial dynamics that simplify internal computation. The data for such a classification is drawn from videos of cooking, assembly and packing, everyday observations in supermarkets, workshops and playrooms, and experimental studies of subjects playing Tetris, the computer game. This study, therefore, focuses on interactive processes in the medium and short term: on how agents set up their workplace for particular tasks, and how they continuously manage that workplace.

 

On Distinguishing Epistemic from Pragmatic Actions

Abstract:

We present data and argument to show that in Tetris—a real‒time interactive video game — certain cognitive and perceptual problems are more quickly, easily, and reliably solved by performing actions in the world rather than by performing computational actions in the head alone. We have found that some translations and rotations are best understood as using the world to improve cognition. These actions are not used to implement a plan, or to implement a reaction; they are used to change the world in order to simplify the problem-solving task. Thus, we distinguish pragmatic actions—actions performed to bring one physically closer to a goal—from epistemic actions—actions performed to uncover information that is hidden or hard to compute mentally.

To illustrate the need for epistemic actions, we first develop a standard information—processing model of Tetris—cognition, and show that it cannot explain performance data from human players of the game–even when we relax the assumption of fully sequential processing. Standard models disregard many actions taken by players because they appear unmotivated or superfluous. However, we describe many such actions that are actually taken by players that are far from superfluous, and that play valuable roles in improving human performance. We argue that traditional accounts are limited because they regard action as having a single function: to change the world. By recognizing a second function of action–an epistemic function–we can explain many of the actions that a traditional model cannot. Although, our argument is supported by numerous examples specifically from Tetris, we outline how the one category of epistemic action can be incorporated into theories of action more generally.

Some Epistemic Benefits of Action: Tetris a Case Study

Abstract: We present data and argument to show that in Tetris—a real‒time interactive video game—certain cognitive and perceptual problems are more quickly, easily, and reliably solved by performing actions in the world rather than by performing computational actions in the head alone. We have found that some translations and rotations are best understood as using the world to improve cognition. They are not being used to implement a plan, or to implement a reaction. To substantiate our position we have implemented a computational laboratory that lets us record keystrokes and game situations, as well as allows us to dynamically create situations. Using the data of over 30 subjects playing 6 games, tachistoscopic tests of some of these subjects, and results frrom our own successful efforts at building expert systems to play Tetris, we show why knowing how to use one's environment to enhance speed and robustness are important components in skilled play.

Perceptive Actions in Tetris, in Control of Selective Perception

Abstract: Cognitive organisms have three rather different techniques for intelligently regulating their intake of environmental information. In order of the time needed to uncover information they are:

1. control of attention: within an image produced by a given sensor certain elements can be selected for additional processing;
2. control of gaze: the orientation and resolution (center of foveation of the sensor can be regulated to crate a new image;
3. control of activity: certain non‒percecptual actions can be performed to increase the probability of unearthing salient information that currently is unavailable, hard to detect, or hard to compute.

In this note we shall discuss some experiments we have been performing on the last variety of active sensing.

Reaction and Reflection in Tetris

Abstract: We present data and argument to show that in Tetris—a real‒time interactive video game—certain cognitive and perceptual problems are more quickly, easily, and reliably solved by performing computational actions in the head alone. We have found that some translations and rotations are best understood as using the world to improve cognition. They are not being used to implement a plan, or to implement a reaction. To substantiate our position we have implemented a computational laboratory that lets us record keystrokes and game situations, as well as allows us to dynamically create situations. Using the data of over 30 subjects playing 6 games, tachistoscopic tests of some of these subjects, and results frrom our own successful efforts at building expert systems to play Tetris, we show why knowing how to use one’s environment to enhance speed and robustness are important components in skilled play.

Architectures of Intelligent Systems

Abstract: Theories of intelligence can be of use to neuroscientists if they:

1. provide illuminating suggestions about the functional architecture of neural systems;
2. suggest specific models of processing that neural circuits might implement.

The objective of our session was to stand back and consider the prospects for this interdisciplinary exchange.

One of the facts that emered early in our discussions was that given our current level of knowledge, it is hard to tie theories and models of intelligence to actual neural machinery....

Learnability and Innate Knowledge of Language

Abstract:
Background It is sometimes argued that if PDP networks can be trained to make correct judgements of grammaticality we have an existence proof that there is enough information in the stimulus to permit learning grammar by inductive means alone. This seems inconsistent superficially with Gold's theorem and at a deeper level with the fact that networks are designed on the basis of assumptions about the domain of the function to be learned. To clarify the issue I consider what we should learn from Gold's theorem, then go on to inquire into what it means to say that knowledge is domain specific. I first try sharpening the intuitive notion of domain specific knowledge by reviewing the alleged difference between processing limitatons due to shartage of resources vs shortages of knowledge. After rejecting different formulations of this idea, I suggest that a model is language specific if it transparently refer to entities and facts about language as opposed to entities and facts of more general mathematical domains. This is a useful but not necessary condition. I then suggest that a theory is domain specific if it belongs to a model family which is attuned in a law-like way to domain regularities. This leads to a comparison of PDP and parameter setting models of language learning. I conclude with a novel version of the poverty of stimulus argument.

Foundations of AI: The Big Issues

Abstract: The objective of research in the foundations of Al is to explore such basic questions as: What is a theory in Al? What are the most abstract assumptions underlying the competing visions of intelligence? What are the basic arguments for and against each assumption? In this essay I discuss five foundational issues: (1) Core Al is the study of conceptualization and should begin with knowledge level theories. (2) Cognition can be studied as a disembodied process without solving the symbol grounding problem. (3) Cognition is nicely described in propositional terms. (4) We can study cognition separately from learning. (5) There is a single architecture underlying virtually all cognition. I explain what each of these implies and present arguments from both outside and inside Al why each has been seen as right or wrong.

Today the Earwig Tomorrow Man?

Abstract: A startling amount of intelligent activity can be controlled without reasoning or thought. By tuning the perceptual system to task relevant properties a creature can cope with relatively sophisticated environments without concepts. There is a limit, however, to how far a creature without concepts can go. Rod Brooks, like many ecologically oriented scientists, argues that the vast majority of intelligent behavior is concept‑free. To evaluate this position I consider what special benefits accrue to concept‑using creatures. Concepts are either necessary for certain types of perception, learning, and control, or they make those processes computationally simpler. Once a creature has concepts its capacities are vastly multiplied.

When is Information Explicitly Represented?

Abstract: Computation is a process of making explicit, information that was implicit. In computing 5 as the solution to ∛125, for example, we move from a description that is not explicitly about 5 to one that is. We are drawing out numerical consequences of the description ∛125. We are extracting information implicit in the problem statement. Can we precisely state the difference between information that is implicit in a state, structure, or process and information that is explicit?

Putting a Price on Cognition

Abstract: In this essay I shall consider a certain methodological claim gaining currency in connectionist quarters: The claim that variables are costly to implement in PDP systems and hence are not likely to be as important in cognitive processing as orthodox theories of cognition assume

Preparation et Improvisation

Abstract: Considérons un plan d’action comme une méthode permettant de contraindre, de façon rationnelle, une séquence de choix (d’actions) locaux. Le rôle d’un dispositive de planification est d’orienter ces choix de telle sorte que les buts globaux qu’il vise à réaliser soient atteients. Si le dispositive est supposé agir dans un monde simple, il pourra parcourir exhaustivement l’ensemble des séquences d’actions localement plausibles. Par contre, s’il est suppose agrir dans un monde qui se rapproche de la richesse et de la complexité du monde réel, la modeéisation se heurte au problemès suivants:
— Le monde comport beaucoup trop de détails pertinents pour qu’il soit possible d’en donner des représentations simples dans un modèle donné. Par voie de conséquence, il ne sera pas possible, au moment de la prise de décision, de prédire de façon correcte quelles actions seront plausibles.
— Le monde est ouvret à des év!nements non prédictibles. On ne peut donc se fier à un calcul des conséquence possibles des actions.
— Quand un choix se presénte, le nombre des actions plausibles est si grand qu’il est impossible de planifier une séquence d’actions, étant donnée l’imporance du temps de calcul exigé.