Cognitive Architectures:

Research Issues and Challenges

Pat Langley
Computational Learning Laboratory
Center for the Study of Language and Information
Stanford University, Stanford, CA 94305

John E. Laird
EECS Department
The University of Michigan
1101 Beal Avenue
Ann Arbor, MI 48109

Seth Rogers
Computational Learning Laboratory
Center for the Study of Language and Information
Stanford University, Stanford, CA 94305

Abstract
In this paper, we examine the motivations for research on cognitive architectures and review some
candidates that have been explored in the literature. After this, we consider the capabilities that a
cognitive architecture should support, some properties that it should exhibit related to representa-
tion, organization, performance, and learning, and some criteria for evaluating such architectures
at the systems level. In closing, we discuss some open issues that should drive future work in this
important area.

Keywords: cognitive architectures, intelligent systems, cognitive processes

Draft: Please do not quote without permission; comments welcome.

Cognitive Architectures Page 1

1. Background and Motivation

A cognitive architecture specifies the underlying infrastructure for an intelligent system. Briefly,
an architecture includes those aspects of a cognitive agent that are constant over time and across
different application domains. These typically include:

• the short-term and long-term memories that store content about the agent’s beliefs, goals, and
knowledge;
• the representation of elements that are contained in these memories and their organization into
larger-scale mental structures;
• the functional processes that operate on these structures, including the performance mecha-
nisms that utilize them and the learning mechanisms that alter them.

Because the contents of an agent’s memories can change over time, one would not consider the
knowledge and beliefs encoded therein to be part of that agent’s architecture. Just as different
programs can run on the same computer architecture, so different knowledge bases and beliefs can
be interpreted by the same cognitive architecture. There is also a direct analogy with a building’s
architecture, which consists of permanent features like its foundation, roof, and rooms, rather than
its furniture and appliances, which one can move or replace.
As we will see, alternative cognitive architectures can differ in the specific assumptions they
make about these issues, just as distinct buildings differ in their layouts. In addition to making
different commitments about how to represent, use, or acquire knowledge and beliefs, alternative
frameworks may claim that more or less is built into the architectural level, just as some buildings
embed shelves and closets into their fixed structures, whereas others handle the same functions
with replaceable furniture.
Research on cognitive architectures is important because it supports a central goal of artificial
intelligence and cognitive science: the creation and understanding of synthetic agents that support
the same capabilities as humans. Some work focuses on modeling the invariant aspects of human
cognition, whereas other efforts view architectures as an effective path to building intelligent agents.
However, these are not antithetical goals; cognitive psychology and AI have a long history of building
on the other’s ideas (Langley, 2006), and research on cognitive architectures has played
general intelligence a keytool
vs an intelligent role
in this beneficial interchange.
In some ways, cognitive architectures constitute the antithesis of expert systems, which provide
skilled behavior in narrowly defined contexts. In contrast, architectural research aims for breadth of
coverage across a diverse set of tasks and domains. More important, it offers accounts of intelligent
behavior at the systems level, rather than at the level of component methods designed for specialized
tasks. Newell (1973a) has argued persuasively for systems-level research in cognitive science and
artificial intelligence, claiming “You can’t play 20 questions with nature and win”. Instead of
carrying out micro-studies that address only one issue at a time, we should attempt to unify many
findings into a single theoretical framework, then proceed to test and refine that theory.
Since that call to arms, there has been a steady flow of research on cognitive architectures. The
movement was associated originally with a specific class of architectures known as production sys-
tems (Newell, 1973b; Neches et al., 1987) and emphasized explanation of psychological phenomena,
Page 2 Cognitive Architectures

with many current candidates still taking this form and showing similar concerns. However, over the
past three decades, a variety of other architectural classes have emerged, some less concerned with
human behavior, that make quite different assumptions about the representation, organization,
utilization, and acquisition of knowledge. At least three invited symposia have brought together
researchers in this area (Laird, 1991; VanLehn, 1991; Shapiro & Langley, 2004), and there have been
at least two edited volumes (Sun, 2005; VanLehn, 1991). The movement has gone beyond basic
research into the commercial sector, with applications to believable agents for simulated training
environments (e.g., Tambe et al., 1995), computer tutoring systems (Koedinger, Anderson, Hadley,
& Mark, 1997), and interactive computer games (e.g., Magerko et al., 2004).
Despite this progress, there remains a need for additional research in the area of cognitive archi-
tectures. As artificial intelligence and cognitive science have matured, they have fragmented into a level
systems
number of well-defined subdisciplines, each with its own goals and its own criteria for evaluation.
Yet commercial and government applications increasingly require integrated systems that exhibit
intelligent behavior, not just improvements to the components of such systems. This demand can
be met by an increased focus on system-level architectures that support complex cognitive behavior
across a broad range of relevant tasks.
In this document, we examine some key issues that arise in the design and study of integrated
cognitive architectures. Because we cannot hope to survey the entire space of architectural the-
ories, we focus on the ability to generate intelligent behavior, rather than matching the results
of psychological experiments.1 We begin with a brief review of some sample architectures, then
discuss the capabilities and functions that such systems should support. After this, we consider a
number of design decisions that relate to the properties of cognitive architectures, followed by some
dimensions along which one should evaluate them. In closing, we note some open issues in the area
and propose some directions that future research should take to address them.

2. Example Cognitive Architectures

Before turning to abstract issues that arise in research on cognitive architectures, we should consider
some concrete examples that have been reported in the literature. Here we review four distinct
frameworks that fall at different points within the architectural space. We have selected these
architectures because each has appeared with reasonable frequency in the literature, and also
because they exhibit different degrees of concern with explaining human behavior. We have ordered
them along this dimension, with more devoted psychological models coming earlier.
In each case, we discuss the manner in which the architecture represents, organizes, utilizes, and
acquires knowledge, along with its accomplishments. Because we review only a small sample of
extant architectures, we summarize a variety of other frameworks briefly in the Appendix. Nev-
ertheless, this set should give readers some intuitions about the space of cognitive architectures,
which later sections of the paper discuss more explicitly.
One common feature of the architectures we examine is that, although they have some theoretical
commitment to parallelism, especially in memory retrieval, they also rely on one or a few decision
modules. We have not included connectionist approaches in our treatment because, to our knowl-

1. Sun (2007) provides another treatment of cognitive architectures that discusses the second topic in greater detail.
 Cognitive Architectures connectionist approaches have Page 3
served as implementation of architecture
components

edge, they have not demonstrated the broad functionality associated with cognitive architectures
in the sense we discuss here. However, they have on occasion served as important components in
larger-scale architectures, as in Sun, Merrill, and Peterson’s (2001) CLARION framework.

2.1 ACT

ACT-R (Anderson & Lebiere, 1998, Anderson et al., 2004) is the latest in a family of cognitive
architectures, concerned primarily with modeling human behavior, that has seen continuous devel-
opment since the late 1970s. ACT-R 6 is organized into a set of modules, each of which processes
a different type of information. These include sensory modules for visual processing, motor mod-
ules for action, an intentional module for goals, and a declarative module for long-term declarative
knowledge. Each module has an associated buffer that holds a relational declarative structure
(often called ‘chunks’, but different from those in Soar). Taken together, these buffers comprise
productions are like operators (if-this-then-do-that)
ACT-R’s short-term memory.
A long-term production memory coordinates the processing of the modules. The conditions
of each production test chunks in the short-term buffers, whereas its actions alter the buffers
upon application. Some changes modify existing structures, whereas others initiate actions in the
associated modules, such as executing a motor command or retrieving a chunk from long-term
declarative memory. Each declarative chunk has an associated base activation that reflects its past
usage and influences its retrieval from long-term memory, whereas each production has an expected
cost (in terms of time needed to achieve goals) and probability of success.
On every cycle, ACT determines which productions match against the contents of short-term
expected
memory. This retrieval process is influenced by the base activation for each chunk it matches. ACT utility
computes the utility for each matched production as the difference between its expected benefit
(the desirability of its goal times its probability of success) and its expected cost. The system
selects the production with the highest utility (after adding noise to this score) and executes its
frequency- actions. The new situation leads new productions to match and fire, so that the cycle continues.
motivated
macro
Learning occurs in ACT-R at both the structural and statistical levels. For instance, the base
learning activation for declarative chunks increases with use by productions but decays otherwise, whereas
the cost and success probability for productions is updated based on their observed behavior. The
architecture can learn entirely new rules from sample solutions through a process of production
compilation that analyzes dependencies of multiple rule firings, replaces constants with variables,
and combines them into new conditions and actions (Taatgen, 2005).
much like macro
The ACT-R community has used its architecture to model a variety of phenomena from the learning

experimental psychology literature, including aspects of memory, attention, reasoning, problem

solving, and language processing. Most publications have reported accurate fits to quantiative data
about human reaction times and error rates. More recently, Anderson (2007) has related ACT-R
modules to different areas of the brain and developed models that match results from brain-imaging
studies. One the more applied front, the framework has played a central role in tutoring systems
that have seen wide use in schools (Koedinger et al., 1997), and it has also been used to control
mobile robots that interact with humans (Trafton et al., 2005).
Page 4 Cognitive Architectures

if-then rules
much like ops
2.2 Soar

Soar (Laird, 2008; Laird, Newell, & Rosenbloom, 1987; Newell, 1990) is a cognitive architecture
that has been under continuous development since the early 1980s. Procedural long-term knowledge
in Soar takes the form of production rules, which are in turn organized in terms of operators
associated with problem spaces. Some operators describe simple, primitive actions that modify the
agent’s internal state or generate primitive external actions, whereas others describe more abstract
activities. For many years, Soar represented all long-term knowledge in this form, but recently
separate episodic and semantic memories have been added. The episodic memory (Nuxoll & Laird,
2007) holds a history of previous states, while semantic memory contains previously known facts.
All tasks in Soar are formulated as attempts to achieve goals. Operators perform the basic
search valley, misleading region of search
deliberative acts of the system, with knowledge used to dynamically determine their selection and
application. The basic processing cycle repeatedly proposes, selects, and applies operators of the
current problem space to a problem state, moving ahead one decision at a time. However, when
knowledge about operator selection is insufficient to determine the next operator to apply or when
an abstract operator cannot be implemented, an impasse occurs; in response, Soar creates a new
goal to determine which operator it should select or how it should implement the abstract operator.
This process can lead to the dynamic generation of a goal hierarchy, in that problems are de-
composed into subproblems as necessary. The ‘state’ of a specific goal includes all features of its
supergoals, plus any additional cognitive structures necessary to select and apply operators in the
subgoal. Processing in a subgoal involves the same basic processing cycle of selecting and applying
operators. Subgoals can also deliberately access episodic or semantic memory to retrieve knowledge
relevant to resolving the impasse. The top state includes all sensor data obtained from the external
environment, so this information is also available to all subgoals. On any cycle, the states at various
levels of the goal hierarchy can change, typically due to changes in sensor values or as the result
of operator applications in subgoals. When the system resolves the impasse that generated a goal,
that goal disappears, along with all the subgoals below it.
Soar has multiple learning mechanisms for different types of knowledge: chunking and reinforce-
ment learning acquire procedural knowledge, whereas episodic and semantic learning acquire their
related to macro
corresponding types of declarative knowledge. Chunking occurs when one or more result is pro- discussion
duced in a subgoal (Laird, Rosenbloom, & Newell, 1986). When this happens, Soar learns a new
chunk , represented as a production rule which summarizes the processing that generated the re-
sults. A chunk’s actions are based on the results, whereas its conditions are based on those aspects
of the goals above the subgoal that were relevant to determining the results. Once the agent has
learned a chunk, it fires in new situations that are similar along relevant dimensions, often giving
the required results directly and thus avoiding the impasse that led to its formation. Reinforcement
learning adjusts numeric values associated with rules that help select operators (Nason & Laird,
2004). Episodic learning records the contents of working memory in snapshots, while semantic
learning stores individual elements of working memory for later retrieval. lots of applications

Researchers have used Soar to develop a variety of sophisticated agents that have demonstrated
impressive functionality. The most visible has been TAC-Air-Soar (Tambe et al., 1995), which
modeled fighter pilots in military training exercises that involved air combat scenarios. More
recently, Soar has supported a number of intelligent agents that control synthetic characters in
Cognitive Architectures Page 5

interactive computer games (Margerko et al., 2004). Another thrust has involved using Soar to
model the details of human language processing (Lewis, 1993), categorization (Miller & Laird, 1996),
and other facets of cognition, but the emphasis has been on demonstrating high-level functionality
rather than on fits to quantitative measurements.

2.3 ICARUS
Icarus is a more recent architecture (Langley, Cummings, & Shapiro, 2004) that stores two distinct
forms of knowledge. Concepts describe classes of environmental situations in terms of other con-
cepts and percepts, whereas skills specify how to achieve goals by decomposing them into ordered
subgoals. Both concepts and skills involve relations among objects, and both impose a hierarchical
organization on long-term memory, with the former grounded in perceptions and the latter in ex-
ecutable actions. Moreover, skills refer to concepts in their heads, their initiation conditions, and
their continuation conditions.
The basic Icarus interpreter operates on a recognize-act cycle. On each step, the architecture
deposits descriptions of visible objects into a perceptual bufffer. The system compares primitive
concepts to these percepts and adds matched instances to short-memory as beliefs. These in
turn trigger matches of higher-level concepts, with the process continuing until Icarus infers all
deductively implied beliefs. Next, starting from a top-level goal, it finds a path downward through
the skill hierarchy in which each subskill has satisfied conditions but an unsatisfied goal. When a
path terminates in a primitive skill with executable actions, the architecture applies these actions
to affect the environment. This leads to new percepts, changes in beliefs, ande.g., STRIPS planning
reactive execution of
additional skill paths to achieve the agent’s goals.
However, when Icarus can find no applicable path through the skill hierarchy that is relevant to
a top-level goal, it resorts to problem solving using a variant of means-ends analysis. This module
interesting
chains backward off either a skill that would achieve the current goal or off the goal concept’s statement.
macros?
definition, and it interleaves problem solving with execution in that it carries out selected skills reactive?
when they become applicable. Whenever problem solving achieves a goal, Icarus creates a new without
deliberation?
skill with that goal as its head and with one or more ordered subgoals that are based on the problem
solution. If the system encounters similar problems in the future, it executes the learned skills to
handle them reactively, without need for deliberative problem solving (Langley & Choi, 2006b).
Researchers have used Icarus to develop agents for a number of domains that involve a combi-
nation of inference, execution, problem solving, and learning. These have included tasks like the
Tower of Hanoi, multi-column subtraction, FreeCell solitaire, and logistics planning. They have also
used the architecture to control synthetic characters in simulated virtual environments, including
ones that involve urban driving (Langley & Choi, 2006a) and first-person shooter scenarios (Choi
et al., 2007). Ongoing work aims to link Icarus to physical robots that carry out joint activities
with humans.

2.4 PRODIGY
Prodigy (Carbonell, Knoblock, & Minton, 1990) is another cognitive architecture that saw ex-
tensive development from the middle 1980s to the late 1990s. This framework incorporates two
main kinds of knowledge. Domain rules encode the conditions under which actions have certain
 Page 6 Cognitive Architectures

effects, where the latter are described as the addition or deletion of first-order expressions. These
refer both to physical actions that affect the environment and to inference rules, which are purely
cognitive. In contrast, control rules specify the conditions under which the architecture should
select, reject, or prefer a given operator, set of operator bindings, problem state, or goal during the
search process.
As in Icarus, Prodigy’s basic problem-solving module involves search through a problem space
to achieve one or more goals, which it also casts as first-order expressions. This search relies on
means-ends analysis, which selects an operator that reduces some difference between the current
state and the goal, which in turn can lead to subproblems with their own current states and
goals. On each cycle, Prodigy uses its control rules to select an operator, binding set, state, or
goal, to reject them out of hand, or to prefer some over others. In the absence of such control
knowledge, the architecture makes choices at random and carries out depth-first means-ends search
with backtracking.
decreasing the
effective breadth of Prodigy’s explanation-based learning module constructs control rules based on its problem-
search solving experience (Minton, 1990). Successful achievement ofa goal after search leads to creation
of selection or preference rules related to that goal and to the operators whose application achieved
it. Failure to achieve a goal leads to creation of rejection or preference rules for operators, goals,
and states that did not produce a solution. To generate these control rules, Prodigy invokes a
learning method that analyzes problem-solving traces and proves the reasons for success or failure.
The architecture also collects statistics on learned rules and retains only those whose inclusion,
over time, leads to more efficient problem solving.
In addition, Prodigy includes separate modules for controlling search by analogy with earlier
solutions (Veloso & Carbonell, 1993), learning operator descriptions from observed solutions or
experimentation (Wang, 1995), and improving the quality of solutions (Pérez & Carbonell, 1994).
Although most research in this framework has dealt exclusively with planning and problem solving,
Prodigy also formed the basis for an impressive system that interleaved planning and execution
for a mobile robot that accepted asychronous requests from users (Haigh & Veloso, 1996).

3. Capabilities of Cognitive Architectures

Any intelligent system is designed to engage in certain activities that, taken together, constitute
its functional capabilities. In this section, we discuss the varied capabilities that a cognitive archi-
tecture can support. Although only a few abilities, such as recognition and decision making, are
strictly required to support a well-defined architecture, the entire set seems required to cover the
full range of human-level intelligent activities.
A central issue that confronts the designer of a cognitive architecture is how to let agents access
different sources of knowledge. Many of the capabilities we discuss below give the agent access
to such knowledge. For example, knowledge about the environment comes through perception,
knowledge about implications of the current situation comes through planning, reasoning, and
prediction, knowledge from other agents comes via communication, and knowledge from the past
comes through remembering and learning. The more such capabilities an architecture supports,
the more sources of knowledge it can access to inform its behavior.
Cognitive Architectures Page 7

Another key question is whether the cognitive architecture supports a capability directly, using
embedded processes, or whether it instead provides ways to implement that capability in terms
of knowledge. Design decisions of this sort influence what the agent can learn from experience,
what the designers can optimize at the outset, and what functionalities can rely on specialized
representations and mechanisms. In this section, we attempt to describe functionality without
referring to the underlying mechanisms that implement them, but this is an important issue that
deserves more attention in the future.

3.1 Recognition and Categorization

An intelligent agent must make some contact between its environment and its knowledge. This
requires the ability to recognize situations or events as instances of known or familiar patterns. For
example, a reader must recognize letters and the words they make up, a chess player must identify
meaningful board configurations, and an image analyst must detect buildings and vehicles in aerial
photographs. However, recognition need not be limited to static situations. A fencing master can
identify different types of attacks and a football coach can recognize the execution of particular
plays by the opposing team, both of which involve dynamic events.
Recognition is closely related to categorization, which involves the assignment of objects, situ-
ations, and events to known concepts or categories. However, research on cognitive architectures
typically assumes recognition is a primitive process that occurs on a single cycle and that underlies
many higher-level functions, whereas categorization is sometimes viewed as a higher-level function.
Recognition and categorization are closely linked to perception, in that they often operate on out-
put from the perceptual system, and some frameworks view them as indistinguishable. However,
they can both operate on abstract mental structures, including those generated internally, so we
will treat them as distinct.
To support recognition and categorization, a cognitive architecture must provide some way to
represent patterns and situations in memory. Because these patterns must apply to similar but
distinct situations, they must encode general relations that hold across these situations. An archi-
tecture must also include some recognition process that lets it identify when a particular situation
matches a stored pattern or category and, possibly, measure the degree to which it matches. In
production system architectures, this mechanism determines when the conditions of each produc-
tion rule match and the particular ways they are instantiated. Finally, a complete architecture
should include some means to learn new patterns or categories from instruction or experience, and
to refine existing patterns when appropriate.

3.2 Decision Making and Choice

To operate in an environment, an intelligent system also requires the ability to make decisions and
select among alternatives. For instance, a student must decide which operation will simplify an
integration problem, a speaker must select what word to use next in an utterance, and a baseball
player must decide whether or not to swing at a pitch. Such decisions are often associated with the
recognition of a situation or pattern, and most cognitive architectures combine the two mechanisms
in a recognize-act cycle that underlies all cognitive behavior.
 how related to
Page 8 planning? problem
Cognitive Architectures
solving?

Such one-step decision making has much in common with higher-level choice, but differs in its
complexity. For example, consider a consumer deciding which brand of detergent to buy, a driver
choosing which route to drive, and a general selecting which target to bomb. Each of these decisions
can be quite complex, depending on how much time and energy the person is willing to devote.
Thus, we should distinguish between decisions that are made at the architectural level and more
complex ones that the architecture enables. preconditions

To support decision making, a cognitive architecture must provide some way to represent alter-
native choices or actions, whether these are internal cognitive operations or external ones. It must
also offer some process for selecting among these alternatives, which most architectures separate
into two steps. The first determines whether a given choice or action is allowable, typically by
associating it with some pattern and considering it only if the pattern is matched. For instance, we
can specify the conditions under which a chess move is legal, then consider that move only when
the conditions are met. The second step selects among allowable alternatives, often by computing
preferences
some numeric score and choosing one or more with better scores. Such conflict resolution takes
quite different forms in different architectures.
Finally, an ideal cognitive architecture should incorporate some way to improve its decisions
through learning. Although this can, in principle, involve learning new alternatives, most mecha-
nisms focus on learning or revising either the conditions under which an existing action is considered
allowable or altering the numeric functions used during the conflict resolution stage. The resulting
improvements in decision making will then be reflected in the agent’s overall behavior.

3.3 Perception and Situation Assessment

Cognition does not occur in isolation; an intelligent agent exists in the context of some external
environment that it must sense, perceive, and interpret. An agent may sense the world through
different modalities, just as a human has access to sight, hearing, and touch. The sensors may
range from simple devices like a thermometer, which generates a single continuous value, to more
complex mechanisms like stereoscopic vision or sonar that generate a depth map for the local
environment within the agent’s field of view. Perception can also involve the integration of results
from different modalities into a single assessment or description of the environmental situation,
which an architecture can represent for utilization by other cognitive processes.
Perception is a broad term that covers many types of processing, from inexpensive ones that
an architecture can support automatically to ones that require limited resources and so must be
invoked through conscious intentions. For example, the human visual system can detect motion in
the periphery without special effort, but the fovea can extract details only from the small region at
which it is pointed. A cognitive architecture that includes the second form of sensor must confront
the issue of attention, that is, deciding how to allocate and direct its limited perceptual resources
to detect relevant information in a complex environment.
An architecture that supports perception should also deal with the issue that sensors are often
noisy and provide at most an inaccurate and partial picture of the agent’s surroundings. Dynamic
environments further complicate matters in that the agent must track changes that sometimes occur
at a rapid rate. These challenges can be offset with perceptual knowledge about what sensors to
Cognitive Architectures Page 9
how related to
3.1 on recognition and
categorization

invoke, where and when to focus them, and what inferences are plausible. An architecture can also
acquire and improve this knowledge by learning from previous perceptual experiences.
An intelligent agent should also be able to move beyond perception of isolated objects and events
to understand and interpret the broader environmental situation. For example, a fire control
officer on a ship must understand the location, severity, and trajectory of fires in order to respond
effectively, whereas a general must be aware of an enemy’s encampments, numbers, and resources
to defend against them successfully. Thus, situation assessment requires an intelligent agent to
combine perceptual information about many entities and events, possibly obtained from many
sources, to compose a large-scale model of the current environment. As such, it relies both on the
recognition and categorization of familiar patterns in the environment, which we discussed earlier,
and on inferential mechanisms, which we will consider shortly.

3.4 Prediction and Monitoring

Cognitive architectures exist over time, which means they can benefit from an ability to predict
future situations and events accurately. For example, a good driver knows approximately when his
car will run out of gas, a successful student can predict how much he must studyA basicto ace a final, and
prediction capability
a skilled pilot can judge how close he can fly to the ground without crashing.
used in larger Perfect
tasks like prediction
may not be possible in many situations, but perfection is seldom necessary planning
to make predictions
that are useful to an intelligent system.
Prediction requires some model of the environment and the effect actions have on it, and the
architecture must represent this model in memory. One general approach involves storing some
mapping from a description of the current situation and an action onto a description of the re-
sulting situation. Another approach encodes the effects of actions or events in terms of changes
to the environment. In either case, the architecture also requires some mechanism that uses these
knowledge structures to predict future situations, say by recognizing a class of situations in which
a part 2 game
manager can enable
an action will have certain effects. An ideal architecture should also include the ability to learn
monitoring
predictive models from experience and to refine them over time.
Once an architecture has a mechanism for making predictions, it can also utilize them to monitor
the environment. For example, a pilot may suspect that his tank has a leak if the fuel gauge goes
down more rapidly than usual, and a commander may suspect enemy action if a reconnaissance
team fails to report on time. Because monitoring relates sensing to prediction, it raises issues of
attentional focus when an architecture has limited perceptual resources. Monitoring also provides
natural support for learning, since errors can help an agent improve its model of the environment.

3.5 Problem Solving and Planning

Because intelligent systems must achieve their goals in novel situations, the cognitive architectures
that support them must be able to generate plans and solve problems. For example, an unmanned
air vehicle benefits from having a sensible flight plan, a project manager desires a schedule that
allocates tasks to specific people at specific times, and a general seldom moves into enemy territory
without at least an abstract course of action. When executed, plans often go awry, but that does
not make them any less useful to an intelligent agent’s thinking about the future.
Page 10 Cognitive Architectures

Planning is only possible when the agent has an environmental model that predicts the effects of like
its actions. To support planning, a cognitive architecture must be able to represent a plan as an (at STRIPS
operators
least partially) ordered set of actions, their expected effects, and the manner in which these effects
enable later actions. The plan need not be complete to guide behavior, in that it may extend only
a short time into the future or refer to abstract actions that can be expanded in different ways.
The structure may also include conditional actions and branches that depend on the outcome of
earlier events as noted by the agent.
An intelligent agent should also be able to construct a plan from components available in memory.
These components may refer to low-level motor and sensory actions but, often, they will be more
abstract structures, including prestored subplans. There exist many mechanisms for generating
plans from components, as well as ones for adapting plans that have been retrieved from memory.
What these methods have in common is that they involve problem solving or search. That is, they
carry out steps through a space of problem states, on each step considering applicable operators,
selecting one or more operator, and applying it to produce a new problem state. This search process
continues until the system has found an acceptable plan or decides to give up.
The notion of problem solving is somewhat more general than planning, though they are typically
real-time search
viewed as closely related. In particular, planning usually refers to cognitive activities within the
agent’s head, whereas problem solving can also occur in the world. Especially when a situation is
complex and the architecture has memory limitations, an agent may carry out search by applying
operators or actions in the environment, rather than trying to construct a plan internally. Problem
solving can also rely on a mixture of internal planning and external behavior, but it generally
involves the multi-step construction of a problem solution. Like planning, problem solving is often
characterized in terms of search through a problem space that applies operators to generate new
states, selects promising candidates, and continues until reaching a recognized goal.
Planning and problem solving can also benefit from learning. Naturally, improved predictive
models for actions can lead to more effective plans, but learning can also occur at the level of
problem space search, whether this activity takes place in the agent’s head or in the physical world.
Such learning can rely on a variety of information sources. In addition to learning from direct
instruction, an architecture can learn from the results of problem-space search (Sleeman et al.,
1982), by observing another agent’s behavior or behavioral cloning (Sammut, 1996), and from
delayed rewards via reinforcement learning (Sutton & Barto, 1998). Learning can aim to improve
problem solving behavior in two ways (Langley, 1995a). One focuses on reducing the branching
factor of search, either through adding heuristic conditions to problem space operators or refining a
numeric evaluation function to guide choice. Another focuses on forming macro-operators or stored
plans that reduce the effective depth of search by taking larger steps in the problem space.
Intelligent agents that operate in and monitor dynamic environments must often modify existing
plans in response to unanticipated changes. This can occur in several contexts. For instance, an
agent should update its plan when it detects a changed situation that makes some planned activities
inapplicable, and thus requires other actions. Another context occurs when a new situation suggests
some more desirable way of accomplishing the agent’s goal; such opportunistic planning can take
advantage of these unexpected changes. Monitoring a plan’s execution can also lead to revised
estimates about the plan’s effectiveness, and, ultimately, to a decision to pursue some other course

plan adaptation could also occur

in a game context where AI players adapt each others’
plans
Cognitive Architectures Page 11

of action with greater potential. Replanning can draw on the same mechanisms as generating a
plan from scratch, but requires additional operators for removing actions or replacing them with
other steps. Similar methods can also adapt to the current situation a known plan the agent has
retrieved from memory.
Piazza question. There
will be more
3.6 Reasoning and Belief Maintenance

Problem solving is closely related to reasoning, another central cognitive activity that lets an agent
augment its knowledge state. Whereas planning is concerned primarily with achieving objectives in
the world by taking actions, reasoning draws mental conclusions from other beliefs or assumptions
that the agent already holds. For example, a pilot might conclude that, if another plane changes
its course to intersect his own, it is probably an enemy fighter. Similarly, a geometry student might
deduce that two triangles are congruent because they share certain sides and vertices, and a general
might infer that, since he has received no recent reports of enemy movement, a nearby opposing
force is still camped where it was the day before.
To support such reasoning, a cognitive architecture must first be able to represent relationships
among beliefs. A common formalism for encoding such relationships is first-order logic, but many
other notations have also been used, ranging from production rules to neural networks to Bayesian
networks. The relations represented in this manner may be logically or probabilistically sound,
what
but this is not required; knowledge about reasoning can also be heuristic or approximate
kinds ofand still
reasoning
prove quite useful to an intelligent agent. Equally important, the formalism may methods
be more did you learn in
or less
CS x260?
expressive (e.g., limited to propositional logic) or computationally efficient.
Naturally, a cognitive architecture also requires mechanisms that draw inferences using these
knowledge structures. Deductive reasoning is an important and widely studied form of inference
that lets one combine general and specific beliefs to conclude others that they entail logically.
However, an agent can also engage in inductive reasoning, which moves from specific beliefs to
more general ones and which can be viewed as a form of learning. An architecture may also
support abductive inference, which combines general knowledge and specific beliefs to hypothesize
other specific beliefs, as occurs in medical diagnosis. In constrained situations, an agent can simply
draw all conclusions that follow from its knowledge base, but more often it must select which
inferential knowledge to apply. This raises issues of search closely akin to those in planning tasks,
along with issues of learning to make that search more effective.
Reasoning plays an important role not only when inferring new beliefs but when deciding whether
to maintain existing ones. To the extent that certain beliefs depend on others, an agent should track
the latter to determine whether it should continue to believe the former, abandon it, or otherwise
alter its confidence. Such belief maintenance is especially important for dynamic environments in
which situations may change in unexpected ways, with implications for the agent’s behavior. One
general response to this issue involves maintaining dependency structures in memory that connect
beliefs, which the architecture can use to propagate changes as they occur.
Page 12 Cognitive Architectures

3.7 Execution and Action

Cognition occurs to support and drive activity in the environment. To this end, a cognitive archi-
tecture must be able to represent and store motor skills that enable such activity. For example,
a mobile ground robot or unmanned air vehicle should have skills or policies for navigating from
one place to another, for manipulating its surroundings with effectors, and for coordinating its
behavior with other agents on its team. These may be encoded solely in terms of primitive or
component actions, but they may also specify more complex multi-step skills or procedures. The
latter may take the form of plans that the agent has generated or retrieved from memory, especially
in architectures that have grown out of work on problem solving and planning. However, other
formulations of motor skill execution, such as closed-loop controllers, have also been explored.
A cognitive architecture must also be able to execute skills and actions in the environment. In
some frameworks, this happens in a completely reactive manner, with the agent selecting one or
more primitive actions on each decision cycle, executing them, and repeating the process on the
next cycle. This approach is associated with closed-loop strategies for execution, since the agent
can also sense the environment on each time step. The utilization of more complex skills supports
open-loop execution, in which the agent calls upon a stored procedure across many cycles without
checking the environment. However, a flexible architecture should support the entire continuum
from fully reactive, closed-loop behavior to automatized, open-loop behavior, as can humans.
Ideally, a cognitive architecture should also be able learn about skills and execution policies from
instruction and experience. Such learning can take different forms, many of which parallel those
that arise in planning and problem solving. For example, an agent can learn by observing another
agent’s behavior, by successfully achieving its goals, and from delayed rewards. Similarly, it can
learn or refine its knowledge for selecting primitive actions, either in terms of heuristic conditions
on their application or as a numeric evaluation function that reflects their utility. Alternatively, an
agent can acquire or revise more complex skills in terms of known skills or actions.

Very relevant in game

3.8 Interaction and Communication playing

Sometimes the most effective way for an agent to obtain knowledge is from another agent, making
communication another important ability that an architecture should support. For example, a
commander may give orders to, and receive reports from, her subordinates, while a shopper in a
flea market may dicker about an item’s price with its owner. Similarly, a traveler may ask and
receive directions on a street corner, while an attorney may query a defendant about where he
was on a particular night. Agents exist in environments with other agents, and there are many
occasions in which they must transfer knowledge from one to another.
Whatever the modality through which this occurs, a communicating agent must represent the
knowledge that it aims to convey or that it believes another agent intends for it. The content so
transferred can involve any of the cognitive activities we have discussed so far. Thus, two agents
can communicate about categories recognized and decisions made, about perceptions and actions,
about predictions and anomalies, and about plans and inferences. One natural approach is to
draw on the representations that result from these activities as the input to, and the output from,
interagent communication.
Cognitive Architectures Page 13

A cognitive architecture should also support mechanisms for transforming knowledge into the
form and medium through which it will be communicated. The most common form is spoken
or written language, which follows established conventions for semantics, syntax, and pragmatics
onto which an agent must map the content it wants to convey. Even when entities communicate
with purely artificial languages, they do not have exactly the same mental structures and they must
translate content into some external format. One can view language generation as a form of planning
and execution, whereas language understanding involves inference and reasoning. However, the
specialized nature of language processing makes these views misleading, since the task raises many
additional issues.
An important form of communication occurs in conversational dialogues, which require both
generation and understanding of natural language, as well as coordination with the other agent
in the form of turn taking. Learning is also an important issue in language and other forms of
communication, since an architecture should be able to acquire syntactic and semantic knowledge
for use at both the sentence and dialogue levels. Moreover, some communicative tasks, like question
answering, require access to memory for past events and cognitive activities, which in turn benefits
from episodic storage.

3.9 Remembering, Reflection, and Learning

A cognitive architecture can also benefit from capabilities that cut across those described in the
previous sections, in that they operate on mental structures produced or utilized by them. Such
abilities, which Sloman (2001) refers to as metamanagement mechanisms, are not strictly required
for an intelligent agent, but their inclusion can extend considerably the flexibility and robustness
e.g., remember a plan
of an architecture.
One capacity of this sort involves remembering – the ability to encode and store the results
of cognitive processing in memory and to retrieve or access them later. An agent cannot directly
remember external situations or its own physical actions; it can only recall cognitive structures that
describe those events or inferences about them. This idea extends naturally to memories of problem
solving, reasoning, and communication. To remember any cognitive activity, the architecture must
store the cognitive structures generated during that activity, index them in memory, and retrieve
them when needed. The resulting content is often referred to as episodic memories.
Was the plan
Another capability that requires access to traces of cognitive activity is reflection. This may helpful?
involve processing of either recent mental structures that are still available or older structures that
the agent must retrieve from its episodic store. One type of reflective activity concerns the justifi-
cation or explanation of an agent’s inferences, plans, decisions, or actions in terms of cognitive steps
that led to them. Another revolves around meta-reasoning about other cognitive activities, which
an architecture can apply to the same areas as explanation, but which emphasizes their generation
(e.g., forming inferences or making plans) rather than their justification. To the extent that reflec-
tive processes lay down their own cognitive traces, they may themselves be subject to reflection.
However, an architecture can also support reflection through less transparent mechanisms, such as
statistical analyses, that are not themselves inspectable by the agent.
Page 14 Cognitive Architectures
extracting macro
subsequences; replacing
constants by variables

A final important ability that applies to many cognitive activities is learning. We have discussed
previously the various forms this can take, in the context of different architectural capacities, but we
should also consider broader issues. Learning usually involves generalization beyond specific beliefs
and events. Although most architectures carry out this generalization at storage time and enter
generalized knowledge structures in memory, some learning mechanisms store specific situations
and generalization occurs at retrieval time through analogical or case-based reasoning. Either
approach can lead to different degrees of generalization or transfer, ranging from very similar tasks,
to other tasks within the same domain, and even to tasks within related but distinct domains. Many
architectures treat learning as an automatic process that is not subject to inspection or conscious
control, but they can also use meta-reasoning to support learning in a more deliberate manner. The
data on which learning operates may come from many sources, including observation of another
agent, an agent’s own problem solving behavior, or practice of known skills. But whatever the source
of experience, all involve processing of memory structures to improve the agent’s capabilities.

4. Properties of Cognitive Architectures

We can also characterize cognitive architectures in terms of the internal properties that produce
the capabilities described in the previous section. These divide naturally into the architecture’s
representation of knowledge, the organization it places on that knowledge, the manner in which
the system utilizes its knowledge, and the mechanisms that support acquisition and revision of
knowledge through learning. Below we consider a number of design decisions that arise within
each of these facets of an intelligent system, casting them in terms of the data structures and
algorithms that are supported at the architectural level. Although we present most issues in terms
of oppositions, many of the alternatives we discuss are complementary and can exist within the
same framework.

4.1 Representation of Knowledge

One important class of architectural properties revolves around the representation of knowledge.
Recall that knowledge itself is not built into an architecture, in that it can change across domains
and over time. However, the representational formalism in which an agent encodes its knowledge
constitutes a central aspect of a cognitive architecture.
Perhaps the most basic representational choice involves whether an architecture commits to a
single, uniform notation for encoding its knowledge or whether it employs a mixture of formalisms.
Selecting a single formalism has advantages of simplicity and elegance, and it may support more
easily abilities like learning and reflection, since they must operate on only one type of structure.
However, as we discuss below, different representational options have advantages and disadvantages,
so that focusing on one framework can force an architecture into awkward approaches to certain
problems. On the other hand, even mixed architectures are typically limited to a few types of
knowledge structures to avoid complexity.
One common tradition distinguishes declarative from procedural representations. Declarative
encodings of knowledge can be manipulated by cognitive mechanisms independent of their content.
For instance, a notation for describing devices might support design, diagnosis, and control. First-
order logic (Genesereth & Nilsson, 1987) is a classic example of such a representation. Generally
Page 26 Cognitive Architectures

We will also need an experimental method that recognizes the fact that cognitive architectures
involve integration of many components which may have synergistic effects, rather than consisting
of independent but unrelated modules (Langley & Messina, 2004). Experimental comparisons
among architectures have an important role to play, but these must control carefully for the task
being handled and the amount of knowledge encoded, and they must measure dependent variables
in unbiased and informative ways. Systematic experiments that are designed to identify sources of
power will tell us far more about the nature of cognitive architectures than simplistic competitions.
Our field still has far to travel before we understand fully the space of cognitive architectures
and the principles that underlie their successful design and utilization. However, we now have over
two decades’ experience with constructing and using a variety such architectures for a wide range
of problems, along with a number of challenges that have arisen in this pursuit. If the scenery
revealed by these initial steps are any indication, the journey ahead promises even more interesting
and intriguing sites and attractions.

Acknowledgments

Production of this paper was supported, in part, by Grant HR0011-04-1-0008 from DARPA IPTO
and by Grant IIS-0335353 from NSF. The document borrows greatly, in both organization and
content, from the University of Michigan Web site at http://ai.eecs.umich.edu/cogarch0/,
which was authored by R. Wray, R. Chong, J. Phillips, S. Rogers, B. Walsh, and J. E. Laird. We
thank Ron Brachman for convincing us that the paper was needed and encouraging us to write it.

References

Aamodt, A., & Plaza, E. (1994). Case-based reasoning: Foundational issues, methodological vari-
ations, and system approaches. AI Communications, 7 , 39–59.
Aha, D. W. (1997). Lazy learning. Dordrecht, Germany: Kluwer.
Albus, J. S., Pape, C. L., Robinson, I. N., Chiueh, T.-C., McAulay, A. D., Pao, Y.-H., & Takefuji,
Y. (1992). RCS: A reference model for intelligent control. IEEE Computer , 25 , 56–79.
Ali, S. S. & Shapiro, S. C. (1993). Natural language processing using a propositional semantic
network with structured variables. Minds and Machines, 3 , 421–451.
Anderson, J. R. (1991). Cognitive architectures in a rational analysis. In K. VanLehn (Ed.),
Architectures for intelligence. Hillsdale, NJ: Lawrence Erlbaum.
Anderson, J. R. (2007). How can the human mind exist in the physical universe? . New York:
Oxford University Press.
Anderson, J. R., & Lebiere, C. (1998). The atomic components of thought. Mahwah, NJ: Lawrence
Erlbaum.
Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y. (2004). An
integrated theory of the mind. Psychological Review, 111, (4). 1036–1060.
Bonasso, R. P., Firby, R. J., Gat, E., Kortenkamp, D., Miller, D., & Slack, M. (1997). Experiences
with an architecture for intelligent, reactive agents. Journal of Experimental and Theoretical
Artificial Intelligence, 9 , 237–256.
Cognitive Architectures Page 27

Bratman, M. E. (1987). Intentions, plans, and practical reason. Cambridge, MA: Harvard Univer-
sity Press.
Brooks, R. A. (1986). A robust layered control system for a mobile robot. IEEE Journal of Robotics
and Automation, RA-2 , 14–23.
Carbonell, J. G., Knoblock, C. A., & Minton, S. (1990). Prodigy: An integrated architecture
for planning and learning. In K. Van Lehn (Ed.), Architectures for intelligence. Hillsdale, NJ:
Lawrence Erlbaum.
Choi, D., Konik, T., Nejati, N., Park, C., & Langley, P. (2007). A believable agent for first-person
shooter games. Proceedings of the Third Annual Artificial Intelligence and Interactive Digital
Entertainment Conference (pp. 71–73). Stanford, CA: AAAI Press.
Clocksin, W. F., & Mellish, C. S. (1981). Programming in Prolog. Berlin: Springer-Verlag.
Drummond, M., Bresina, J., & Kedar, S. (1991). The Entropy Reduction Engine: Integrating
planning, scheduling, and control. SIGART Bulletin, 2 , 61–65.
Ernst, G., & Newell, A. (1969). GPS: A case study in generality and problem solving. New York:
Academic Press.
Fikes, R., Hart, P. E., & Nilsson, N. J. (1972). Learning and executing generalized robot plans.
Artificial Intelligence, 3 , 251–288.
Firby, R. J. (1994). Task networks for controlling continuous processes. Proceedings of the Second
International Conference on AI Planning Systems (pp. 49–54). Chicago: AAAI Press.
Freed, M. (1998). Managing multiple tasks in complex, dynamic environments. Proceedings of
the Fifteenth National Conference on Artificial Intelligence (pp. 921–927). Madison, WI: AAAI
Press.
Gat, E. (1991). Integrating planning and reacting in a heterogeneous asynchronous architecture
for mobile robots. SIGART Bulletin, 2 , 17–74.
Genesereth, M. R., & Nilsson, N. J. (1987). Logical foundations of artificial intelligence. Los Altos,
CA: Morgan Kaufmann.
Gratch, J. (2000). Emile: Marshalling passions in training and education. Proceedings of the Fourth
International Conference on Autonomous Agents (pp. 325–332). Barcelona, Spain.
Haigh, K., & Veloso, M. (1996). Interleaving planning and robot execution for asynchronous user
requests. Proceedings of the International Conference on Intelligent Robots and Systems (pp.
148–155). Osaka, Japan: IEEE Press.
Hayes-Roth, B., Pfleger, K., Lalanda, P., Morignot, P., & Balabanovic, M. (1995). A domain-
specific software architecture for adaptive intelligent systems. IEEE Transactions on Software
Engineering, 21 , 288–301.
Hendler, J., Tate, A., & Drummond, M. (1990). AI planning: Systems and techniques. AI Maga-
zine, 11 , 61–77.
Ingrand, F. F., Georgeff, M. P., & Rao, A. S. (1992). An architecture for real-time reasoning and
system control. IEEE Expert, 7 , 34–44.
Page 28 Cognitive Architectures

Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. (1997). Intelligent tutoring goes to
school in the big city. International Journal of Artificial Intelligence in Education, 8 , 30–43.
Konolige, K., Myers, K. L., Ruspini, E. H., & Saffiotti, A. (1997). The Safira architecture: A design
for autonomy. Journal of Experimental & Theoretical Artificial Intelligence, 9 , 215–235.
Laird, J. E. (1991). Preface for special section on integrated cognitive architectures. SIGART
Bulletin, 2 , 12–123.
Laird, J. E. (2008). Extending the Soar cognitive architecture. Proceedings of the Artificial General
Intelligence Conference. Memphis, TN: IOS Press.
Laird, J. E., Rosenbloom, P. S., & Newell, A. (1986). Chunking in Soar: The anatomy of a general
learning mechanism. Machine Learning, 1 , 11–46.
Laird, J. E., Newell, A., & Rosenbloom, P. S. (1987). Soar: An architecture for general intelligence.
Artificial Intelligence, 33 , 1–64.
Langley, P. (1995a). Elements of machine learning. San Francisco: Morgan Kaufmann.
Langley, P. (1995b). Order effects in incremental learning. In P. Reimann & H. Spada (Eds.), Learn-
ing in humans and machines: Towards and interdisciplinary learning science. Oxford: Elsevier.
Langley, P. (2006). Intelligent behavior in humans and machines (Technical Report). Computa-
tional Learning Laboratory, CSLI, Stanford University, CA.
Langley, P., & Choi, D. (2006a). A unified cognitive architecture for physical agents. Proceedings
of the Twenty-First AAAI Conference on Artificial Intelligence. Boston: AAAI Press.
Langley, P., & Choi, D. (2006b). Learning recursive control programs from problem solving. Journal
of Machine Learning Research, 7 , 493–518.
Langley, P., Cummings, K., & Shapiro, D. (2004). Hierarchical skills and cognitive architectures.
Proceedings of the Twenty-Sixth Annual Conference of the Cognitive Science Society (pp. 779–
784). Chicago, IL.
Langley, P., & Messina, E. (2004). Experimental studies of integrated cognitive systems. Proceed-
ings of the Performance Metrics for Intelligent Systems Workshop. Gaithersburg, MD.
Lewis, R. L. (1993). An architecturally-based theory of sentence comprehension. Proceedings of
the Fifteenth Annual Conference of the Cognitive Science Society (pp. 108–113). Boulder, CO:
Lawrence Erlbaum.
Magerko, B., Laird, J. E., Assanie, M., Kerfoot, A., & Stokes, D. (2004). AI characters and
directors for interactive computer games. Proceedings of the Sixteenth Innovative Applications of
Artificial Intelligence Conference (pp. 877-884). San Jose, CA: AAAI Press.
McCarthy, J. (1963). Situations, actions and causal laws (Memo 2). Artificial Intelligence Project,
Stanford University, Stanford, CA.
Meyer, M., & Kieras, D. (1997). A computational theory of executive control processes and human
multiple-task performance: Part 1. Basic mechanisms. Psychological Review , 104 , 3–65.
Miller, C. S., & Laird, J. E. (1996). Accounting for graded performance within a discrete search
framework. Cognitive Science, 20 , 499–537.
Cognitive Architectures Page 29

Minsky, M. (1975). A framework for representing knowledge. In P. Winston (Ed.), The psychology
of computer vision. New York: McGraw-Hill.
Minton, S. N. (1990). Quantitative results concerning the utility of explanation-based learning.
Artificial Intelligence, 42 , 363–391.
Muscettola, N., Nayak, P. P., Pell, B., & Williams, B. (1998). Remote Agent: To boldly go where
no AI system has gone before. Artificial Intelligence, 103 , 5–48.
Musliner, D. J., Goldman, R. P., & Pelican, M. J. (2001). Planning with increasingly complex
models. Proceedings of the International Conference on Intelligent Robots and Systems.
Nardi, D., & Brachman, R. J. (2002). An introduction to description logics. In F. Baader et al.
(Eds.), Description logic handbook . Cambridge: Cambridge University Press.
Nason, S., & Laird, J. E. (2004). Soar-RL: Integrating reinforcement learning with Soar. Proceed-
ings of the Sixth International Conference on Cognitive Modeling (pp. 220–225).
Neches, R., Langley, P., & Klahr, D. (1987). Learning, development, and production systems. In
D. Klahr, P. Langley, & R. Neches (Eds.), Production system models of learning and development.
Cambridge, MA: MIT Press.
Newell, A. (1973a). You can’t play 20 questions with nature and win: Projective comments on
the papers of this symposium. In W. G. Chase (Ed.), Visual information processing New York:
Academic Press.
Newell, A. (1973b). Production systems: Models of control structures. In W. G. Chase (Ed.),
Visual information processing. New York: Academic Press.
Newell, A. (1982). The knowledge level. Artificial Intelligence, 18 , 87–127.
Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press.
Nuxoll, A. M. & Laird, J. E. (2007). Extending cognitive architecture with episodic memory.
Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence. Vancouver, BC:
AAAI Press.
Pell, B., Bernard, D. E., Chien, S. A., Gat, E., Muscettola, N., Nayak, P. P., Wagner, M. D., &
Williams, B. C. (1997). An autonomous spacecraft agent prototype. Proceedings of the First
International Conference on Autonomous Agents (pp. 253–261). Marina del Rey, CA: ACM
Press.
Pérez, M. A. & Carbonell, J. G. (1994). Control knowledge to improve plan quality. Proceedings
of the Second International Conference on AI Planning Systems (pp. 323–328). Chicago: AAAI
Press.
Remington, R., Matessa, M., Freed, M., & Lee, S. (2003). Using Apex / CPM-GOMS to de-
velop human-like software agents. Proceedings of the Second International Joint Conference on
Autonomous Agents and Multiagent Systems. Melbourne: ACM Press.
Richardson, M., & Domingos, P. (2006). Markov logic networks. Machine Learning, 62 , 107–136.
Sammut, C. (1996). Automatic construction of reactive control systems using symbolic machine
learning. Knowledge Engineering Review , 11 , 27–42.
Page 30 Cognitive Architectures

Schlimmer, J. C., & Langley, P. (1992). Machine learning. In S. Shapiro (Ed.), Encyclopedia of
artificial intelligence (2nd ed.). New York: John Wiley & Sons.
Schmidt, R. A. (1975). A schema theory of discrete motor skill learning. Psychological Review , 82 ,
225–260.
Shapiro, D., & Langley, P. (2004). Symposium on learning and motivation in cognitive architectures:
Final report. Institute for the Study of Learning and Expertise, Palo Alto, CA.
http://www.isle.org/symposia/cogarch/arch.final.pdf
Simon, H. A. (1957). Models of man. New York: John Wiley.
Sleeman, D., Langley, P., & Mitchell, T. (1982). Learning from solution paths: An approach to the
credit assignment problem. AI Magazine, 3 , 48–52.
Sloman, A. (2001). Varieties of affect and the CogAff architecture schema. Proceedings of the
AISB’01 Symposium on Emotion, Cognition, and Affective Computing. York, UK.
Sowa, J. F. (Ed.). (1991). Principles of semantic networks: Explorations in the representation of
knowledge. San Mateo, CA: Morgan Kaufmann.
Sun, R. (Ed.). (2005). Cognition and multi-agent interaction: Extending cognitive mdoeling to
social simulation. Cambridge University Press.
Sun, R. (2007). The importance of cognitive architectures: An analysis based on CLARION.
Journal of Experimental and Theoretical Artificial Intelligence, 19 , 159–193.
Sun, R., Merrill, E., & Peterson, T. (2001). From implicit skills to explicit knowledge: A bottom-up
model of skill learning. Cognitive Science, 25 , 203–244.
Sutton, R. S. & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA:
MIT Press.
Taatgen, N. A. (2005). Modeling parallelization and speed improvement in skill acquisition: From
dual tasks to complex dynamic skills. Cognitive Science, 29 , 421–455.
Tambe, M., Johnson, W. L., Jones, R. M., Koss, F., Laird, J. E., Rosenbloom, P. S., & Schwamb,
K. B. (1995). Intelligent agents for interactive simulation environments. AI Magazine, 16 , 15–39.
Trafton, J. G., Cassimatis, N. L., Bugajska, M., Brock, D., Mintz, F., & Schultz, A. (2005). Enabling
effective human-robot interaction using perspective-taking in robots. IEEE Transactions on
Systems, Man and Cybernetics, 25 , 460–470.
Tulving, E. (1972). Episodic and semantic memory. In E. Tulving & W. Donaldson (Eds.), Orga-
nization of memory. New York: Academic Press.
VanLehn, K. (Ed.) (1991). Architectures for intelligence. Hillsdale, NJ: Lawrence Erlbaum.
Veloso, M. M., & Carbonell, J. G. (1993). Derivational analogy in Prodigy: Automating case
acquisition, storage, and utilization. Machine Learning, 10 , 249–278.
Wang, X. (1995). Learning by observation and practice: An incremental approach for planning
operator acquisition. Proceedings of the Twelfth International Conference on Machine Learning
(pp. 549–557). Lake Tahoe, CA: Morgan Kaufmann.