Cognition 214 (2021) 104806

Contents lists available at ScienceDirect

Cognition
journal homepage: www.elsevier.com/locate/cognit

A simple definition of ‘intentionally’

Tadeg Quillien *, Tamsin C. German
Department of Psychological & Brain Sciences, University of California Santa Barbara, 93106 Santa Barbara, CA, USA

A R T I C L E I N F O A B S T R A C T

Keywords: Cognitive scientists have been debating how the folk concept of intentional action works. We suggest a simple
Intentional action account: people consider that an agent did X intentionally to the extent that X was causally dependent on how
Causation much the agent wanted X to happen (or not to happen). Combined with recent models of human causal
Concepts
cognition, this definition provides a good account of the way people use the concept of intentional action, and
Theory of mind
offers natural explanations for puzzling phenomena such as the side-effect effect. We provide empirical support
for our theory, in studies where we show that people’s causation and intentionality judgments track each other
closely, in everyday situations as well as in scenarios with unusual causal structures. Study 5 additionally shows
that the effect of norm violations on intentionality judgments depends on the causal structure of the situation, in
a way uniquely predicted by our theory. Taken together, these results suggest that the folk concept of intentional
action has been difficult to define because it is made of cognitive building blocks, such as our intuitive concept of
causation, whose logic cognitive scientists are just starting to understand.

1. Introduction concept. Roughly, an agent did X intentionally to the extent that X was
causally dependent on how much the agent wanted X to happen (or not to
Much of cognition works under the radar of consciousness. This puts happen). This hypothesis stems from the fact that causation is at the core
us in a strange position: even though we know the meaning of the words of our commonsense psychology; the way we explain and predict the
we use, often we are unable to give them explicit definitions. This has behavior of others relies on a mental causal model of how mental states
caused many debates over the precise meaning of certain concepts. and states of the world interact with one another. Therefore, it makes
These debates can be interesting because the way we use words provides sense that causation would be central to folk concepts about the mind.
important clues about the hidden format of mental representations We want to show that many features of the concept of intentional
(Pinker, 2007; Strickland, 2017). action emerge naturally from our simple theory, given i) the way that
As an example, what do we mean when we say that someone did commonsense psychology works, and ii) the way that the human mind
something “intentionally”? We all have an intuitive understanding of the represents causation. In other words, we are trying to understand the
concept, yet a lot of ink has been spilled by cognitive scientists searching concept by making a hypothesis about its basic building blocks, and
for an explicit definition (e.g., Bennett, 1965; Bratman, 1984; Cova, looking at what cognitive scientists know about these building blocks.
Dupoux, & Jacob, 2012; Cushman & Mele, 2008; Davidson, 1980; Malle This means that our theory can only be as good as our current scientific
& Knobe, 1997; Mele, 2001; Searle, 1983). So far, no strong consensus understanding of the building blocks involved. Nonetheless, we hope to
has been reached – why? show that it can illuminate several puzzling phenomena, and offers new
Maybe the algorithms that our brain uses in order to decide whether fruitful predictions about the way people use the word “intentionally”.
something is intentional are very complicated, or maybe there are not We proceed as follows. First, we briefly review existing accounts of
one but several different concepts of intentional action (e.g. Cova et al., intentional action. Second, we highlight relevant research on
2012; Cushman & Mele, 2008; Malle & Knobe, 1997). Here we defend an commonsense psychology and causal cognition, and from that research
alternative approach: the folk concept of intentional action is relatively derive our definition of intentional action. Third, we show how this
simple, but looks complicated because it is made out of building blocks simple account can explain many known features of people’s use of
the structure of which is not completely obvious a priori. “intentionally”. Fourth, we present the results of six studies testing
We suggest that the concept of intentional action is at its core, a causal predictions of the account. In study 1, we show that our definition

* Corresponding author.
E-mail addresses: tadeg.quillien@gmail.com (T. Quillien), Tamsin.german@psych.ucsb.edu (T.C. German).

https://doi.org/10.1016/j.cognition.2021.104806
Received 8 August 2020; Received in revised form 4 June 2021; Accepted 5 June 2021
Available online 17 June 2021
0010-0277/© 2021 Elsevier B.V. All rights reserved.
T. Quillien and T.C. German Cognition 214 (2021) 104806

closely tracks people’s intuitions about intentionality in everyday cases. that side-effect. By contrast, people do not judge that a CEO who helps the
In study 2, we examine a case where traditional philosophical analyses environment as a side-effect does so intentionally. This result is inconsis­
of causation hold that the agent’s desires caused the outcome, yet people tent with the five-component model, which predicts that in both cases the
do not think the agent acted intentionally (cases of so-called ‘deviant CEO should be viewed as not having intentionally caused the side-effect.
causation’). We show that in such cases, people’s causation judgments The discovery prompted an avalanche of research aimed at explaining
are actually almost as low as their intentionality judgments. In study 3, this “side-effect effect” (e.g. Nadelhoffer, 2006; Wright & Bengson, 2009,
we show that people think that agents can act intentionally even when Pettit & Knobe, 2009; Uttich & Lombrozo, 2010; Cova et al., 2012; Adams
the agent has only a very weak belief that their action will lead to the & Steadman, 2004; Hindriks, 2014; Sripada, 2012; Sloman, Fernbach, &
outcome – this is consistent with our account, but inconsistent with Ewing, 2012; Machery, 2008; Leslie, Knobe, & Cohen, 2006).1 On one
standard theories which posit that belief is a central pre-requisite of account, people may consider that an agent does something intentionally
intentionality. In study 4, we show that causal judgments exhibit a “side- when the agent’s attitude toward an outcome exceeds a given threshold,
effect effect” which parallels that observed for intentionality. Study 5 and moral considerations influence where people put this threshold
shows that statistical norms interact with the causal structure of a sit­ (Knobe, 2010). On another theory, the concept of intentional action might
uation to shape intentionality judgments, in a way uniquely predicted by be fundamentally sensitive to whether people comply with the normative
recent models of causal cognition. Finally, study 6 demonstrates that in a reasons for or against acting (Hindriks, 2014). On yet other accounts, the
case where our account predicts a dissociation between intentionality side-effect effect is not intrinsically about morality, because similar effects
and causation judgments, they do indeed come apart. arise in non-moral scenarios. For instance, people judge that an agent who
pays $1 extra to get an extra-large beverage intentionally pays the extra
2. Existing accounts of intentional action money (despite the fact that paying extra money was not the agent’s goal
in ordering the extra-large beverage; Machery, 2008). According to
In the Philosophical Investigations, Wittgenstein (1953) asks, “What Machery (2008), people view side-effects as intentional when there is a
is left over if I subtract the fact that my arm goes up from the fact that I trade-off between the costs generated by this side effect and the benefits of
raise my arm?”. The puzzle of what makes an event an intentional action the primary goals of the action.
has attracted a large amount of attention from philosophers (e.g., Yet, as Cova (2016) notes, most of the theories that aim to explain the
Anscombe, 1957; Davidson, 1980; Mele, 2009). side-effect effect (or related empirical findings) are relatively limited in
In parallel, psychologists have been interested in the concept because scope: they usually account for, at best, a handful of empirical phenomena
perceptions of intentionality play an important role in social cognition. about how people use the concept of intentional action, but remain silent
For example, whether we perceive an action as intentional influences about, or are inconsistent with, other features of the concept.
how we form impressions about the actor (Heider, 1958), how we judge The lack of prospect for a unified theory of intentional action has even
the action morally and legally (Cushman, 2015), and the nature and led some researchers to suggest that there is none to be found. Instead, they
intensity of the emotions invoked by the action (Sell et al., 2017; Tesser, argue, there might actually be several distinct concepts of intentionality,
Gatewood, & Driver, 1968). each of them invoked depending on the context at hand. For instance,
Intuitively, it seems easy to generate a list of criteria that an action Nichols and Ulatowski (2007) suggest that we sometimes use “S inten­
must exhibit in order to count as intentional. But these list-based defi­ tionally did X" to mean “S had a motive to do X", and sometimes to mean “S
nitions are vulnerable to counter-examples. It is then tempting to deal knew that his action would result in X" (for other polysemic theories, see
with these counter-examples by adding new criteria to the original list. Sousa & Holbrook, 2010; Cushman & Mele, 2008; Cova et al., 2012).
As a result, there has been a steady increase in the complexity of We think that these difficulties may be explained by the fact that
theories of intentional action over the years (as documented by Malle & current approaches tend to follow an inductive strategy. That is, re­
Knobe, 1997). searchers start from the intuitions that people have about intentional
Early philosophical accounts (Aristotle, 2009 (330BC/2009); Hume, action, and try to construct an account that fits these intuitions. Here we
1740) put forward two criteria for acting intentionally: one needs to take a theory-driven approach instead. We start from what cognitive
have a desire for the outcome, and a belief that the act would lead to the scientists know about the mechanisms via which people reason about
outcome. The two-components theory was later found lacking: one can the mind, and we ask: “if a concept of intentional action emerged from
imagine (for example) a basketball player who wants to win the game, the operation of these mechanisms, what would it look like?”
and thinks that fouling would help her achieve that goal, yet does not Our theory shares similarities with causalist approaches in the phi­
foul intentionally when she does. losophy of action (Mele, 2009), notably that of Donald Davidson (1980).
Accordingly, later theories were three-component models: they According to Davidson, what makes an event an intentional action is the
stipulated that beliefs and desires must jointly cause an intention to act fact that it was jointly caused by the relevant beliefs and desires of the
(Brand, 1984; Bratman, 1987; Searle, 1983; Thalberg, 1984). In parallel, agent.
social psychologists identified a fourth component of intentionality: an Causalist approaches have traditionally had difficulty dealing with
agent needs some degree of skill (or control, ability) in carrying out the cases of ‘causal deviance’: scenarios where an agent’s beliefs and desires
action (Heider, 1958; Jones & Davis, 1965; Ossorio & Davis, 1968; jointly cause an outcome but that few people would consider as
Shaver, 1985). For instance, a novice dart player who hits a difficult involving intentional action (see section 4.1 for examples). The exis­
target due to pure luck did not intentionally hit the target, despite the fact tence of such cases led Davidson to specify that causation must happen
that he wanted to (Knobe, 2003b; Malle & Knobe, 1997). “in the right way” to count as intentional; but he did not provide a theory
To probe people’s explicit concept of intentionality, Malle and Knobe of what makes a causal link the right kind of causal link. Indeed, he
(1997) asked undergraduate students to write down their definition of
what it means for someone to do something intentionally. They found
that the explicit folk concept of intentionality contains the four com­ 1
Note that some researchers argue that the effect actually tells us nothing
ponents identified above, as well as a fifth component, awareness of what
about the folk concept of intentional action. Instead of reflecting people’s core
one is doing while doing it.
concept, it arises because motivated reasoning (Alicke & Rose, 2010), or the
To make matters worse, it was later discovered that moral consider­ pragmatics of ordinary conversation (Adams & Steadman, 2004) makes people
ations can have a profound influence on people’s attributions of inten­ use “intentionally” as a way to imply blameworthiness. Or perhaps the effect
tionality (Knobe, 2003a, 2003b). For instance, people judge that a CEO shows that the emotions we feel when evaluating a situation distort our ability
who harms the environment as a side-effect of implementing a new policy to correctly use the concept (Nadelhoffer, 2006). See Knobe (2010) for argu­
did so intentionally, despite the fact that the CEO was indifferent toward ments against such interpretations.

2
T. Quillien and T.C. German Cognition 214 (2021) 104806

Diagram 1. a) The simple ‘Belief+Desire’ causal model. b) The generative causal model suggested by an expected utility framework. “Attitudes” determine how
much the agent values a given state of the world; an attitude can be positive or negative.

considered that such a theory would be a matter of empirical discovery: approximately rational way given their beliefs and desires (Woodward,
facts about the way the mind works ultimately determine what it means 1998; Gergely & Csibra, 2003; Baker, Saxe, & Tenenbaum, 2009; Scott &
for mental causation to count as intentional (Davidson, 1980; see Baillargeon, 2013; Johnson & Rips, 2015; Jara-Ettinger, Gweon, Schulz,
Goldman, 1970, for a similar view). Unlike Davidson, here we are not & Tenenbaum, 2016; Lucas et al., 2014; Liu, Ullman, Tenenbaum, &
trying to give a philosophical or scientific definition of intentional ac­ Spelke, 2017; Jara-Ettinger, Schulz, & Tenenbaum, 2020).
tion: we are interested in the folk concept. This means that it is easier for An expected utility framework suggests a computationally explicit
us to actually provide a theory of what counts as causation “in the right causal model of how people make decisions (sketched in diagram 1b; see
way”, since we only need to determine what counts as such according to Jara-Ettinger et al., 2020 for a more detailed computational model, and
commonsense psychology. empirical tests of its fit to human intuitions).2 Just as the simple causal
model shown in diagram 1a, it partitions the relevant mental states into
3. Building blocks a motivational and an epistemic component.
At the motivational level, agents assign different values (utilities) to
3.1. Commonsense psychology different states of the world; for instance an agent may assign a positive
value to getting food, and a negative value to receiving electric shocks.
As part of their mental toolkit, humans are equipped with a set of We will refer to these value assignments as attitudes: an agent has a
reliably-developing cognitive mechanisms that allow them to predict positive attitude toward an outcome if it assigns a positive value to that
and explain the behavior of others – collectively, these are referred to as outcome, and a negative attitude if it assigns a negative value to the
Theory of Mind, or commonsense psychology (Baillargeon, Scott, & outcome. Thus, to a rough approximation attitudes toward an outcome
Bian, 2016; Dennett, 1987; Leslie, 1994; Leslie, Friedman, & German, can be seen as lying on a continuum from extremely negative to
2004). There are many competing theories of commonsense psychology, extremely positive (see Pettit & Knobe, 2009).3 Henceforth we will use
but most of them share the idea that it is essentially a causal inference this notion of attitude because it is more computationally explicit than
engine: it leverages causal knowledge in order to generate inferences the naive concept of ‘desire’. In particular, ‘desire’ tends to denote a
about people’s mental states and their behavior (Apperly & Butterfill, positive attitude toward an outcome, whereas people are also able to
2009; Baker, Jara-Ettinger, Saxe, & Tenenbaum, 2017; Dennett, 1987; represent the negative attitudes an agent may have toward an outcome.
Gopnik & Wellman, 1992; Leslie, 1994). According to an expected utility framework, beliefs and attitudes
In other words, at a computational level of analysis (Marr, 1982), we jointly determine an agent’s decisions by determining the expected utility
can think of commonsense psychology as relying on an internal causal that the agent assigns to a given action. The expected utility of an action is
model of the way mental states and states of the world interact with each a weighted sum of the utility of all possible outcomes of the action, where
other. Commonsense psychology consists of a set of inference algorithms the utility of an outcome is weighted by its estimated probability. Then,
that leverage this causal model to make a variety of useful inferences (such the agent selects a course of action according to some procedure where
as predicting an agent’s behavior given its mental states, or vice-versa). actions with a higher expected utility are more likely to be selected.
Here we are interested in the part of this causal model that is used to
predict and explain an agent’s behavior given its mental states. A pop­
2
ular idea has been that this part of commonsense psychology relies on By “expected utility framework”, we have in mind something broader than
the kind of causal model depicted in Diagram 1a: people have beliefs and expected utility theory stricto sensu (Morgenstern & Von Neumann, 1953;
desires, which jointly cause their actions (Davidson, 1963; Dennett, Savage, 1954). Expected utility theory assumes that agents obey strict standards
of rationality (for instance, they have transitive preferences). There are other
1987; Wertz & German, 2007).
theories of decision-making that model agents as expected utility maximizers,
without assuming that they obey every axiom of rationality (e.g., prospect
In recent years, cognitive scientists have conducted extensive empir­ theory, Kahneman & Tversky, 1979). The argument we make here does not
ical and modeling work to refine our understanding of this causal model. depend on the exact extent to which people assume other agents to be rational.
Their work suggests that people explain the behavior of others in roughly 3
Although note that at a more mechanistic level, commonsense psychology
the same way an economist would: from an early age, people spontane­ might represent negative and positive attitudes differently at certain stages of
ously model agents as expected-utility maximizers that behave in an processing; see Leslie & Polizzi, 1998; Leslie, German & Polizzi, 2005.

3
T. Quillien and T.C. German Cognition 214 (2021) 104806

intuitively the king did not intentionally build the bridge.

3.2. Causal judgment

Most of our causal knowledge is embedded in domain-specific

cognitive mechanisms (such as commonsense psychology), but we also
clearly possess an abstract concept of causation that we can apply across
domains: people spontaneously use words like “cause” and “because” to
talk about almost anything (Gerstenberg & Tenenbaum, 2017; Icard,
Kominsky, & Knobe, 2017; Quillien, 2020).
To the human mind, causation is a matter of counterfactual depen­
dence. “C caused E" means that in a counterfactual alternative scenario
where we ‘force’ C to not occur, E does not occur either (Gerstenberg,
Peterson, Goodman, Lagnado, & Tenenbaum, 2017; for philosophical
analyses of causation along these lines see e.g. Lewis, 1973; Hitchcock,
2001; Halpern & Pearl, 2005; Weslake, 2015)5.6 Consider a forest that
Diagram 2. Causal model for the King scenario. Attitudes affect the state of the catches on fire after a lightning bolt strikes a tree. If the lightning had not
world via a causal pathway (black solid arrows), which is different from the one struck, the forest would not have caught fire: therefore the lightning bolt
pre-specified by commonsense psychology (grey dashed arrows). is a cause of the fire. Similarly, if there was no oxygen in the air to fuel
the combustion, the fire would not have started: therefore oxygen is a
cause of the fire.
Most philosophical analyses of causation have been egalitarian: they
do not discriminate between different causes (Hall, 2004; Lewis, 1973;
This framework motivates our suggestion that, to the mind, inten­ Mill, 1856), and would regard the lightning bolt and the oxygen as
tionality is about the existence of a causal relationship between an equally good causes of the fire. By contrast, our intuitive sense of
agent’s attitude toward a state of the world and that state of the world causation does discriminate between causes (Hart & Honoré, 1985;
obtaining. Morris, Phillips, Gerstenberg, & Cushman, 2019). Most people view the
We can now formulate our hypothesis: lightning bolt, rather than the oxygen, as the main cause of the fire.
For the human mind, an agent did X intentionally if the agent’s attitude Similarly, most people think that a presidential candidate won the
toward X caused X, and caused X according to the typical causal model election because he won the swing state, not because he won the party
implicit in our commonsense psychology. stronghold (Quillien & Barlev, 2021).
Recently, cognitive scientists have made progress in understanding
Here are two examples4: what drives gradation in causal judgment (Gerstenberg, Goodman,
Lagnado, & Tenenbaum, 2021; Icard et al., 2017; Quillien, 2020).7
Window. “Anne opens the window in order to let sunlight into the
When people think about the causes of an event, they implicitly
room. She believed that opening the window would let sunlight into
consider several different counterfactuals to the event. For example,
the room, and had a positive attitude toward that outcome. This
when thinking about the forest fire, people may consider other possible
attitude – belief pair led her to compute a high expected utility for
versions of the event. These can include a counterfactual where there is
the action of opening the window, and she chose to do so as a result.”
no lightning bolt (and the fire does not start), a counterfactual where the
Anne’s desire to let sunlight into the room caused sunlight to enter wind is stronger (and the fire spreads even faster), a counterfactual
the room, and caused it in the typical way specified by commonsense where the ground is wet, etc. People tend to think that C is a cause of E to
psychology. Accordingly, it feels natural to say that Anne intentionally the extent that C and E are highly correlated across these counterfactuals
let sunlight into the room. (Quillien, 2020; for empirical evidence consistent with this account see,
e.g., Lombrozo, 2010; Icard et al., 2017; Gerstenberg & Icard, 2019;
King. “The king’s advisors have put a high-tech brain sensor on the Kominsky, Phillips, Gerstenberg, Lagnado, & Knobe, 2015; Kominsky &
king’s head, which gives them a direct readout on what the king Phillips, 2019; Henne, Pinillos & De Brigard, 2017; Henne, Niemi,
wants. Wondering whether they should build a bridge over the river,
the advisors consult the brain sensor, and thereby learn that the king
would be in favor of building a bridge if he were to be asked. Now 5
Contemporary models of causation, though they rely heavily on this
that they know how the king feels, they go ahead and build the counterfactual criterion, are of course more complicated. They are also
bridge, without bothering to formally ask him. designed to deliver the correct intuition in cases where a factor is not necessary
for the effect. For example, when two soldiers in a firing squad fire at a prisoner
Here, the king’s desire to build a bridge caused the bridge to be built,
at the same time, neither soldier is individually necessary for the prisoner’s
but this causal path clearly deviates from the way that, according to death, although it makes sense to think of each soldier as a cause of the pris­
diagram 1b, desires cause outcomes: the king’s desire did not cause the oner’s death; see Halpern (2016) for review.
bridge to be built via a decision that the king made. In order to model the 6
In this paper, we focus on so-called ‘dependence’ theories of causation.
causal structure of this scenario, one needs to add extra causal links to Some philosophers (Dowe, 2000) and psychologists (Wolff, 2007) favor ‘pro­
the typical causal model depicted in Diagram 1b; see Diagram 2. So, cess’ theories, which view causation as being (or being represented as) a
physical exchange of entities between events – but we lack the space to discuss
them.
7
4 Although for some alternatives to the general framework described here,
For the sake of argument, here we assume that commonsense psychology
see Wolff, 2007; Alicke, Rose, & Bloom, 2011; Sytsma, Livengood, & Rose,
does contain a causal model resembling the one sketched in diagram 1b. The
2012.
general principle stated above could work with alternative versions of this
causal model – the exact form this model takes is ultimately a matter of
empirical discovery.

4
T. Quillien and T.C. German Cognition 214 (2021) 104806

Pinillos, De Brigard, & Knobe, 2019; and especially Morris et al., 2019; cases come in two kinds.
Quillien & Barlev, 2021).8 The first kind of case is where an agent’s attitude causes an outcome
Additionally, some counterfactuals come to mind more readily than in a way that deviates from the domain-specific causal model of
others. Intuitively, if you witnessed the event leading to the forest fire, commonsense psychology. In such situations, our account explicitly
your first thought would probably not be “what if there had been no predicts that people will not attribute intentionality. The King scenario
oxygen in the air?”, because this possibility is extremely unlikely a in section 3.1 is one such example. Another case was famously discussed
priori. Instead you probably would be thinking about the fact that by Donald Davidson:
lightning might not have struck.
Climber. “A climber might want to rid himself of the weight and
Across the counterfactuals that people spontaneously generate, the
danger of holding another man on a rope, and he might know that by
correlation between “there is oxygen in the air” and “the forest is on fire" is
loosening his hold on the rope he could rid himself of the weight and
very low (notably because of the many counterfactuals where there is
danger. This belief and want might so unnerve him as to cause him to
oxygen but nothing to spark the fire). This fact explains why it feels strange
loosen his hold, and yet it might be the case that he never chose to
to say that oxygen caused the fire. By contrast, lightning bolts and forest
loosen his hold, nor did he do it intentionally” (Davidson, 1980).
fires tend to strongly co-occur across counterfactuals, and therefore we
intuitively say that the lightning bolt caused the forest fire (Quillien, 2020). We suggest that people do not judge the climber’s action as inten­
Counterfactual models of causation also successfully predict that tional because the event cannot be represented using the standard causal
normative considerations impact causal intuitions. People are biased to model depicted in Diagram 1b. Under this causal model, the only way the
generate counterfactuals that are statistically normal (as explained climber’s desire can cause him to loosen his hold is by affecting the ex­
above) but they are also biased toward counterfactuals that are pre­ pected utilities he computes for each alternative course of action, altering
scriptively normal, i.e. where agents do not violate ethical or legal norms the decision he eventually makes. But this is not what happens in
(Byrne, 2016). For instance, if two cars collide at an intersection, it feels Climber. In order to represent the event, we need to use an ‘augmented’
more natural to ask “what if the car that went through the red light had causal model, namely the one depicted in Diagram 3, which includes a
stopped instead” rather than mentally changing the behavior of the car new causal path involving nervousness. This alternative causal pathway
that went through the green light. Therefore, we are more inclined to say prevents people from judging that the climber’s desire (i.e. his positive
that the car that went through the red light caused the collision (see attitude toward the outcome) caused the outcome in the right way, and
Hitchcock & Knobe, 2009; Samland, Josephs, Waldmann, & Rakoczy, therefore it makes them reluctant to judge the event as intentional.9
2016; Icard et al., 2017; for the thesis that normality has both a
descriptive and prescriptive meaning, see Kahneman & Miller, 1986; The second kind of case stems from the fact that people’s domain-
Bear & Knobe, 2017; Bear, Bensinger, Jara-Ettinger, Knobe, & Cushman, general concept of causation is not egalitarian (recall that, e.g., it feels
2020; Phillips, Morris, & Cushman, 2019). strange to say that oxygen in the air caused the forest fire). As a result,
In sum, the psychology of causal judgment is no longer an entirely people are sometimes reluctant to judge that an agent’s attitude caused
black box to cognitive scientists. Recent models naturally explain why an outcome, even when the outcome counterfactually depended on the
people often tend to deny a causal role to events that would be attitude.
considered causal under the egalitarian conception of causation preva­ For example, Knobe (2003b) asked participants about the following
lent in philosophy. Additionally, counterfactual models of causal judg­ case:
ment can make (often fine-grained) predictions about the causal
intuitions that people will have in a given situation. If our theory of Bull’s-eye. Jake desperately wants to win the rifle contest. He knows
intentional action is correct, then the variables that these models iden­ that he will only win the contest if he hits the bull’s-eye. He raises the
tify as important to causal judgment should also shape intentionality rifle, gets the bull’s-eye in the sights, and presses the trigger. But Jake
judgments. We will test this prediction in studies 2, 4 and 5. isn’t very good at using his rifle. His hand slips on the barrel of the
Recall that earlier we stated our hypothesis as: gun, and the shot goes wild … Nonetheless, the bullet lands directly
on the bull’s-eye. Jake wins the contest.
For the human mind, an agent did X intentionally if the agent’s attitude
toward X caused X, and caused X according to the typical causal model Only 28% of participants judged that Jake intentionally hit the bull’s
implicit in our commonsense psychology. eye (by contrast, 79% of participants ascribed intentionality to Jake
when he was described as an expert marksman executing a perfect
We are now able to specify that by “caused”, we mean the intuitive, shot).10 Under our account “Jake intentionally hit the bull’s eye" means,
graded concept of causation, instead of the egalitarian notion. Next we roughly: “The bullet hit the bull’s eye because Jake wanted it to”. The
explore the fit between our account and people’s intuitions. latter statement is true according to an egalitarian theory of causation:
the bullet would not have hit the bull’s eye if Jake had not wanted it to.
4. Explaining intuitions about intentional action However, we suspect that people would not share the verdict of the
egalitarian theory. Intuitively, the real cause of Jake’s success was dumb
4.1. Deviant causation luck.
Why does it feel strange to say that the bullet hit the bull’s eye
We first address cases in which people have the kinds of intuition because Jake wanted it to? The counterfactual model of causal judgment
that seem a priori most damning to our causalist account. In cases of (Quillien, 2020) described in section 3.2 provides an explanation. When
“deviant causation”, an agent’s attitude toward X caused X, but intuition people think about the case, they implicitly generate several counter­
suggests that the agent did not intentionally do X. We suggest that these factuals to the event. Across these counterfactuals, they compute the
correlation between “Jake wants to hit the bull’s eye" and “the bullet hits

8
For ease of exposition we are somewhat simplifying the theory. Obviously, a
9
correlation between two variables is not always indicative of causation – for Cosmides (1985, chapter 5) makes a similar argument to account for cases
instance, lightning causes both thunder and fire, so there will be a high cor­ of causal deviance in social exchange.
10
relation between thunder and fire across counterfactual worlds, but it would be Interestingly, in a scenario pair which is similar, except that Jake’s inten­
invalid to judge that the thunder caused the forest fire. The model developed in tion is immoral, causal deviance has much weaker effects on judgments of
Quillien (2020) easily deals with such cases, but getting into these details is not intentionality (Knobe, 2003b; see also Sousa, Holbrook & Swiney, 2015). We
crucial here. return to this point in the General Discussion.

5
T. Quillien and T.C. German Cognition 214 (2021) 104806

When reading the story, we infer that the chairman’s attitude toward
harming the environment is neutral, or perhaps mildly negative: the
chairman might view harming the environment as a somewhat unfor­
tunate outcome, but not enough so as to overcome his lust for profit. But
the normative expectation is that the chairman should be sufficiently
opposed to harming the environment that he would refuse to implement
the program. Therefore we tend to mentally replace the event with one
where the chairman’s attitude conforms to this normative expectation
(we are unlikely to mentally replace the event with one where the
chairman values the environment less than he does in the actual situa­
tion). In such a counterfactual, changing the chairman’s attitude pre­
vented harm to the environment.
Thus, computational accounts of causal judgment (Icard et al., 2017;
Quillien, 2020) predict that people will judge that the environment was
harmed because the chairman did not care about the environment (we test
this prediction in study 4). In turn, this predicts that people will judge that
the chairman intentionally harmed the environment. This is indeed what
Knobe (2003a) finds: people judge that the chairman intentionally
Diagram 3. Causal model for the climber scenario. Attitudes affect the state of harmed the environment, contradicting the standard assumption that
the world via a causal pathway (black solid arrows), which is different than the only agents who want X to happen can do X intentionally.
one pre-specified by commonsense psychology (grey dashed arrows).

4.2.2. Belief is important, but can be weak

the bull’s eye". This correlation is low: Jake is a novice marksman, so in
Our account implies that for X to be intentional, the agent must
most counterfactuals, he wants to hit the target but completely misses.
believe that his action has a non-zero likelihood of bringing about X.
Therefore, Jake’s attitude is not an important cause of the outcome, and
This requirement follows from the causal model implicit in common­
Jake did not intentionally hit the bull’s eye.
sense psychology. In this causal model, attitudes can only influence the
state of the world by affecting expected utilities, and attitudes can only
influence expected utilities if the agent has the relevant beliefs. Let us
4.2. Explaining recurrent features in existing accounts
say I have a choice between doing A and not doing A. If I believe that
regardless of what I choose, X will not happen, then my attitude toward
There are many different accounts of the meaning of “intentionally”,
X cannot possibly have the right kind of causal influence on whether I
but historically most of them have shared the two following re­
choose to do A.11 Therefore I must believe that doing A will lead to X
quirements. In order for an action to be intentional, the agent must have
with non-zero probability in order to do X intentionally.
a Desire for the outcome to occur and a Belief that their action will bring
Note that this leaves open the following possibility: an agent may be
about this outcome. When asked for their explicit definition of the
intentional even if he believes that his action has a very low probability
concept, laypeople also systematically say that Desire and Belief are
to lead to X. What matters is that the agent believes that his action in­
central to intentional action (see Malle & Knobe, 1997, for a historical
creases the probability of X at least a little. For instance, if the agent has a
review, and for empirical data about the explicit folk concept).
sufficiently strong desire for X, and/or taking the action is not very
By contrast, our account does not explicitly mention either Desire or
costly, then they may decide to take the action because they think it may
Belief as being necessary for an action to be intentional. Nevertheless,
lead to X, even if they think that this probability is very dim.
we show below that it can explain why in most cases people will tend to
Intuition seems consistent with this possibility. For instance, Davidson
view these features as essential to intentionality. Our account also pre­
(1980) remarks: “in writing heavily on this page I may be intending to
dicts that people will sometimes attribute intentionality to agents who
produce ten legible carbon copies. I do not know, or believe with any
do not have a desire for the outcome, or to agents who have only a weak
confidence, that I am succeeding. But if I am producing ten legible carbon
belief that their action would bring about the outcome. As we will show,
copies, I am certainly doing it intentionally” (Davidson, 1980, Essay 4,
such situations do occur.
p.82). We test this intuition more systematically in study 3.
In summary, our account can explain why people do not attribute
4.2.1. Desire is important, but not essential
intentionality to agents in cases of deviant causation, and why standard
A desire for X is simply a positive attitude toward X. Attitudes toward
theories of intentional action have the features that they do. Our account
an outcome are more likely to lead to that outcome when they are
can also explain why people’s intuitions sometimes deviate from the
positive. For instance, if someone eats vanilla ice cream, it is usually
predictions made by their explicit theory of the concept and by standard
because they wanted to eat vanilla ice cream. Therefore, the typical case
scholarly theories.
when an attitude leads to X is when the agent has a desire for X. Ergo,
We now turn to empirical tests of our definition.
under our account, in most cases agents do X intentionally because they
want X to happen.
Yet even a neutral, or a negative attitude toward X can cause X. 5. Empirical tests
Notably, a negative or neutral attitude can be considered causal if it is
not negative enough. Consider the well-known Chairman vignette Here we report the results of six empirical tests of our account. Our
designed by Knobe (2003a): general strategy is relatively simple: if people’s judgments of inten­
tionality derive from their mental representations of causation, then
Chairman. The vice-president of a company went to the chairman of their judgments of causation and their judgments of intentionality
the board and said, ‘We are thinking of starting a new program. It should track each other. In other words, when people judge that an
will help us increase profits, but it will also harm the environment.’
The chairman of the board answered, ‘I don’t care at all about
harming the environment. I just want to make as much profit as I can. 11
More generally, if I believe that the probability that X will happen is the
Let’s start the new program.’ They started the new program. Sure same regardless of whether I choose A or not A, then my attitude toward X
enough, the environment was harmed. cannot play a causal role in my decision.

6
T. Quillien and T.C. German Cognition 214 (2021) 104806

agent did X intentionally, they should also judge that X was causally condition. Participants in the “Intentionality” condition were asked to
dependent on the agent’s attitude toward X. As a corollary, manipula­ rate, for each statement, whether Anne did what she did intentionally,
tions that are known to affect causation judgments should affect inten­ on a likert scale from 1 (not intentional at all) to 7 (very intentional).
tionality judgments, and vice-versa. Participants in the “Dependence” condition were asked to rate, for each
Note that our account does not strongly predict that causation and statement, whether the event described depended on whether Anne
intentionality judgments will always perfectly track each other. We as­ wanted it to happen or not, on a scale from 1 (completely independent)
sume that when people compute whether X was caused “in the right way” to 7 (completely dependent). We asked about dependence instead of
for the purpose of assessing intentionality, they rely on two kinds of explicitly using the expression ‘causally depended’ because the latter
cognitive systems: the domain-specific model implicit in commonsense sounds less natural, and in the context of our statements it is clear that
psychology, and the domain-general concept of causation. By contrast, ‘dependence’ refers to a causal link.12
when people are explicitly asked whether the agent’s attitude toward X
caused X, it is possible that they rely more exclusively on their domain- 5.1.2. Results and discussion
general concept of causation. If this is the case, then one might The item-level correlation between causal dependence and inten­
construct contrived thought experiments, similar to King (see section 3.1) tionality ratings was almost perfect, r(17) = .96, p < .001; see Fig. 1.
where people judge that the agent’s attitude toward X was highly causal, Interestingly, intentionality ratings look like “stretched out” versions
even though they judge that he did not do X intentionally. We test this of the dependence ratings: they are more likely to lie close to the end­
prediction in study 6. More generally, a variety of pragmatic and moti­ points of the scale. We do not really know why this is the case. It may be
vational factors may distort how people answer queries about intention­ that, compared to causation, people are more reluctant to treat inten­
ality and causation, leading to non-identical patterns of responses. tionality as a graded concept. Or maybe participants were slightly more
Nonetheless, the prediction that causation and intentionality judg­ confused by the causal dependence question.
ments will tend to track each other constitutes a non-trivial prediction of To get a better subjective sense of the tight fit between ratings on the
our account, which makes it worth testing. Some of our studies (studies two questions across events, we can look at scaled mean ratings: we
2–4) were additionally designed to provide empirical support to the created Fig. 2 by computing z-scored means for dependence and inten­
explanations we have given for some phenomena in the previous sec­ tionality ratings independently (by, e.g., subtracting the mean inten­
tions of this paper. In summary, the studies we report are meant to tionality rating for a given event from the grand mean rating for
provide evidence for a causalist definition of “intentionally”, and to intentionality, and dividing by the standard deviation in mean inten­
serve as a proof of concept that cognitive science models of causal tionality ratings across events).
cognition can shed light on people’s intuitions about intentional action. The results strongly support the idea that people understand inten­
Data and R code for all studies are available at the Open Science tionality in terms of the causal dependence of the outcome on the agent’s
Framework at https://osf.io/42x7h/ attitude toward the outcome. However, the sentences we used depict
everyday situations. In the following studies, we “stress-test” our theory
by exposing it to more exotic cases.
5.1. Study 1: intuitions in everyday situations

5.2. Study 2: causal deviance

Study 1 was a very simple preliminary test of our theory. We asked
participants to read a series of 19 short statements about various events
On the surface, cases of ‘causal deviance’, where an agent’s attitude
involving a person called Anne. Half the participants were asked to rate
whether Anne was doing what she was doing intentionally. The other
half were asked whether what happened depended on whether Anne
wanted it to happen. We predicted that answers to both questions would
closely track each other.

5.1.1. Methods

5.1.1.1. Participants. We recruited 200 participants from Ama­

zonMTurk. Five participants were excluded from analysis for failing a
catch item (typing “4” in response to a picture displaying the question
“What is 12-8?” – we used this catch item in all studies reported here),
leaving a total of 195 participants (97 female).

5.1.1.2. Stimuli. Participants read a series of 19 short sentences, which

we adapted from study 1 in Malle and Knobe (1997). All sentences
describe a person called Anne doing something, for instance “Anne was
sweating”, “Anne got admitted to Princeton”, “Anne stole a pound of
peaches”, etc. (see Appendix for complete list of statements). The sen­
tences were identical to the statements used in the original study, except
that we modified some of them so that they were all in the past tense.
Fig. 1. mean ratings for Dependence and Intentionality, for each sentence in
These stimuli were originally designed by Malle and Knobe (1997) for a
Study 1. Error bars represent standard errors of the mean.
different purpose than the present study, namely to study inter-rater
agreement in intentionality ratings. Conveniently, they were designed
so that ratings would span a wide range, with different stimuli expected 12
More generally, across the studies reported here we could not systematically
to elicit low, intermediate and high intentionality ratings.
use the same measure of how much the participants judged that the agent’s
attitude toward X caused X, because the abstract concept of “attitude toward X"
5.1.1.3. Procedure. All 19 statements were presented on the same page, is difficult to express in English. So, for instance, in studies where the situation
in random order. Participants were randomly assigned, in a between- makes it clear that the agent had a desire for X, we ask about whether X
subjects design, to either an “Intentionality” or a “Dependence” happened because the agent wanted X.

7
T. Quillien and T.C. German Cognition 214 (2021) 104806

(between-subjects) to one of the two following vignettes:

Alice hates Bob. One day she decides to go buy a gun, in order to kill
him. She gets in her car and starts driving in the direction of the gun
shop. Someone suddenly crosses the street in front of her. [She re­
alizes that the pedestrian is Bob. Seizing the opportunity, she steps on
the gas and runs him over/Unbeknownst to her, the pedestrian is
Bob. She steps on the brake, but it is too late and she runs him over].
Bob dies on the spot.

Each participant was asked two questions: an Intentionality and a

Causation question. Question order was randomized across participants:
half the participants answered the Intentionality question first, the other
half answered the Causation question first. Questions appeared on
different pages of the computer-based survey. The Intentionality ques­
tion was the same for every participant: they were asked to rate their
agreement with the statement “Alice intentionally killed Bob” on a 1–7
likert scale (1:strongly disagree, 7:strongly agree). For exploratory
purposes, we varied the wording of the Causation question across par­
ticipants: half the participants were asked to rate their agreement with
the statement “Alice’s desire to kill Bob caused Bob to die", the other half
Fig. 2. Mean scaled ratings for dependence and intentionality, study 1. Error
were asked to rate their agreement with the statement: “Bob died
bars represent standard errors of the mean.
because of Alice’s desire to kill Bob”.

toward X caused X, yet the agent did not do X intentionally, seem like 5.2.2. Results
obvious counter-examples to our theory. Results were consistent with our predictions (see Fig. 3). Intention­
We have argued that in many such cases, causation occurs only in the ality ratings were higher in the Normal condition (M = 6.78, SD = .80)
egalitarian sense of metaphysicians; to our mind’s intuitive concept of than in the Deviant condition (M = 2.76, SD = 1.89), t(128.95) = 19.03,
causation, the attitude did not really cause X, or did so only weakly – p < .001, d = 1.89. Similarly, Causation ratings were higher in the
hence the intuition that the agent did not do X intentionally. For Normal condition (M = 6.17, SD = 1.52) than in the Deviant condition
instance, a typical case of causal deviance (inspired by Chisholm, 1966) (M = 3.48, SD = 2.10), t(175.9) = − 10.38, p < .001, d = 1.49.
goes like this: Alice wants to kill Bob and decides to drive to the gun In order to compare the size of these effects, we conducted a mixed
store; while driving, she runs over a pedestrian, who turns out to be Bob. Anova, which revealed that the type of causal link (Normal vs Deviant)
This case elicits the intuition that Alice did not intentionally kill Bob. had a larger effect on intentionality than on causation ratings (interac­
Yet, according to the egalitarian notion of causality, Alice’s desire to kill tion: F(1, 199) = 27.10, p < .001, η2partial = .12).
Bob caused Bob to die. We did not find any order effects or wording effects. The effect of the
This case can be reconciled with our account by positing that people type of causal link on Causation ratings did not depend on the wording
view “Alice wants to kill Bob” as only weakly causal for “Bob died”. of the causation question (“caused” vs “because”)13, F(1,197) = 0.00, p
Counterfactual models of causal judgment (Quillien, 2020; Icard et al., = .95; and the order of questions did not affect the effect of causal link
2017; see also Kominsky et al., 2015) predict that people will indeed on either the causation or the intentionality rating (all Fs < .85, all ps >
assign a relatively low causal weight to Alice’s desire (i.e. to her positive .35).
attitude toward the outcome). The fact that Bob happened to cross the We also tested whether the effect of the type of causal link on
street right at this particular moment is a coincidence, i.e., an event with intentionality ratings was mediated by causation ratings. To do so, we
low a priori probability. It is easy to think of counterfactuals where Alice conducted a mediation analysis with 5000 resamples using the lavaan
wants to kill Bob and drives toward the gun store, but Bob does not cross package in R (Rosseel, 2012). We found that there was a significant
the street, or does so at a slightly different time. In these counterfactuals, indirect effect of condition on intentionality ratings via causation ratings
Bob is still alive right after Alice drives through that particular street. (95% CI [0.642, 1.561]; see Supplementary Information at https://osf.
This means that, across possible counterfactuals to the event, there is a io/dp5xr/ for the full model).
relatively low correlation between “Alice wants to kill Bob”, and “Bob
dies”. As a consequence, people will be reluctant to judge that Alice’s 5.2.3. Discussion
desire to kill Bob was the cause of Bob’s death. They should then also In both versions of the story we used, the egalitarian answer to the
deny that Alice intentionally killed Bob. causation question is that Alice’s desire to kill Bob caused Bob to die: if
In study 2, we test this prediction by probing people’s intuitions about Alice had not wanted to kill Bob, she would not have taken the wheel,
causation and intentionality in the “causal deviance" case described and Bob would still be alive. Yet people’s causal intuitions were more
above. We also created a matched story where the causal link between subtle: people viewed Alice’s desire as causally important to Bob’s death
Alice’s desire and Bob’s death is straightforward. We predict that in the scenario where the causal link was straightforward, but viewed it
compared to the straightforward causal link story, the causal deviance as much less so in a “causally deviant” scenario where counterfactual
story will elicit lower intentionality and lower causation ratings. models of causal judgment assign low causal strength to Alice’s desire.
Correspondingly, they only considered that Alice killed Bob intention­
5.2.1. Methods ally in the scenario involving a non-deviant causal link.
Though our manipulation of the causal link had a similar effect for
5.2.1.1. Participants. We recruited 203 participants from Amazon both causation and intentionality ratings, this effect was somewhat
MechanicalTurk. Two participants were excluded from analysis for
failing a catch item, leaving a final sample of 201 participants (112 fe­
male, 1 other). 13
We are not aware of much research that has looked at the differences be­
tween ‘cause’ and ‘because’ statements, although see Livengood and Machery
5.2.1.2. Stimuli and Procedure. Participants were randomly assigned (2007) for a preliminary investigation in the context of causation by absence.

8
T. Quillien and T.C. German Cognition 214 (2021) 104806

We designed Study 3 to test the generalizability of this finding, and to

confirm that in this sort of case, people judge that the agent only has a
very weak belief that his action will lead to the outcome. We asked
participants to read Mele & Cushman’s Bowling scenario; in addition to
their ratings of intentionality, we also asked them whether the agent
believed that his action would lead to the outcome, and whether the
outcome was caused by the agent wanting the outcome to occur. We also
designed two additional vignettes that we predicted would elicit high
intentionality ratings despite low belief ratings.

5.3.1. Methods

5.3.1.1. Participants. We recruited 90 participants on Amazon

MechanicalTurk. Twenty-four participants were excluded from analysis
for failing a catch and/or a comprehension item (see below), yielding a
final sample of 66 participants (32 female, 1 who declined to state).
Fig. 3. Ratings of Causation and Intentionality for Normal and Deviant causal
links. Error bars represent standard errors of the mean. Individual data points 5.3.1.2. Stimuli and procedure. Participants were randomly assigned
are jittered for better visibility. (between-subjects) to one of three vignettes. The Bowling vignette was
adapted verbatim from Mele and Cushman (2007). The Sabotage
stronger for intentionality than for causation. We do not really know vignette depicted a disgruntled worker in a power plant who tries to shut
why this is the case, but we note that this is consistent with the pattern down the main reactor by pushing a red button, even though he knows
found in Study 1: compared to causation ratings, average intentionality he does not have the necessary security key – because of an oversight,
ratings are closer to the endpoints of the scales. the safety feature was turned off, and by pushing the red button he shuts
Overall, the results of study 2 suggest one way that cases of causal down the reactor. In the Shooter vignette, a shooter realizes at the last
deviance can be consistent with a causalist account of “intentionally”. moment that he forgot to put bullets in his rifle, but decides to fire at his
Even when the agent’s attitude toward the outcome was technically victim anyway – as it turns out, there were actually some bullets left over
necessary for the outcome, people may not be judging it as strongly in the rifle, and the victim dies. See appendix for the full text of the
causal. If, across counterfactuals to an event, the outcome was only vignettes.
weakly causally dependent on the agent’s attitude, people will tend to Each participant was asked three questions: an Intentionality, a
deny that the agent’s attitude was the cause of the outcome. As a result, Belief, and a Causation question. Since our main hypothesis was about
they will deny that the event was intentional. intentionality and belief, the causation question was always presented
In sum, the logic of our intuitive, domain-general concept of causa­ last. Half the participants answered the Intentionality question first, the
tion might explain many cases of causal deviance. Our account also other half answered the Belief question first. Questions appeared on
predicts that in other cases, a causal link may be treated as deviant different pages of the computer-based survey. Participants were asked to
because it travels outside of the causal model of commonsense psy­ rate their agreement with the following statements, on a 1–7 likert scale
chology. We examine this possibility in a later study (study 6). (1: strongly disagree, 7: strongly agree):

- Earl intentionally knocked down all the pins.

5.3. Study 3: intentionality despite weak belief - Earl believed that he would knock down all the pins.
- All the pins were knocked down because Earl wanted to knock down
A standard feature of virtually all accounts of intentional action is all the pins.
that an agent who does X intentionally must believe that their action will
lead to X (see section 2). As we explain in section 4.2, our account shares The first page also featured a Comprehension question (e.g. “Earl’s
this feature, but makes the additional prediction that there will be cases friends didn’t know that the pins on lane 12 were normal wooden ones”,
where an agent is judged as intentional even if their belief that the action with options: True/False/Impossible to tell); participants failing to
will lead to the outcome is very weak. provide the correct answer were excluded from analysis.
A study by Mele and Cushman (2007) provides some support for the (See Appendix for the questions associated with the other two
prediction. Participants in that study read the following vignette: vignettes).
Bowling. Earl is an excellent and powerful bowler. His friends tell
him that the bowling pins on lane 12 are special 200-pound metal 5.3.2. Results and discussion
pins disguised to look like normal pins for the purposes of a certain Participants tended to attribute weak belief to the agent (M = 2.24,
practical joke. They also tell him that it is very unlikely that a bowled SD = 1.63), although they attributed medium-to-high amounts of
ball can knock over such pins. Apparently as an afterthought, they intentionality (M = 4.64, SD = 2.30) and causation (M = 4.88, SD =
challenge Earl to knock over the pins on lane 12 with a bowled ball 2.17), see Figs. 4-5.
and offer him ten dollars for doing so. Earl believes that his chance of Intentionality ratings were higher than Belief ratings, as assessed by
knocking over the pins on lane 12 is very slim, but he wants to knock a 2*3 mixed Anova with Question Type (Intentionality vs Belief) and
them down very much. He rolls an old bowling ball as hard as he can Vignette as predictors: the main effect of Question Type on ratings was
at the pins, hoping that he will knock down at least one. To his great significant, F(1,63) = 69.6, p < .001, η2partial = .51. There was no
surprise, he knocks them all down! The joke, it turns out, was on Earl: interaction, F(2,63) = 1.43, p = .25, and no main effect of Vignette, F
The pins on lane 12 were normal wooden ones. (Mele & Cushman, (2,63) = .47, p = .63.
2007, p. 187). We found no effect of Question Order (whether participants saw the
Intentionality or the Belief question first) on ratings: a 2*2 Anova with
Participants overwhelmingly agreed that Earl intentionally knocked Question Type and Question Order as predictors failed to find a main
down the pins (M = 6.36 on a 1–7 scale), even though the story mentions effect of Question Order, F(1,128) = 1.00, p = .32, or an interaction
that he believes his chance of doing so is very slim.

9
T. Quillien and T.C. German Cognition 214 (2021) 104806

though he does not desire X. Knobe also found a striking asymmetry in

people’s judgments: participants reading the same vignette, with the
word “harm” replaced by “help” overwhelmingly denied that the
chairman intentionally helped the environment. There have been a large
number of attempts to explain this asymmetry, known as the Knobe
effect or side-effect effect (Nadelhoffer, 2006; Wright & Bengson, 2009,
Pettit & Knobe, 2009; Uttich & Lombrozo, 2010; Cova et al., 2012;
Adams & Steadman, 2004; Hindriks, 2014; Sripada, 2012; Sloman et al.,
2012; Machery, 2008; Leslie et al., 2006).
The side-effect effect makes intuitive sense in our framework: in the
Harm vignette, it seems natural to say that the environment was harmed
because the chairman does not care about the environment, while in the
Help vignette, it seems unnatural to say that the environment was hel­
ped because the chairman does not care about the environment.
This asymmetry in causal attributions can be explained by the psy­
chological logic of causal judgment. We assume that there is a normative
expectation that people ought to strongly value the environment.
Fig. 4. Belief, causation and intentionality ratings for each vignette. Error bars
represent standard errors of the mean. Individual data points are jittered for Therefore, the attitude of the chairman toward the environment is
better visibility. abnormal. When they make causal judgments, people will tend to
generate counterfactuals where the chairman has a more normal atti­
tude, i.e. counterfactuals where the chairman values the environment
more than he does in the vignette.
In the Harm case, considering these counterfactuals ends up chang­
ing the outcome: in counterfactuals where the chairman values the
environment sufficiently highly, he will oppose the program. By
contrast, in the Help case, whether the chairman has a neutral or a
positive valuation of the environment does not matter for the outcome
(in both cases the chairman implements the program, and the environ­
ment is helped). Therefore, when people consider different attitudes that
the chairman could have had, they see that this has a large effect on the
outcome in the Harm case, but a very small effect in the Help case. As a
result, they judge that the chairman’s attitude is much more causally
important in the Harm case.
Study 4 was designed to test the prediction that people make higher
causal attributions in the Harm case than in the Help case.

5.4.1. Methods
Fig. 5. Belief, causation and intentionality ratings, collapsed across vignettes.
Error bars represent standard errors of the mean. Individual data points are 5.4.1.1. Participants. We recruited 210 participants from Amazon
jittered for better visibility. MechanicalTurk. Twelve participants were excluded from analysis for
failing a catch item, yielding a final sample of 198 participants (83
between Question Type and Question Order, F(1,128) = 1.27, p = .26. female).
Results confirm that a strong belief that the action will lead to the
outcome is not a necessary requirement for intentionality. According to 5.4.1.2. Stimuli and procedure. Participants were randomly assigned
our account, this is because some attitudes can be strong enough that (between-subjects) to read one of the following vignettes, adapted from
they are considered to have causal power even in the absence of a strong Knobe (2003a):
belief that the action will lead to the outcome. Consider for instance the
Shooter vignette. For most people, even a remote possibility that there “The vice-president of a company went to the chairman of the board
might be bullets left in the rifle is enough to prevent them from shooting and said, ‘We are thinking of starting a new program. It will help us
an unloaded rifle at someone. Someone who does so anyway demon­ increase profits, but it will also [harm/help] the environment.’ The
strates an abnormally low valuation of the target’s life. Because it is so chairman of the board answered, ‘I don’t care at all about the envi­
abnormal, this low valuation is readily selected as the cause of the vic­ ronment. I just want to make as much profit as I can. Let’s start the
tim’s death. Therefore the shooter intentionally killed the victim. new program.’ They started the new program. Sure enough, the
We also observe that overall, causation ratings were close to inten­ environment was [harmed/helped].”
tionality ratings. Study 3 was not explicitly designed to test this pre­
In both conditions, participants were asked a Causation question
diction (as causal judgments were always elicited last), so this result
first, and then an Intentionality question on a separate page. They were
should be interpreted with caution, but it is one additional piece of
asked to rate their agreement with the following statements, on a 1–9
evidence that causation is what matters for intentionality, even in the
likert scale (1: strongly disagree, 9: strongly agree):
realm of somewhat contrived thought experiments.
In the next study, we turn to what is arguably the most famous of
- The fact that the chairman does not care about the environment
these thought experiments.
caused the environment to be [harmed/helped].
- The chairman intentionally [harmed/helped] the environment.
5.4. Study 4: a side-effect effect for causality
5.4.2. Results
In section 4.2, we introduced Knobe’s (2003a) chairman vignette as As predicted, we found a side-effect effect for causation judgments:
an example of a case where an agent is judged to do X intentionally even

10
T. Quillien and T.C. German Cognition 214 (2021) 104806

causation ratings were higher in the Harm (M = 7.23, SD = 2.16) than in of how normal they are – where ‘normal’ has a broad meaning, encom­
the Help case (M = 4.42, SD = 2.92), t(180.55) = 7.69, p < .001, d = passing statistical, normative and functional considerations.
1.10. We also replicated the original side-effect: intentionality ratings As such, our account predicts that in many cases, norm violators (e.g.
were higher in the Harm (M = 7.26, SD = 2.09) than in the Help case (M agents who engage in immoral actions, violate a conventional norm,
= 3.06, SD = 2.61), t(187.04) = 12.5, p < .001, d = 1.79. See Fig. 6. behave in a different way than they usually do, etc) will be judged as
To compare the size of these effects, we conducted a 2*2 mixed more intentional than non-norm-violators. However, it also predicts the
Anova with Question Type (intentionality vs causation) and Condition existence of cases where this effect reverses: cases where norm violators
(Harm vs Help) as predictors. We found a significant interaction, F(1, are judged as less intentional than non-norm-violators.
196) = 15.87, p < .001, η2partial = .07, showing that the effect of Con­ Specifically, computational models of causal judgment (Icard et al.,
dition on ratings is higher for intentionality than causation. 2017; Quillien, 2020) predict the following interaction between
We also tested whether the effect of condition on intentionality normality and causal structure (see SI at https://osf.io/dp5xr/ for an
ratings was mediated by causation ratings. To do so, we conducted a informal explanation):
mediation analysis with 5000 resamples using the lavaan package in R
(Rosseel, 2012). We found a significant indirect effect of condition on - In situations that have a conjunctive causal structure (i.e. situations
intentionality ratings via causation ratings (95% CI [0.80, 1.93]; see where several factors are jointly necessary to bring about an
Supplementary Information at https://osf.io/dp5xr/ for the full model). outcome), abnormal events are judged more causal than normal
events. This predicted effect is known as abnormal inflation.
5.4.3. Discussion - In situations that have a disjunctive causal structure (i.e. situations
Results of Study 4 are consistent with a causalist account of the side- where several factors led to an outcome, but any one of them would
effect effect. On the other hand, these results are not completely sur­ have been sufficient), abnormal events are judged less causal than
prising: analogues of the side-effect effect have been found for many normal events. This predicted effect is known as abnormal deflation.
other types of judgments, such as judgments of whether an agent knows
something or is in favor of something (Beebe & Buckwalter, 2010; Pettit Correspondingly, we should observe the same interaction for inten­
& Knobe, 2009). As such, one could also have predicted the existence of tionality judgments. Study 5 was designed to test this prediction.
a causal side-effect effect on a completely atheoretical basis, simply by We asked participants to read a story (adapted from Icard et al.,
generalizing from these already known similar effects. 2017) where a committee must vote to approve or reject a request. We
Therefore, a challenge for our account is to show that the explanation manipulated the causal structure of the situation, such that in one
we give for the side-effect effect can also be used to make genuinely condition, all committee members must vote Yes for the request to be
novel predictions: predictions that would not follow from other accounts approved (conjunctive causal structure), and in the other condition, the
of the effect, or from simple generalization. We take on this challenge in request is approved if at least one committee member votes Yes
the next study. (disjunctive causal structure). We also manipulated whether the com­
mittee members violated a statistical norm, by giving background in­
5.5. Study 5: reversing the effect of norm violation on intentionality formation about what the committee members usually do. One member,
Mr. A, was described as almost always voting Yes, while the other
Many accounts of the side-effect effect rely on the idea that people member, Mr. B, was described as almost always voting No. Then we
attribute higher intentionality to agents who violate a norm (Nadelhoffer, described a vote where both members vote Yes and the request is
2006; Pettit & Knobe, 2009; Hindriks, 2014; Uttich & Lombrozo, 2010; approved: in this case, Mr. A is behaving normally with respect to his
Holton, 2010; Alicke, 2008). For instance, people may make such attri­ usual behavior, while Mr. B is violating a statistical norm: even though
butions because of a motivation to blame the norm violator (Alicke, 2008; he usually votes No, this time he is voting Yes.
Nadelhoffer, 2006), or because one can make stronger mental state in­ We predicted that in the conjunctive causal structure, Mr. B (the
ferences about someone who violates a norm (Uttich & Lombrozo, 2010). norm-violator) would be judged as more intentional than Mr. A, while
Our account also relies on the notion of norm violation, but makes the effect would reverse in the disjunctive causal structure. We also
more subtle predictions. It holds that norm violations have an effect on predicted that we would find the same pattern of effects for causation
intentionality judgments because they have an effect on causation judg­ judgments, conceptually replicating previous empirical findings (Icard
ments. Causation judgments are sensitive to norm violations because they et al., 2017; Gerstenberg & Icard, 2019; Morris et al., 2019; Kominsky &
rely on counterfactuals, and people sample counterfactuals as a function Phillips, 2019; Henne et al., 2019).
Finally, we included a question designed to test an alternative account
for the effect of norm violation on intentionality judgment. According to
Uttich and Lombrozo (2010), people attribute higher intentionality to
norm violators because norm violations allow stronger inferences about
an agent’s attitudes. Therefore, we asked participants which inferences
they thought could be made about the agents’ attitudes from their de­
cisions. We hoped to find a dissociation between causation and inference
judgments, which would permit a critical test between the two accounts.

5.5.1. Methods

5.5.1.1. Participants. We recruited 199 participants from Amazon

MechanicalTurk. Sixty-one participants who failed either a catch item or
a comprehension item (see below) were excluded from analysis, yielding
a final sample of 133 participants (71 female, 1 other).

5.5.1.2. Stimuli and procedure. We used a 2 (Causal structure) * 2

Fig. 6. Causation and intentionality ratings as a function of condition, Study 4. (Normality) mixed design, with Causal Structure manipulated Between-
Error bars represent standard errors of the mean. Individual data points are subjects, and Normality manipulated within-subjects.
jittered for better visibility.

11
T. Quillien and T.C. German Cognition 214 (2021) 104806

Participants were randomly assigned to read one of the following

vignettes:

At a local university, a committee is in charge of evaluating new

requests for funding from professors. The committee has two mem­
bers, Mr. A and Mr. B. In order for a request to be approved, it must
be that [both committee members vote/at least one committee
member votes] in favor of the request. While Mr. A almost always
votes Yes, Mr. B is notorious for almost always voting No.

Today, the committee is examining Professor Smith’s request for new

computers. Although neither committee member knows Professor
Smith, they both read her application carefully. Then, both com­
mittee members cast their vote at the same time. As usual, Mr. A
voted in favor of the request; surprisingly, Mr. B also voted in favor of
the request.

Since [both committee members/at least one committee member]

voted in favor of the request, Prof Smith gets funding for her new Fig. 8. Causation ratings for norm-conforming (Mr A) and norm-violating (Mr
computers. (adapted from Icard et al., 2017). B) agents, for conjunctive and disjunctive structure. Error bars represent stan­
dard errors of the mean. Individual data points are jittered for better visibility.
Participants were either asked two Intentionality questions (one for
each committee member) followed by two Causation questions, or two - In order for a request to be approved, how many committee members
Causation questions followed by two Intentionality questions. The order need to vote Yes? (One/Two/Thre/Impossible to tell).
in which committee members appeared in the questions was random­ - Both committee members usually reach the same decision most of
ized across participants but fixed within-participant. Participants were the time. (True/False/Impossible to tell).
asked how much they agreed with the following statements, on a 1–7
likert scale (1: strongly disagree, 7: strongly agree): Participants who failed either question were excluded from analysis.

- [Mr A /Mr. B] intentionally gave Professor Smith new computers. 5.5.2. Results
- The fact that [Mr A/Mr. B] wanted Professor Smith to get new For intentionality ratings, we found the predicted abnormal inflation
computers caused her to get new computers.14 effect in the conjunctive structure, as well as the predicted abnormal
- The fact that [Mr A/Mr. B] voted Yes tells us that it was important for deflation effect in the disjunctive structure. The cross-over interaction
him that Professor Smith get new computers. was also statistically significant (see Fig. 7).
In the conjunctive structure, intentionality ratings were higher for
Additionally, participants were asked the following two compre­ the norm-violating agent (M = 5.67, SD = 1.67) than for the norm-
hension questions, on the same page just below the intentionality conforming agent (M = 5.22, SD = 1.94), t(71) = 3.01, p = .004, dz
question: = .35. By contrast, in the disjunctive structure, intentionality ratings
were lower for the norm-violating agent (M = 4.83, SD = 2.00) than for
the norm-conforming agent (M = 5.39, SD = 1.61), t(58) = − 2.41, p =
.02, dz = − .31.
A 2*2 mixed Anova showed a significant interaction between Agent
and Causal Structure on intentionality ratings, F(1, 129) = 14.22, p <
.001, η2partial = .10.
We also replicated the known pattern of results for causation judg­
ments (see Fig. 8). In the conjunctive causal structure, causation ratings
were higher for the norm-violating agent (M = 5.57, SD = 1.32) than for
the norm-conforming agent (M = 4.58, SD = 1.53), t(71) = − 5.90, p <
.001, dz = .69. By contrast, in the disjunctive causal structure, causation
ratings were lower for the norm-violating agent (M = 4.20, SD = 1.88)
than for the norm-conforming agent (M = 4.95, SD = 1.74), t(58) =
− 2.71, p = .009, dz = − .35.
A 2*2 mixed Anova showed a significant interaction between Agent
and Causal Structure on causation ratings, F(1,129) = 31.20, p < .001,
η2partial = .19.
Given the similarity in the pattern of results between causation and
intentionality ratings, it seems likely that intentionality judgments are
Fig. 7. Intentionality ratings for norm-conforming (Mr A) and norm-violating
shaped by normality considerations because normality influences
(Mr B) agents, for conjunctive and disjunctive structure. Error bars represent
causation judgments. But we wanted to test a possible alternative
standard errors of the mean. Individual data points are jittered for bet­
ter visibility.
interpretation for the cross-over interaction depicted in Fig. 7. In the
disjunctive structure, since Mr. A is known to always vote Yes, and it
takes only one committee member for a request to be approved, par­
ticipants may reason that Mr. B knows that he cannot change the
outcome of the vote. Therefore, they may suspect that Mr. B has little
14
incentive to care, and has decided to vote randomly. If participants
For exploratory purposes, half the participants were asked the following
reason in this way, then they will think that in the disjunctive causal
causation question instead: “Professor Smith got new computers because Mr. A
structure, Mr. B’s vote does not tell us much about how much he values
wanted her to get new computers”.

12
T. Quillien and T.C. German Cognition 214 (2021) 104806

Prof Smith’s getting new computers. If intentionality judgments are the potential to cause a professor to get funding. We asked half the
driven by inferences about attitudes (Uttich & Lombrozo, 2010), then participants to rate whether the vote of each committee member caused
this line of reasoning will lead participants to give lower intentionality Professor Smith to get new computers, and the other half to rate whether
ratings to Mr. B in the disjunctive structure compared to the conjunctive the committee member gave new computers to Professor Smith. We
structure, which could explain our results. found that while participants gave high causation ratings (their mean is
The results for the inference ratings (“The fact that [Mr A/Mr B] voted above 5 on a 1–7 scale), they were reluctant to agree that the committee
Yes tells us that it was important for him that Professor Smith get new member gave computers to Professor Smith (mean ratings below 3).
computers.”) are not consistent with this interpretation (see Fig. 9). Par­ Additionally, we observed an abnormal inflation effect for causation
ticipants consistently judged that we learn more about the norm violator’s ratings, but not for ratings of giving. These results suggest that partici­
attitude than about the other agent’s attitude, and this effect was of the pants in study 5 (which used almost the same scenario as study 5B) did
same size in both causal structures. In a 2*2 mixed Anova with Causal not interpret “the agent gave computers to Professor Smith” as simply
Structure and Agent as predictors, and inference ratings as outcome meaning “the fact that the agent voted Yes caused Professor Smith to get
variable, there was a main effect of Agent, F(1, 129) = 87.12, p < .001, new computers”.
such that participants gave higher inference ratings for the norm-violating Therefore, the specific pattern of effects of normality on intention­
agent (M = 6.07, SD = 1.27) compared to the norm-conforming agent (M ality ratings found in study 5 probably indicates that people computed
= 4.49, SD = 1.80). There was also a main effect of Causal Structure, F(1, how much the agent’s attitude toward the outcome caused the outcome.
129) = 5.09, p = .03, such that participants gave higher inference ratings
in the conjunctive (M = 5.49, SD = 1.59) than the disjunctive causal 5.5.4. Discussion
structure (M = 5.02, SD = 1.89). However, there was no interaction be­ Results of Study 5 provide evidence in favor of a novel prediction of
tween Agent and Causal Structure, F(1,129) = 0.95, p = .33. our account: the influence of normality on intentionality judgments de­
We also conducted a series of Anovas to check for order and wording pends on the causal structure of the situation. In a conjunctive structure,
effects. Most tests were negative, with the following exceptions. There was we found that participants judged an agent violating a statistical norm as
a 2 × 2 interaction between the Wording of the causation question and the more intentional than a norm-conforming agent. In a disjunctive causal
identity of the Agent on the causation ratings, F(1, 127) = 6.33, p = .01; structure, this effect was reversed: the norm-conforming agent was judged
such that, averaging across causal structures, the norm-violating agent was as more intentional than the norm-violating agent. These results are
rated as more causal than the norm-conforming agent, but only for the difficult to explain on any account that predicts a general bias to consider
“Caused” wording of the causation question. There was also a 2x2x2 norm violations as more intentional (e.g. Alicke & Rose, 2010; Hindriks,
interaction between the Order of presentation of agents (Mr A vs Mr. B 2014; Uttich & Lombrozo, 2010).
first), the Causal Structure, and the identity of the Agent, on the causation
ratings, F(1,127) = 4.60, p = .03; such that in the disjunctive causal
structure, the norm-violating agent was seen as less causal than the norm- 5.6. Study 6: intentionality requires a domain-specific causal pathway
conforming agent only when the norm-violating agent was presented last;
see Supplementary Information (https://osf.io/dp5xr/) for details. In the studies described so far, judgments of intentionality closely
track judgments of causation. As a reviewer observed, our framework
5.5.3. Study 5B also predicts that intentionality and causation judgments will sometimes
According to our account, intentionality judgments are computed diverge from each other. Specifically, if an agent’s attitude toward an
from a mental representation of the strength with which the agent’s outcome causes that outcome in a way that deviates from the causal
attitude toward the outcome caused the outcome. The abnormal infla­ model implicit in commonsense psychology, people will tend to deny
tion and abnormal deflation effects we found for intentionality judg­ that the outcome was brought about intentionally, even when they judge
ments suggest that, when answering the intentionality question, people that it was caused by the agent’s attitude toward that outcome.
computed how much the agent’s attitude toward the outcome caused We test this prediction in study 6. We designed a vignette about a
the outcome. futuristic corporation whose employees have brain implants. The AC in
But maybe people were engaging in another kind of causal strength the building is regulated as a function of how the employees feel about
computation, in which mentalizing plays no role. Our intentionality the temperature, as recorded by their brain implants. In one version of
question asked participants how much they agreed that the agent the vignette, the causal link between the agents’ attitude and the
intentionally gave computers to Professor Smith; therefore, some of the outcome conforms to the causal model in commonsense psychology (the
variation in intentionality ratings might be due to variation in agree­ employee needs to make a decision in order for the AC to be turned on).
ment that the agent gave computers to Professor Smith. Furthermore, In the other vignette, the causal link is deviant, in the same way as the
people may have been interpreting “the agent gave computers to Pro­ causal link in the King example from section 3.1. Specifically, the brain
fessor Smith” as “the fact that the agent voted Yes caused Professor implant automatically detects and implements the employee’s desires.
Smith to get new computers”, and where “the fact that the agent voted We predict that in the normal link condition, judgments of whether
Yes” was construed in a purely mechanical sense. Under these two as­ an agent intentionally turned on the AC will be relatively close to
sumptions, one would predict abnormal inflation and abnormal defla­ judgments of whether the AC was turned on because the agent wanted it
tion effects for the intentionality question, even if people were not to. By contrast, in the deviant link condition, judgments of intentionality
computing the causal dependence between the outcome and the agent’s will be lower than judgments of causation. In addition, we also manip­
attitude toward the outcome. ulated the normality of the agents’ attitudes, in an attempt to replicate
This interpretation is a priori unlikely, because researchers working the abnormal deflation effect found in study 5.
on the semantics of “giving” have suggested that intentionality is itself
an important part of the concept (Newman, 1996). Nonetheless, we 5.6.1. Methods
wanted to make sure that in the scenario we used in study 5, people did
not interpret “the agent gave computers to Professor Smith” as meaning 5.6.1.1. Participants. Our main prediction is an interaction between
simply “the fact that the agent voted Yes caused Professor Smith to get Question Type (causation vs intentionality, within-subject) and causal
new computers”. link (normal vs deviant, between-subject). Although we expected a large
To that end, we conducted study 5B (see supplementary information effect, we conservatively set our intended sample size (228 participants)
at https://osf.io/dp5xr/), where we used a similar scenario as in study 5, so that we would be able to detect a small effect (η2partial = .02) with
except that the committee members could not know that their vote had 99% power. Anticipating an exclusion rate similar to study 5 (33%), we

13
T. Quillien and T.C. German Cognition 214 (2021) 104806

For every room in the building, the rule is that the computer turns on
the AC in that room if it detects that at least one person in the room
would like the room to be cooler.

In room 42, there are two employees, Mr. A and Mr. B. While Mr. A is
very sensitive to heat, Mr. B usually doesn’t really care. Today is a
very hot day, and the system accurately detects that both employees
[would be in favor of turning on the AC if they were asked/made a
request in their head to turn on the AC]. Since at least one employee
in the room would like the room to be cooler, the system turns on the
AC, and the room gets cooler.”

Participants were then asked two Causation questions (one for each
employee) and two Intentionality questions, on separate pages. Half the
participants saw the two Causation questions first, followed by the two
Intentionality questions; this order was reversed for the other half. The
order in which the employees appeared in the questions was randomized
across participants but fixed within-participant. Participants were asked
Fig. 9. Inference ratings for norm-conforming (Mr A) and norm-violating (Mr
B) agents, for conjunctive and disjunctive structure. Error bars represent stan­ how much they agreed with the following statements, on a 1–7 likert
dard errors of the mean. Individual data points are jittered for better visibility. scale (1: strongly disagree, 7: strongly agree):

- [Mr A/Mr. B] intentionally made the room cooler

set our recruitment target at 350 participants.
- The room got cooler because [Mr A/ Mr. B] wanted the room to be
We recruited 349 US residents from Mechanical Turk. We excluded
cooler
from analysis 86 participants failing either a catch item (N = 3), or either
of three comprehension questions (N = 18, 29, 53), leaving a final
Additionally, participants were asked the following three compre­
sample of 263 participants (139 female, 1 unspecified).
hension questions, on the same page just below the first question
5.6.1.2. Design. We manipulated Causal Link between-subjects. Half
- In order for the AC to be activated, how many people need to be in
the participants read a story in which the computer can automatically
favor of making the room cooler? (0,1,2,3)
detect employee’s attitudes toward making the room cooler (deviant link
- Both employees are equally sensitive to heat (True/False/Impossible
condition). The other half of participants read a story in which employees
to tell)
need to formulate a request in their head in order to communicate their
- The computer cannot automatically detect what employees want:
desire to make the room cooler (normal link condition).
they need to formulate a request in their head (True/False)
We also manipulated the normality of agents’ attitudes, in a similar
manner as in study 5. All participants read about two employees, Mr. A
Participants who failed any of these questions were excluded from
and Mr. B. Mr. A was described as very sensitive to heat: thus, his desire
analysis.
to make the room cooler is statistically normal. By contrast, Mr. B was
described as usually indifferent to the room temperature, thus his cur­
5.6.2. Results
rent desire to make the room cooler is statistically abnormal.
We first look at whether our main prediction is supported. Then we
The story we used describes a disjunctive causal structure: the
turn to the effect of normality.
computer activates the AC if it detects that at least one agent would like
In the normal link condition, participants’ mean causation and
to make the room cooler. In this kind of setting, counterfactual models of
intentionality ratings were almost identical. By contrast, and as pre­
causal judgment predict that the normal agent (here, Mr. A) should be
dicted, in the deviant link condition intentionality ratings were much
viewed as more causal than the abnormal agent (Mr B).
lower than causation ratings (see Fig. 10).
For each agent, participants were asked a Causation question
We analyzed the data with a 2 (Causal link) * 2 (Question) * 2
(probing whether the agent’s attitude caused the room to be cooler), and
(Agent) mixed Anova.
an Intentionality question (probing whether the agent intentionally
There was a main effect of Causal link, F(1, 261) = 79.9, p < .001, such
made the room cooler).
that participants in the Normal link condition gave higher ratings (M =
5.24, SD = 1.92) than participants in the Deviant link condition (M = 3.90,
5.6.1.3. Stimuli and procedure. Participants read the following vignette:
SD = 1.92). There was also a main effect of Question, F(1, 783) = 58.6, p <
“In the FutureCorp building, employees have a chip inside their head .001, such that participants gave higher ratings of causation (M = 4.93, SD
that can read their brain activity. [On the basis of your brain activity, = 1.93) than intentionality (M = 4.23, SD = 2.22). Crucially, this effect
the chip can predict how you would respond to the question “would was much higher in the Deviant link condition (causation: M = 4.64,
you like the room to be cooler?” if someone were to ask you/If you intentionality: M = 3.15) than in the Normal link condition (causation: M
feel that it is too hot, you can request the AC to be turned on by = 5.22, intentionality: M = 5.26), as shown by an interaction between
thinking in your head “I would like the room to be cooler”]. When­ Causal link and Question, F(1, 783) = 68.8, p < .001, η2partial = .081.
ever the chip detects that you would like the room to be cooler, it We also conceptually replicate the abnormal deflation effect found in
sends this information to the computer that regulates the room’s study 5. There was a main effect of Agent, F(1, 783) = 261.7, p < .001,
temperature. Because the chip is perfectly accurate, [the computer such that participants gave higher ratings for the norm-conforming
can determine whether you want the room to be cooler without agent, Mr. A (M = 5.33, SD = 1.77) than the norm-violating agent,
actually asking you/the computer can determine whether you want Mr. B (M = 3.83, SD = 2.15). This abnormal deflation effect held for
the room to be cooler if you formulate the request in your head]. both intentionality and causation ratings. Intentionality ratings for the
Employees know how the system works. norm-conforming agent, Mr. A (M = 4.90, SD = 2.05) were higher than
for the norm-violating agent, Mr. B (M = 3.55, SD = 2.17), t(262) =
10.72, p < .001, dz. = .66. The norm-conforming agent was also judged

14
T. Quillien and T.C. German Cognition 214 (2021) 104806

judgments. One possible interpretation is that deviance from the causal

model of commonsense psychology attenuates intentionality judgments,
but does not completely disconnect them from computations of causal
strength. We leave it to future research to shed more light on this finding.

6. General discussion

What does it mean to do something intentionally? We have argued

that, to the human mind, the concept is defined in the following way: “an
agent did X intentionally if the agent’s attitude toward X caused X, and
caused X according to the typical causal model implicit in our
commonsense psychology”.
Philosophers and cognitive scientists have found it difficult to define
intentional action, because people’s intuitions about the use of the
concept are very complex, such that it is easy to find counterexamples to
a given definition. We suggested that this complex pattern of intuitions
can be explained by taking a closer look at the building blocks of the
Fig. 10. Causation and intentionality ratings for the deviant and normal causal
concept.
links. Colored points represent average judgment, averaging across agents (e.g.
a green point represents the average causation rating for Mr. A and Mr. B
According to our proposal, these building blocks include the causal
combined). Error bars represent standard errors of the mean. Individual data model of commonsense psychology, and the intuitive concept of
points are jittered for better visibility. (For interpretation of the references to causation. By incorporating recent insights from cognitive science
colour in this figure legend, the reader is referred to the web version of models of these building blocks, our account can (i) provide a unifying
this article.) explanation for a variety of known patterns in human intuitions about
intentional action, such as the side-effect effect; and (ii) make successful
as more causal (M = 5.75, SD = 1.31) than the norm-violating agent (M novel predictions.
= 4.11, SD = 2.10), t(262) = 11.6, p < .001, dz. = .72. Interestingly, Below we discuss limitations of our account, address alternative
abnormal deflation was equally strong in the deviant link and the explanations for our data, and explore implications for the learnability
normal link conditions, for both intentionality and causation ratings: of Theory of Mind concepts.
there was no significant interaction between Causal link and Agent on
intentionality ratings, F(1, 261) = .287, p = .593, or on causation rat­ 6.1. Limitations
ings, F(1, 261) = 2.235, p = .136.
We also ran an Anova with Causal link, Agent, Question type, Order On our account, a full understanding of the folk concept of inten­
of questions (causation or intentionality first), and Order of agents (Mr A tional action will have to rely on a full understanding of its building
or Mr. B first) as predictors, and found no significant interactions blocks, such as mechanisms for causal judgment. Current models of
involving the order in which the questions were asked. causal cognition have great explanatory and predictive power, but are
In an exploratory analysis, we also found that the participants giving still incomplete; as a result, one may not currently be able to provide a
the highest causation ratings also tended to give the highest intention­ fully mechanistic explanation for every intuition that people have.
ality ratings, but that this correlation was higher in the normal Consider for instance a case where Jake shoots at his aunt from a
compared to the deviant link condition. Across participants, the corre­ great distance with the intention of killing her, and manages to reach his
lation between intentionality and causation judgments for Mr. A was target by sheer luck, despite being a poor marksman. Most people
descriptively higher15 in the normal link (r(132) = .63) than in the intuitively judge that Jake intentionally killed his aunt (Knobe, 2003b).
deviant link condition (r(127) = .44), p = .06. The correlation between This result makes sense in our framework, since intuitively the aunt died
intentionality and causation judgments for Mr. B was significantly because Jake wanted to kill her. Yet to have a fully mechanistic expla­
higher in the normal link (r(132) = .74) than in the deviant link con­ nation, we would need to explain where this causal intuition comes
dition (r(127) = .47), p = .005. from. Computational models of causal judgment do not make a clear
prediction in this case. On the one hand, Jake’s murderous desire is
5.6.3. Discussion clearly morally abnormal, so we should expect that people will judge it
We find that judgments of intentionality diverge from judgments of to be causal (because people will readily generate counterfactuals where
causation when the causal link between an agent’s attitude and an Jake does not have the desire to kill his aunt, and in these counterfac­
outcome goes outside of the typical causal model of commonsense tuals his aunt does not dies). On the other hand, the shot reached Jake’s
psychology. In a situation where a machine can automatically detect and aunt by a stroke of luck: it is easy to entertain counterfactuals where
implement an agent’s desire, participants were much more willing to Jake shoots and misses his target; this will make people less likely to
agree that the agent’s attitude toward the outcome caused the outcome, think of his murderous desire as causal.
than to agree that the agent acted intentionally. Therefore we have two variables (the agent’s abnormal morality, and
On the other hand, there was one way that judgments of intentionality his lack of control over the outcome), which theoretically should pull
were similar to judgments of causation. Participants tended to attribute causation judgments in opposite directions, but current models of causal
lower intentionality to the norm-violating agent than the norm- judgment do not make clear predictions about which variable will have
conforming agent, even when the causal link between attitude and a stronger effect. In order to explain why it feels intuitive to consider
outcome was deviant. This pattern reflected the one found for causation Jake’s murderous desire as a strong cause of his aunt’s death, we must
make ad-hoc assumptions, for instance, the assumption that in this case
people are more likely to generate counterfactuals as a function of moral
15 normality. Hopefully, as cognitive scientists develop increasingly more
To compare the strength of the two correlations, we first z-scored the
intentionality and causation scores within each Causal link condition, and then accurate models of causal cognition, our account will be able to make
computed the interaction between causation scores and causal link in a multiple more fine-grained, mechanistic predictions.
regression predicting intentionality scores. The p-value we report is the p-value
for that interaction.

15
T. Quillien and T.C. German Cognition 214 (2021) 104806

6.2. Alternative explanations causation judgments were computed from intentionality judgments.
Therefore, ‘causation first’ is the account that best explains our data:
Many of the empirical tests we have presented here involved intentionality judgments track causation judgments because they are
showing that intentionality judgments and causation judgments track computed from a mental representation of causation.
each other. Our interpretation of these results is that intentionality
judgments are computed from a mental representation of the causal 6.3. Minimality and learnability
dependence between an outcome and the agent’s attitude toward this
outcome. Are there plausible alternative explanations? Intuitively, a strong appeal of our account is its minimality. Having
Maybe the causal arrow runs in the reverse direction: causal judg­ said that, we are not arguing that it would be simple to teach the meaning
ments are computed from a mental representation of intentionality. For of “intentionally” to a machine. Indeed, our account predicts exactly the
instance, our questions about causation may have sounded unnatural contrary: it would be very difficult to teach the concept to an artificial
and confusing, and therefore participants defaulted to interpreting them intelligence with a “blank-slate" architecture. Our account is very simple
as questions about intentionality. because it assumes that the concept of intentional action is constructed in
Maybe a third variable explains why causation and intentionality a relatively straightforward way from pre-existing building blocks. These
judgments track each other. Notably, researchers have found that building blocks include complex cognitive mechanisms for reasoning
judgments about intentional action and judgments about causation are about the minds of others and for making causal attributions.
both influenced by the degree to which people regard certain counter­ In other words, we do not aim to provide a comprehensive and
factuals as relevant when asked to consider how things could have gone transparent definition of the folk concept of intentional action, that
differently (Phillips, Luguri, & Knobe, 2015). This effect of counterfac­ could be hard-wired into a computer devoid of any other specialized
tual relevance seems quite wide-ranging, since Phillips et al. (2015) also knowledge. Rather, we are trying to decipher the recipe by which the
found that it has an impact on people’s judgments of freedom and reliably-developing human brain acquires the concept. Thus, one
judgments about the doing/allowing distinction. Therefore, one natural appealing feature of our theory is that it suggests a solution to a learn­
interpretation is that intentionality and causation judgments track each ability problem (Jackendoff, 1989; Pinker, 1989) that we think has been
other simply because they are both independently impacted by judg­ neglected in existing debates about the meaning of the concept. Existing
ments of counterfactual relevance. accounts of “intentionally” can be quite complex, for instance requiring
We see at least three reasons to favor our “causation first” account at least 5 necessary components for something to intentional (e.g. Malle
over these two alternatives. First, even if our minimalist account turned & Knobe, 1997), or positing that people attribute different meanings to
out to be incomplete, it is almost undeniable that causation has to be a the word, with the relevant meaning being determined by the context (e.
central component of intentional action. We challenge the reader to find g. Cova et al., 2012; Cushman & Mele, 2008). How do children manage
an example of a case where an agent intentionally does X, yet there is to acquire such a complex concept from the linguistic stimuli they are
absolutely no causal connection between X’s attitude toward X and X exposed to? And why do they acquire this very concept (or set of con­
happening. By contrast, it is equally obvious that causation does not cepts) as opposed to any other? As far as we know, most existing theories
require intentionality – people spontaneously attribute causation in sit­ are silent about these questions.
uations devoid of any mental states, such as physical collisions between By contrast, if, as we suggest, the folk concept of intentional action is
billiard balls, and the mechanism for this inference appears to be present built in a relatively simple way from a set of pre-existing building blocks,
in infants as young as 6 months of age (Gerstenberg & Icard, 2019; Leslie then we can start to sketch an account of how children acquire the
& Keeble, 1987; Michotte, 1963). concept. Here is one suggestion. Assume that children already possess a
Second, the results of study 5 are difficult to interpret on the alter­ set of Theory of Mind mechanisms, an intuitive concept of causation, and
native explanations sketched above. On an ‘intentionality first’ account, that they are predisposed to infer that, within the psychological domain,
there are no a priori reasons why normality considerations would interact words that refer to a link between two entities refer to a causal link.16
with causal structure in shaping intentionality judgments (to our knowl­ Therefore, when they come to understand that intentionality refers to
edge, no existing account of intentional action predicts this interaction some kind of relationship between an agent’s attitude toward X and the
effect). On a “third variable" account involving counterfactual relevance, occurrence of X, they spontaneously assume that an agent does X inten­
there is no a priori reason to expect the interaction effect either. Indeed, tionally if his attitude toward X caused X.
preliminary evidence suggests that people make the same judgments of
counterfactual relevance in disjunctive and conjunctive cases: in both 7. Conclusion
causal structures, they tend to view the norm-violating event as more
relevant (Kominsky & Phillips, 2019, experiment 2). Therefore, in order People can use the word “intentionally” in very strange ways. Our
to explain the interaction effect, we must appeal to a richer explanatory intuitions about whether something is intentional are swayed by moral
framework. Computational models of causal judgments such as Icard et al. considerations, are pulled one way or another depending on the amount
(2017) and Quillien (2020) provide exactly such a framework: they pre­ of control an agent exerts, and are influenced by how circuitous the
dict such an interaction effect despite assuming that people generate causal chain between the agent and the outcome is. Intentionality re­
counterfactuals in the same way in both kinds of causal structure. quires a relevant belief, but the latter can be present in very small doses.
Third, the results of study 6 show that causation and intentionality Norm-violating actions are judged as more intentional than norm-
judgments do not always track each other. We constructed a situation conforming actions – except when they are judged as less intentional.
where an agent’s attitude causes an outcome in a way that deviates from These seemingly erratic intuitions can be anxiety-inducing. One
the causal model of commonsense psychology, but otherwise satisfies might conclude that our commonsense psychology is fundamentally
the intuitive, domain-general concept of causation. As predicted by our moralistic; that linguistic meaning is hopelessly entangled in its context;
account, people were reluctant to judge that the agent intentionally or that motivational and pragmatic factors constantly warp our in­
brought about the outcome, but tended to agree that the agent’s attitude tuitions about the proper extension of words.
caused the outcome. This pattern of intuitions would be unlikely if We think such anxiety might be misplaced. Instead, we view the

16
A predisposition to assume that mentalizing concepts involve causal links also makes sense of the fact that concepts such as ‘perceiving’, ‘remembering’ and
‘knowing’ seem to have a causal component (see Grice, 1961; Martin & Deutscher, 1966; Goldman, 1967. For instance, cross-cultural evidence suggests that people
everywhere think that justified true belief in p that is not caused by p does not constitute knowledge, Machery et al., 2017).

16
T. Quillien and T.C. German Cognition 214 (2021) 104806

strangeness of “intentionally” as emerging naturally from the core Acknowledgements

structure of the concept. The way people use the concept of intentional
action offers a fascinating window on some of the building blocks that For discussion and / or comments on earlier drafts of this paper we
make up human thought: it lets us glimpse into our implicit causal model thank Pascal Boyer, Leda Cosmides, Dan Korman, Zoe Liberman, Denis
of the mind, and the algorithms with which we assign causes to events. Tatone, three anonymous reviewers, and Ori Friedman. We are also
grateful to audiences at the Center for Evolutionary Psychology (UCSB),
Author statement the Cognitive Development Laboratory (UCSB), the Oklahoma Center
for Evolutionary Analysis (Oklahoma State University) and the Causality
T.Q: Conceptualization, data curation, Writing - original draft in Cognition Lab (Stanford University).
T.G.: Conceptualization, Resources, Writing - review & editing

Appendix A. Appendix

A.1. Statements used in Study 1

Anne was sweating.

Anne was yawning during the lecture.
Anne was grinding her teeth during the test.
Anne had a craving for cherries after dinner.
Anne believed that she had the flu.
Anne was in a great mood today.
Anne was infatuated with Ben.
Anne was worrying about the test results.
Anne got admitted to Princeton.
Anne interrupted her mother.
Anne ignored Greg’s arguments.
Anne drove way above the speed limit.
Anne applauded the musicians.
Anne greeted her uncle politely.
Anne refused the salesman’s offer.
Anne stole a pound of peaches.
Anne asked Mike out for dinner.
Anne invited Sue to have lunch with her.
Anne watered her new plants.

A.2. Additional vignettes used in Study 3

Shooter. Bob wants to kill Alice. After weeks of careful study of her daily routine, he finally has her in the line of sight of his rifle while she is
walking in a quiet area of town. As he steadies his aim, he suddenly realizes that, in his excitement at the thought of carrying out his plan, he forgot to
put bullets in his rifle before leaving his house that morning. Although he is now convinced that shooting will have no effect, he decides to pull the
trigger anyway to release his anger.
Unbeknownst to him, a few bullets had been actually left in the rifle from one of his training sessions a few days before. His shot is perfectly
accurate, and sends a bullet right through Alice’s heart. To Bob’s surprise, Alice dies instantly.
Belief question: “By pulling the trigger, Bob believed that he would
kill Alice.”

Intentionality question: “By pulling the trigger, Bob intentionally

killed Alice".

Causation question: “Alice died because Bob wanted to kill her”.

Comprehension question: “The bullets in the rifle were put there in

the morning of the event” (True/False/Impossible to tell).

Sabotage. John is a worker at a power plant. In the control room of the power plant, there is a red button that engineers can push to cause the main
reactor to shut down. As an extra safety feature, in addition to pressing the red button, shutting the reactor also requires a special key card. One day,
John sees that there is nobody in the control room. He hates his boss and wants to make him look incompetent, so he decides to sneak in and press the
red button.
He is aware that he doesn’t have the special key card, and that therefore pressing the red button will not trigger the reactor shutdown. Yet John
really wants to shut down the main reactor, so he presses the red button anyway.
As it happens, because of an oversight on the part of the engineers, the safety feature was not yet in place. To John’s surprise, the main reactor shuts
down, creating considerable confusion.
Belief question: “John believed that he would shut down the main Intentionality question: “John intentionally shut down the main
reactor”. reactor”.

17
T. Quillien and T.C. German Cognition 214 (2021) 104806

Causation question: “The reactor shut down because John wanted to

shut down the reactor”.

Comprehension question: “There were a few engineers in the control

room” (True/False/Impossible to tell).

Appendix B. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.cognition.2021.104806.

References Hall, N. (2004). Two concepts of causation. In J. Collins, N. Hall, & L. A. Paul (Eds.),
Causation and counterfactuals. Cambridge, MA: MIT Press.
Halpern, J. (2016). Actual causality. MIT Press.
Adams, F., & Steadman, A. (2004). Intentional action in ordinary language: Core concept
Halpern, J. Y., & Pearl, J. (2005). Causes and explanations: A structural-model approach.
or pragmatic understanding? Analysis, 64, 173–181.
Part I: Causes. The British Journal for the Philosophy of Science, 56(4), 843–887.
Alicke, M. (2008). Blaming badly. Journal of Cognition and Culture, 8(1–2), 179–186.
Hart, H. L. A., & Honoré, T. (1985). Causation in the law (OUP Oxford).
Alicke, M., & Rose, D. (2010). Culpable control or moral concepts? Behavioral and Brain
Heider, F. (1958). The psychology of interpersonal relations. Psychology Press.
Sciences, 33(4), 330.
Henne, P., Niemi, L., Pinillos, A., De Brigard, F., & Knobe, J. (2019). A counterfactual
Alicke, M. D., Rose, D., & Bloom, D. (2011). Causation, norm violation, and culpable
explanation for the action effect in causal judgment. Cognition, 190, 157–164.
control. The Journal of Philosophy, 108(12), 670–696.
Hindriks, F. (2014). Normativity in action: How to explain the Knobe effect and its
Anscombe, G. E. M. (1957). Intention. Harvard University Press.
relatives. Mind & Language, 29, 51–72.
Apperly, I. A., & Butterfill, S. A. (2009). Do humans have two systems to track beliefs and
Hitchcock, C. (2001). The intransitivity of causation revealed in equations and graphs.
belief-like states? Psychological Review, 116(4), 953.
The Journal of Philosophy, 98(6), 273–299.
Aristotle (330BC/2009). (2009). The Nicomachean ethics. Oxford: Oxford University
Hitchcock, C., & Knobe, J. (2009). Cause and norm. The Journal of Philosophy, 106(11),
Press.
587–612.
Baillargeon, R., Scott, R. M., & Bian, L. (2016). Psychological reasoning in infancy.
Hume, D. (1740/1978). In L. A. Selby-Bigge (Ed.), Treatise of human nature (2nd ed.).
Annual Review of Psychology, 67, 159–186.
Oxford University Press.
Baker, C. L., Jara-Ettinger, J., Saxe, R., & Tenenbaum, J. B. (2017). Rational quantitative
Icard, T. F., Kominsky, J. F., & Knobe, J. (2017). Normality and actual causal strength.
attribution of beliefs, desires and percepts in human mentalizing. Nature Human
Cognition, 161, 80–93.
Behaviour, 1(4), 0064.
Jackendoff, R. (1989). What is a concept, that a person may grasp it? Mind & Language, 4
Baker, C. L., Saxe, R., & Tenenbaum, J. B. (2009). Action understanding as inverse
(1–2), 68–102.
planning. Cognition, 113(3), 329–349.
Jara-Ettinger, J., Gweon, H., Schulz, L. E., & Tenenbaum, J. B. (2016). The naïve utility
Bear, A., Bensinger, S., Jara-Ettinger, J., Knobe, J., & Cushman, F. (2020). What comes to
calculus: Computational principles underlying commonsense psychology. Trends in
mind? Cognition, 194, 104057.
Cognitive Sciences, 20(8), 589–604.
Bear, A., & Knobe, J. (2017). Normality: Part descriptive, part prescriptive. Cognition,
Jara-Ettinger, J., Schulz, L. E., & Tenenbaum, J. B. (2020). The naive utility calculus as a
167, 25–37.
unified, quantitative framework for action understanding. Cognitive Psychology, 123,
Beebe, J. R., & Buckwalter, W. (2010). The epistemic side-effect effect. Mind & Language,
101334.
25(4), 474–498.
Jones, E. E., & Davis, K. E. (1965). From acts to dispositions: The attribution process in
Bennett, D. (1965). Action, reason and purpose. Journal of Philosophy, 62, 85–95.
person perception. In L. Berkowitz (Ed.), Vol. 2. Advances in experimental social
Brand, M. (1984). Intending and acting: Toward a naturalized action theory. Cambridge,
psychology (pp. 371–388). Hillsdale, NJ: Erlbaum.
MA: MIT Press.
Kahneman, D., & Miller, D. T. (1986). Norm theory: Comparing reality to its alternatives.
Bratman, M. (1984). Two faces of intention. The Philosophical Review, 93(3), 375–405.
Psychological Review, 93(2), 136.
Bratman, M. E. (1987). Intention, plans, and practical reason. Cambridge, MA: Harvard
Knobe, J. (2003a). Intentional action and side effects in ordinary language. Analysis, 63
Univ. Press.
(279), 190–194.
Byrne, R. M. (2016). Counterfactual thought. Annual Review of Psychology, 67, 135–157.
Knobe, J. (2003b). Intentional action in folk psychology: An experimental investigation.
Cosmides, L. (1985). Deduction or Darwinian algorithms? An explanation of the “elusive”
Philosophical Psychology, 16(2), 309–324.
content effect on the Wason selection task (Unpublished doctoral dissertation). Harvard
Knobe, J. (2010). Person as scientist, person as moralist. Behavioral and Brain Sciences, 33
University.
(4), 315–329.
Cova, F. (2016). The folk concept of intentional action: Empirical approaches. In
Kominsky, J. F., & Phillips, J. (2019). Immoral professors and malfunctioning tools:
W. Buckwalter, & J. Sytsma (Eds.), Blackwell companion to experimental philosophy.
Counterfactual relevance accounts explain the effect of norm violations on causal
Wiley-Blackwell.
selection. Cognitive Science, 43(11), Article e12792.
Cova, F., Dupoux, E., & Jacob, P. (2012). On doing things intentionally. Mind & Language,
Kominsky, J. F., Phillips, J., Gerstenberg, T., Lagnado, D., & Knobe, J. (2015). Causal
27(4), 378–409.
superseding. Cognition, 137, 196–209.
Cushman, F. (2015). Deconstructing intent to reconstruct morality. Current Opinion in
Leslie, A. M. (1994). ToMM, ToBy, and agency: Core architecture and domain specificity.
Psychology, 6, 97–103.
In Mapping the mind: Domain specificity in cognition and culture (pp. 119–148).
Cushman, F., & Mele, A. (2008). Intentional action: Two-and-a-half concepts?
Leslie, A. M., Friedman, O., & German, T. P. (2004). Core mechanisms in “theory of
Experimental Philosophy, 171.
mind”. Trends in Cognitive Sciences, 8(12), 528–533.
Davidson, D. (1963). Actions, reasons, and causes. The Journal of Philosophy, 60(23),
Leslie, A. M., & Keeble, S. (1987). Do six-month-old infants perceive causality? Cognition,
685–700.
25(3), 265–288.
Davidson, D. (1980). Essays on actions and events. Oxford University Press.
Leslie, A. M., Knobe, J., & Cohen, A. (2006). Acting intentionally and the side-effect
Dennett, D. C. (1987). The intentional stance. MIT press.
effect: Theory of mind and moral judgment. Psychological Science, 17(5), 421–427.
Gergely, G., & Csibra, G. (2003). Teleological reasoning in infancy: The naive theory of
Lewis, D. (1973). Causation. The Journal of Philosophy, 70(17), 556–567.
rational action. Trends in Cognitive Sciences, 7(7), 287–292.
Liu, S., Ullman, T. D., Tenenbaum, J. B., & Spelke, E. S. (2017). Ten-month-old infants
Gerstenberg, T., Goodman, N. D., Lagnado, D. A., & Tenenbaum, J. B. (2021).
infer the value of goals from the costs of actions. Science, 358(6366), 1038–1041.
A counterfactual simulation model of causal judgments for physical events.
Livengood, J., & Machery, E. (2007). The folk probably don’t think what you think they
Psychological Review. https://doi.org/10.1037/rev0000281. Advance online
think: Experiments on causation by absence. Midwest Studies in Philosophy, 31,
publication.
107–127.
Gerstenberg, T., & Icard, T. (2019). Expectations affect physical causation judgments.
Lombrozo, T. (2010). Causal explanatory pluralism: How intentions, functions, and
Journal of Experimental Psychology: General, 149(3), 599.
mechanisms influence causal ascriptions. Cognitive Psychology, 61(4), 303–332.
Gerstenberg, T., Peterson, M. F., Goodman, N. D., Lagnado, D. A., & Tenenbaum, J. B.
Lucas, C. G., Griffiths, T. L., Xu, F., Fawcett, C., Gopnik, A., Kushnir, T., … Hu, J. (2014).
(2017). Eye-tracking causality. Psychological Science, 28(12), 1731–1744.
The child as econometrician: A rational model of preference understanding in
Gerstenberg, T., & Tenenbaum, J. B. (2017). Intuitive theories. In Oxford handbook of
children. PLoS One, 9(3), Article e92160.
causal reasoning (pp. 515–548).
Machery, E. (2008). The folk concept of intentional action: Philosophical and
Goldman, A. I. (1967). A causal theory of knowing. The Journal of Philosophy, 64(12),
experimental issues. Mind & Language, 23, 165–189.
357–372.
Machery, E., Stich, S., Rose, D., Chatterjee, A., Karasawa, K., Struchiner, N., …
Goldman, A. I. (1970). Theory of human action. Princeton University Press.
Hashimoto, T. (2017). Gettier across cultures. Noûs, 51(3), 645–664.
Gopnik, A., & Wellman, H. M. (1992). Why the child’s theory of mind really is a theory.
Malle, B. F., & Knobe, J. (1997). The folk concept of intentionality. Journal of
Mind & Language, 7(1–2), 145–171.
Experimental Social Psychology, 33(2), 101–121.
Grice, H. P. (1961). The causal theory of perception. Proceedings of the Aristotelian Society,
Martin, C. B., & Deutscher, M. (1966). Remembering. Philosophical Review, 75(2),
35, 121–168.
161–196.

18
T. Quillien and T.C. German Cognition 214 (2021) 104806

Mele, A. (2001). Acting intentionally: Probing folk notions. In B. Malle, L. Moses, & Searle, J. R. (1983). Intentionality: An essay in the philosophy of mind. Cambridge:
D. Baldwin (Eds.), Intentions and intentionality: Foundations of social cognition (pp. Cambridge Univ. Press.
27–43). Cambridge, MA: MIT/Bradford. Sell, A., Sznycer, D., Al-Shawaf, L., Lim, J., Krauss, A., Feldman, A., … Tooby, J. (2017).
Mele, A. (2009). Causation, action, and free will. In H. Beebee, C. Hitchcock, & The grammar of anger: Mapping the computational architecture of a recalibrational
P. Menzies (Eds.), The Oxford handbook of causation. Oxford University Press. emotion. Cognition, 168, 110–128.
Mele, A. R., & Cushman, F. (2007). Intentional action, folk judgments, and stories: Shaver, K. G. (1985). The attribution of blame. New York: Springer-Verlag.
Sorting things out. Midwest Studies in Philosophy, 31, 184–201. Sloman, S. A., Fernbach, P. M., & Ewing, S. (2012). A causal model of intentionality
Michotte, A. (1963). The perception of causality. Basic books. judgment. Mind & Language, 27(2), 154–180.
Mill, J. S. (1856). A system of logic, ratiocinative and inductive. London: John W. Parker Sousa, P., & Holbrook, C. (2010). Folk concepts of intentional action in the contexts of
and Son. amoral and immoral luck. Review of Philosophy and Psychology, 1(3), 351–370.
Morris, A., Phillips, J., Gerstenberg, T., & Cushman, F. (2019). Quantitative causal Sousa, P., Holbrook, C., & Swiney, L. (2015). Moral asymmetries in judgments of agency
selection patterns in token causation. PLoS One, 14(8). withstand ludicrous causal deviance. Frontiers in Psychology, 6, 1380.
Nadelhoffer, T. (2006). Bad acts, blameworthy agents and intentional actions: Some Sripada, C. (2012). Mental states attribution and the side-effect effect. Journal of
problems for jury impartiality. Philosophical Explorations, 9, 203–220. Experimental Social Psychology, 48, 232–238. https://doi.org/10.1016/j.
Newman, J. (1996). Give: A cognitive linguistic study. Walter de Gruyter. jesp.2011.07.008.
Nichols, S., & Ulatowski, J. (2007). Intuitions and individual differences: The Knobe Strickland, B. (2017). Language reflects “core” cognition: A new theory about the origin
effect revisited. Mind & Language, 22(4), 346–365. of cross-linguistic regularities. Cognitive Science, 41(1), 70–101.
Ossorio, P. G., & Davis, K. E. (1968). The self, intentionality, and reactions to evaluations Sytsma, J., Livengood, J., & Rose, D. (2012). Two types of typicality: Rethinking the role
of the self. In C. Gordon, & K. J. Gergen (Eds.), The self in social interaction. New York: of statistical typicality in ordinary causal attributions. Studies in History and
Wiley. Philosophy of Science Part C: Studies in History and Philosophy of Biological and
Pettit, D., & Knobe, J. (2009). The pervasive impact of moral judgment. Mind & Language, Biomedical Sciences, 43(4), 814–820.
24, 586–604. https://doi.org/10.1111/j.1468- 0017.2009.01375.x. Tesser, A., Gatewood, R., & Driver, M. (1968). Some determinants of gratitude. Journal of
Phillips, J., Luguri, J. B., & Knobe, J. (2015). Unifying morality’s influence on non-moral Personality and Social Psychology, 9(3), 233.
judgments: The relevance of alternative possibilities. Cognition, 145, 30–42. Thalberg, I. (1984). Do our intentions cause our intentional actions? American
Phillips, J., Morris, A., & Cushman, F. (2019). How we know what not to think. Trends in Philosophical Quarterly, 21, 249–260.
Cognitive Sciences, 23(12), 1026–1040. Uttich, K., & Lombrozo, T. (2010). Norms inform mental state ascriptions: A rational
Pinker, S. (1989). Learnability and cognition: The acquisition of argument structure. MIT explanation for the side-effect effect. Cognition, 116(1), 87–100.
Press. Wertz, A. E., & German, T. C. (2007). Belief–desire reasoning in the explanation of
Pinker, S. (2007). The stuff of thought: Language as a window into human nature. Penguin. behavior: Do actions speak louder than words? Cognition, 105(1), 184–194.
Quillien, T. (2020). When do we think that X caused Y? Cognition., 205. Weslake, B. (2015). A partial theory of actual causation. The British Journal for the
Quillien, T., & Barlev, M. (2021). Causal judgment in the wild: Evidence from the 2020 Philosophy of Science.
US presidential election. PsyArXiv.. https://doi.org/10.31234/osf.io/7w9re. Wittgenstein, L. (1953). Philosophical investigations. John Wiley & Sons.
Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling. Journal of Woodward, A. L. (1998). Infants selectively encode the goal object of an actor’s reach.
Statistical Software, 48(2), 1–36. Cognition, 69(1), 1–34.
Savage, L. J. (1954). The foundations of statistics. New York: Wiley. Wright, J., & Bengson, J. (2009). Asymmetries in folk judgments of responsibility and
Scott, R. M., & Baillargeon, R. (2013). Do infants really expect agents to act efficiently? A intentional action. Mind & Language, 24, 237–251.
critical test of the rationality principle. Psychological Science, 24(4), 466–474.

19