Neural bases of speech and language
Linguistics vs. neurolinguistics
Linmin Zhang
Sept. 4, 2024
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 1 / 25
Today
David Marr (1945–1980) and his book Vision (1979/1982/2010)
Three levels of David Marr in understanding/explaining cognition
What is linguistics about? ‘the study of behavior’
What is neurolinguistics about? ‘the study of circuits’
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 2 / 25
Outline
1 Three levels of David Marr in understanding/explaining cognition
2 What is linguistics about?
3 What is neurolinguistics about?
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 3 / 25
What is cognition?
cognitio-: ‘get to know’
▸ What does know mean?
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 4 / 25
What is cognition?
cognitio-: ‘get to know’
▸ What does know mean?
David Marr: information-processing
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 4 / 25
What is cognition?
cognitio-: ‘get to know’
▸ What does know mean?
David Marr: information-processing
Cognition: including all kinds of human intelligence
▸ Reasoning
▸ Planning
▸ Decision making
▸ Problem solving
▸ Abstract thinking
▸ Learning
▸ Language
▸ Vision (https://www.youtube.com/watch?v=o5xjXTOmWP8)
▸ …
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 4 / 25
What is cognition?
cognitio-: ‘get to know’
▸ What does know mean?
David Marr: information-processing
Cognition: including all kinds of human intelligence
▸ Reasoning
▸ Planning
▸ Decision making
▸ Problem solving
▸ Abstract thinking
▸ Learning
▸ Language
▸ Vision (https://www.youtube.com/watch?v=o5xjXTOmWP8)
▸ …
(Question: are humans unique in having language cognition?)
▸ Waggle dance of bees:
https://www.youtube.com/watch?v=-7ijI-g4jHg
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 4 / 25
Marr’s 3 levels on understanding/explaining
cognition
Computational level: what does the system do (e.g.: what
problems does it solve or overcome) and similarly, why does it do
these things
Algorithmic level (sometimes representational level): how does
the system do what it does, specifically, what representations
does it use and what processes does it employ to build and
manipulate the representations
Implementational/physical level: how is the system physically
realized (in the case of biological vision, what neural structures
and neuronal activities implement the visual system)
(https://en.wikipedia.org/wiki/David_Marr_(neuroscientist)#Levels_of_analysis)
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 5 / 25
Marr’s 3 levels: illustrations
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 6 / 25
Interdisciplinary study of human cognition
Combining methods from
▸ Psychology
▸ Linguistics
▸ Philosophy
▸ Computer science
▸ Neuroscience – Cognitive neuroscience
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 7 / 25
Theoretical linguistics vs. neurolinguistics
Theoretical linguistics: Studying language cognition at the
computational level / behavioral level
Neurolinguistics: Studying language cognition at the
representational and physical levels
– joining force between linguistics and cognitive neuroscience
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 8 / 25
Outline
1 Three levels of David Marr in understanding/explaining cognition
2 What is linguistics about?
3 What is neurolinguistics about?
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 9 / 25
Language is a cognitive ability
Two meanings of English word language:
▸ as a social construction (e.g., English, Chinese) – what French word
langue means
▸ as a cognitive ability, a human-specific intuition – what French
word langage means
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 10 / 25
Language is a cognitive ability
Language: the ability to recover from sensory stimulation the
mental model that the interlocutor has in mind when generating
the auditory output
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 11 / 25
Language is a cognitive ability
Language: the ability to recover from sensory stimulation the
mental model that the interlocutor has in mind when generating
the auditory output
Discussion: Are you satisfied with this definition?
▸ Does language have to involve auditory output?
▸ Does language have to be face-to-face?
▸ How about waggle dance of bees? Is this a language?
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 11 / 25
Language is a cognitive ability
Language: the ability to recover from sensory stimulation the
mental model that the interlocutor has in mind when generating
the auditory output
Discussion: Are you satisfied with this definition?
▸ Does language have to involve auditory output?
▸ Does language have to be face-to-face?
▸ How about waggle dance of bees? Is this a language?
Discussion: Is our language ability innate or learned? What is
language ability similar to?
▸ Walking?
▸ Seeing?
▸ Playing tennis?
▸ Playing the piano?
▸ Solving mathematical problems?
▸ How about speaking a foreign language (i.e., non native language)?
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 11 / 25
Theoretical linguistics vs. neurolinguistics
Theoretical linguistics: Studying language cognition at the
computational level / behavioral level
▸ What does the system do? What problems does it solve? Why these
problems?
▸ Essentially it studies what constitutes speakers’ subconscious
knowledge (phonology, morphology, syntax, semantics,
pragmatics, ⋯)
At the computational / behavioral level, usually empirical
observations and measurements about language phenomena can
be done directly, without specific tools
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 12 / 25
What does our language intuition include?
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 13 / 25
What does our language intuition include?
Semantics: Study of the meaning of linguistic expressions
Semantics
Syntax: Study of the internal structure of sentences
Syntax
Morphology: Study of the internal structure of words
”words”/morphemes
Phonology: Study of the abstract categories of speech sounds
sound categories
Phonetics: Study of speech sounds continues speech signal
(What else? A mental dictionary, i.e., lexicon)
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 14 / 25
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 15 / 25
Stevens (1946): What is measurement?
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 16 / 25
Stevens (1946): there are 4 levels of measurement and
scales
ratio scales: interval scales: ordinal scales: nominal scales
+ an ordering + an ordering + an ordering
+ a unit + a unit
+ a zero point
At the behavioral level, which of the following sentence is natural
for native speakers? (How natural are these sentences?)
(1) Is the man that is bald your uncle?
(2) Is the man that bald is your uncle?
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 17 / 25
Outline
1 Three levels of David Marr in understanding/explaining cognition
2 What is linguistics about?
3 What is neurolinguistics about?
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 18 / 25
Theoretical linguistics vs. neurolinguistics
Neurolinguistics: Studying language cognition at the
representational and physical levels
▸ How does the system do what it does?
▸ What representations does it use and what
processes/computations does it employ to build and manipulate
the representations?
▸ What is the relation between language and other cognitive abilities?
▸ What are neural/physical bases of linguistic mental representations
and computations?
At the physical level, usually empirical observations and
measurements about language-related brain activation cannot be
done directly
Specific measurement techniques are required for measuring
▸ (reaction time) psycholinguistics
▸ (eye movement) psycholinguistics
▸ blood flow neurolinguistics
▸ electric activity neurolinguistics
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 19 / 25
Focus of this course
Language production: executing the motor commands necessary
to externalize meaning
Language comprehension: uncovering meaning from sound in
real-time processing
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 20 / 25
The same physical signals are not necessarily the same
linguistic representations: an example
phonemes: speech sounds are grouped into categories, called
phonemes
▸ English words: stop vs. top
☀ the red part vs. the green part: are they the same sound?
☀ the red part vs. the green part: do they belong to the same category?
i.e., do they share the same linguistic representation?
▸ Korean words: 딸 [tal] ‘daughter’ vs. 탈 [tal] ‘mask’
☀ the red part vs. the green part: are they the same sound?
☀ the red part vs. the green part: do they belong to the same category?
i.e., do they share the same linguistic representation?
We convert physical signals into abstract categorical
representations.
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 21 / 25
The same physical signals are not necessarily the same
linguistic representations: another example
▸ Mary came to the party, but Bill didn’t.
▸ Mary kissed her mother, but Bill didn’t.
What does but Bill didn’t mean in these sentences?
The same physical signal ; the same abstract, mental
representation?
▸ The author wrote the book.
▸ The author began the book.
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 22 / 25
Within neurolinguistics
How do we study language cognition at the representational and
implementational levels?
Not so easy! (see Poeppel 2015, a.o.)
‘The brain region involved in the processing of X’ ≠
‘the brain region responsible for the processing of X’
▸ The ‘how’ question (i.e., the brain mechanisms of computation)
matters more than the ‘where’ question (i.e., which brain region is
activated in which activity)
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 23 / 25
Key concepts that you should understand by now
Cognition and Marr’s three levels
Language as a human-specific cognition
Theoretical linguistics vs. neurolinguistics
What our language intuition includes: phonetics and phonology,
morphology, syntax, semantics, lexicon.
The fundamental research questions in neurolinguistics: mental
representations and computations and their neural correlates
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 24 / 25
Reading for next time
Kemmerer, 2015/2022, Cognitive neuroscience of language.
Chapter 1. ; Basics of the brain
Fall 2024, NEUR-SHU 131 Linguistics vs. neurolinguistics Sept. 4, 2024 25 / 25
