0HV60 Thinking and Deciding

Lecture 7: Comprehending and producing language

Rianne Conijn
In this lecture
• Introduction to language
• Components of language
• Language properties
• Neuroscientific evidence of language -- neurolinguistics
• Language comprehension
• Reading
• Sentence and discourse understanding
• Language production
• Speaking
• Writing (in the workshop)
• Bilingualism

2
Components of language
We first need some language to talk about language…

3
Language
• Organized way to combine words to communicate
• Written, spoken, or signed
• Production and comprehension

• Involves many cognitive processes

4
Components of language
• Phonology: speech sounds
• Morphology: word structure
• Syntax: sentence structure
• Semantics: meaning
• Pragmatics: use of language in
communication
• Discourse: language units larger than a
sentence

5
Phonology
• Study of speech sounds
• Phoneme
• Smallest unit of speech
• Different languages use different phonemes
• 46 for English
• 12 in Hawaiian
• 147 in Xu (spoken near Kalahari desert)

6
Morphology
• Study of word structure
• Morpheme
• Smallest unit of a word that denotes meaning
• Root words and affixes

Root words Prefixes Suffixes

Cake Chair Happy Pre- Non- Un- -ly -ist -ness

7
Syntax
• Study of sentence structure
• Describes how words are connected
• Sentence [S] = Noun phrase [NP] + Verb phrase [VP]

• Constituency
• Constituent: group of words that behave as
a single unit, e.g., noun phrase
• Grammatical relations
• Relationship between constituents

8
Semantics
• Study of meaning (of words and sentences)
• Homonym
• Same word, different meaning
• Synonym
• Same meaning, different words

9
Discourse
• Language units larger than a sentence
• Written, spoken, and sign language
• Phone conversation
• Report
• …

10
Pragmatics
• Study of the use of language
• Explaining discourse in its context
• Knowledge about social rules, intentions
• Considerations of the speaker/audience
• Linguistic: turn-taking, clarification, …
• Non-linguistic: eye contact, body language, voice, …

11
Studying language – linguistics
• A caution: linguistics is English-centered
• 6,000-7,000 spoken languages
• Languages differ in all their components
• Pitch can change meaning
• Use of passive voice
• Use of nouns with grammatical gender
• …
• English is an outlier
• Simple grammar, difficult/irregular pronunciation

12
Our discussion of language
• Mostly focus on psycholinguistics
• Cognitive processes involved in language production and understanding
• Social rules in language usage
• Some neurolinguistics
• Presentation of language in the brain
• Some applications (also in the workshops)
• Applied linguistics
• Computational linguistics

13
Language properties

14
Properties of language
• Communicative
• Communicate emotions and feelings
• Arbitrary
• Meanings are defined by conventions
• Flexible representations possible
• Dynamic
• Dutch word of 2022: “klimaatklever”
• Dutch word of 2023: “graaiflatie”
• Generative
• We can make sentences we have never heard before

15
Noam Chomsky
• Behaviorism
• Language is learned through interaction with the environment
(conditioning)
• Generative property of language
• Children would never be able to produce infinite number of
sentences only based on imitation and conditioning

Universal grammar
1. The ability to learn grammar is innate
2. There are properties that all human languages share

16
Universal Grammar
• Generative syntax to study the universal grammar
• Syntax trees show possible sentence structures

• Structure of the sentence (grammar) of unfamiliar

sentence points us towards its meaning
“Colorless green ideas sleep furiously”

17
Transformational Grammar
• By applying a set of rules, you can produce an
infinite number of correct sentences
• Process to convert meaning into words

• Surface structure: words

• Deep structure: meaning
• S1 & S2 same meaning, different structure
• S1 & S3 same words, different meaning

• More complex sentences require more

transformations, so take longer to process
• Carroll (2004) failed to support this hypothesis

18
Finding universal rules
• Finding universal rules (much data needed!)
• E.g., Adverbs must appear in a verb phrase
• Proved to be very hard!

• 2000s: One principle: recursion

• Sentence structures can be nested, e.g., The bike, which is painted, is mine

• Criticism
• Everett (2005): Amazonic language Pirahã does not have recursive
structures…
• Properties might be an effect of general cognitive abilities and context

19
Towards focus on meaning (from 1970s)
• Less emphasis on the structure of language
• Cognitive-functional approach
• Function of human language is to communicate meaning
• Focus on communicative functions & social conventions

20
Neurolinguistics

21
Neurolinguistics
• fMRI and ERP studies
• Initial work: individuals with aphasia
• Broca's area/Broca's aphasia: trouble
producing language
• Wernicke's area/Wernicke's aphasia: trouble
understanding language

22
Broca’s aphasia Wernicke’s aphasia

https://youtu.be/1aplTvEQ6ew https://youtu.be/B-LD5jzXpLE

Sarah 11 years after her stroke:

23 https://youtu.be/FUH-kLgsqBw
Broca’s area revisited
• Hard to study: brain damage rarely in the same area
• Broca’s area is not specific to language
• Music, gestures, imagery
• Broca’s area might be more related to
inhibition/resolving conflict
• Executive attention network (Stroop!)

24
Hemispheric specialization
• Left hemisphere
• Does most of the work (highest neural activity)
• Active in speech perception and interpretation
• Right hemisphere
• Abstract processes: emotional tone, appreciating humor
• Understanding ambiguous sentences or subtle meanings
“Sometimes I wake up grumpy
Other times I let him sleep in”

Note: Not all people show dominant left hemisphere localization for language
processes

25
Language comprehension

26
Reading
• Reading requires many cognitive processes
• Reading is remarkably efficient and accurate (200-300 words/minute)
• Context-dependent
• Meaning of unfamiliar words recognized from the context
• Working memory is important when reading
• Large span: better performance

27
Two pathways for reading words
• How do we recognize a word?
• Do we ‘sound out’ the words?

• Dual-route approach to reading

• Direct-access route: recognize word directly through vision
• Indirect-access route: recognize word by first sounding out the word

28
Research on dual-route
• Direct access route
• Mown-down, horse-worse
• Read first word, pronounce second word
• Sound of first word does not interfere with pronouncing second word

• Indirect access route

• Lion-bare, lion-bean
• State if related or not
• More errors if second word sounded like a related word (bare-bear)

29
Dual-route compromise
• We tend to use both
• Effects of words
• Indirect approach more often for uncommon words (tongue twisters!)
• Individual differences
• Indirect approach more often for beginning readers

30
Factors affecting sentence comprehension
• Negatives
• “Few people strongly deny that the world is not flat”
• BRM2 exam: “Therefore there was no evidence to reject the null hypothesis that
the groups are equal.”

• Passive voice
• “The paper was written by the student”

• Nested structures: syntactic complexity

• “The paper that the student that was sitting in the front row has written”
• More demanding (working memory)
• Or: less practice

31
Factors affecting comprehension - ambiguity
• Lexical ambiguity
• Homonyms (same word, different meaning)
• Longer pauses when processing ambiguous words
• Choice for: (1) Most common word; (2) Most likely word in context

• Syntactic ambiguity
GRANDMOTHER OF EIGHT MAKES HOLE IN ONE
• Sentences with ambiguous structures
• Linguistic and non-linguistic context
can influence expected structure

32
Good-enough processing – 10ed only
• Grasp the meaning of the sentence
• Only read part of the sentence
• We can guess the rest (top-down processing)

• Heuristic: good enough! (most of the times…)

• Speed reading

33
Understanding discourse
• Discourse
• Language units larger than a sentence

• Theory of mind
• Trying to figure out the mental state of other people (book character, partner)

• Importance of context, background, expertise, scripts, and schemas

• We make (a lot of) inferences!
• Inconsistent information takes longer to process

34
Language production

35
Speaking: producing a word
• Selecting a word from 75,000 candidates…
• Grammatical, semantic, and phonological
accuracy
• Is this information retrieved simultaneously or
independently?
• Van Turennout and colleagues (1998): ERP
showed independent access

36
Slips-of-the-tongue / speech errors
• Speech errors: We mean to say one thing, but utter another
• Errors can occur at any stage of the speech production
• Error analysis tell us how speech production is planned

• Sound errors: phoneme exchange

• “At the lead of spite” → “Speed of light”
• “Go and shake a tower” → “Go and take a shower”
• Morpheme errors
• “Self-instruct destruction” → “self-destruct instruction”
• Word errors
• “Your model is renosed” → “Your nose is remodeled”
• “I have to fill up my gas with car”→ “I have to fill up my car with gas”

37
Explaining speech errors: Connectionism (ch. 8)
• Cognitive processes can be understood in terms of networks
• Activation flows through networks of many simple, neuron-like units

• Parallel distributed processing

• Activation happens in different locations (distributed)
• Activation happens simultaneously (parallel)

38
Parallel Distributed Processing
• Based on physiological and structural properties of neurons
• Neurons can be inactive, excitatory and inhibitory
• Patterns of activation represent cognitive processes

39
Speech errors
• Dell’s model of sound processing in
sentence production
• Similar to connectionist approach
• Spreading activation
• Planning activates sound elements

• Each sound can be activated by several different words

• High activation can cause the incorrect sound to be produced

40
Speaking: producing a sentence
• Limits of attention and memory
• Stages of producing a sentence
• Planning the gist
• Choose specific words and grammatical form
• Convert these intentions into speech
• Linearization problem
• Transforming thought or mental image into an ordered, linear sequence of words
• We also need to plan the prosody ("melody", rhythm, emphasis)

41
Speaking: producing discourse
• Discourse
• Language units larger than a sentence
• Little research on this… (some on narratives, directives)

• Establishing common ground

• Sharing background knowledge, schemas, and experiences necessary for mutual
understanding

42
Naturalistic communication
• Motor movements of vocal system and
gestures
• Frick-Horbury and Guttentag (1998):
movements help!
• De Wit et al. (2018): Movements also help in
language learning with a robot tutor

• Embodied cognition De Wit et al. (2018)

• People use their bodies to express their https://doi.org/10.1145/3171221.3171277
knowledge

43
Writing
• Many cognitive processes involved
in writing
• Yet, writing is remarkably fluent
• Many theories (see also Short paper
on writing theories)
• Showing (sub)processes in writing
• Importance of working memory and
long-term memory

44 Hayes (1996)
Writing
• Coordination of writing processes: cascading model (Olive, 2014)
• Pauses before sentences are remarkably short
• Planning needs to happen in parallel! (Roeser et al., 2024)

• Measuring writing process – workshop

45 Roeser et al. (2024). https://osf.io/qr58k

Bilingualism
• Bilingual speaker
• Being fluent in two different languages

• Bilingualism has many advantages

• E.g., more aware of form, structure, and pragmatics of language
• Bialystok (2001,2002): can be traced to selective-attention skills; inhibition of the
most obvious response to produce an alternative response

• Only minor disadvantages

• Altered pronunciation, slightly slower processing of language

46
Second-language acquisition
I’m too old to learn a new language…

• Vocabulary: not related to age

• Phonology: easier when younger
• Grammar: sometimes easier when younger, if first language is largely
different from second language

• How babies learn a new language:

https://www.ted.com/talks/patricia_kuhl_the_linguistic_genius_of_babies

47
Applications

48
Applications
• Computer-based systems for (second) language acquisition

• Natural language processing

• Machine translation
• Automated summarization
• Automated essay grading

• Natural language generation

• LLMs(!)
• Chatbots, voice assistants

49
Summary
• Language comprehension & production involve many cognitive processes
• Father of modern linguistics: Noam Chomsky
• Psycholinguistic & neuroscientific evidence of language

• Still quite some debates

• Not easy to study language as so much is going on!

• Allows for many applications

50