Natural Language
Processing
Lecture 10: Semantic Role Labeling.
11/23/2020
COMS W4705
Yassine Benajiba
Word Meaning and Sentence
Meaning
• So far we have discussed the meaning of individual words.
• Now: meaning of entire predicate-argument structures and
sentences.
• What should the representations be?
• How do we compute predicate or sentence-level
representations from word representations?
• What is the role of syntax?
Approaches to Sentence
Level Semantics
• Semantic Role Labeling (SRL) / Frame Semantic Parsing.
• Target representation: PropBank predicate argument
structures, FrameNet-style annotations.
• Full-sentence semantics (next week)
• Target representations: Predicate-logic, Abstract
Meaning Representation
Frame Semantics
(Fillmore, 1992)
• Long history in cognitive science, AI, ... (Minksy 1974, Barsalou 1992)
• A frame represents a situation, object, event providing background
needed to understand a word ('cognitive schemata').
• Different words (of different part-of-speech) can evoke the same
frame
Giving → {donate.v, gift.n, give.v, hand over.v, treat.v, ... }
• A pair of a word and a frame is called a lexical unit (LU).
Frame Elements
• Frames describe the interaction/relation between a set of frame-
specific semantic roles called Frame Elements (FEs).
Giving: A Donor transfers a Theme from a Donor to a Recipient.
Core:
Donor The person that begins in possession of the Theme and causes it to be in
the possession of the Recipient
Recipient The entity that ends up in possession of the Theme.
Theme The object that changes ownership.
Non-core:
Means The Means by which the Donor gives the Theme to the Recipient.
Purpose The Purpose for which the Donor gives the Theme to the Recipient.
Place
Time
...
FrameNet
(Baker et al, 1998)
• Lexical resource based on Frame Semantics: 13640 lexical units
in 1087 frames.
• Example annotations illustrate how frame elements are realized
linguistically.
• Frames evoked by frame evoking elements (FEE).
• Central interest: mapping from Grammatical Function (Subj,
Obj, ...) to Frame Elements.
Apple wanted to donate a computer to every school in the country .
POS NNP VVD TO VB DT NN PRP DT NN IN DT NN .
FE Donor FEE Theme Receipient
GF Subj Obj Dep-to
PT NP NP PPto
Valence Pattern
• Valence patterns (derived from annotated sentences)
specify different ways grammatical roles (subject, object,
...) can be mapped to frame elements for a given lexical
unit.
Valence pattern Example sentence
(subj/DONOR) V (obj/RECIPIENT) (obj2/THEME) John gave Mary the book
(subj/DONOR) V (obj/THEME) (dep-to/RECIPIENT) John gave the book to Mary
(subj/DONOR) V (dep-of/THEME) (dep-to/RECIPIENT) John gave of his time to people like M.
(subj/DONOR) V (dep-to/RECIPIENT) John gave to charity
(Slide by Bob Coyne)
Frame-to-Frame Relations
• Frames are related via frame-to-frame relations.
Giving_scenario Lose_posession
subframe of
inherits
perspective on
Transfer Giving
inherits inherits
Commerce_sell Commerce_pay
Frame-Element Relations
inherits
Giving Commerce_sell
Donor Buyer
Recipient Goods
Theme Seller
PropBank
(Baker et al, 2005)
• Another corpus annotated with semantic roles, based on
English Penn Treebank & OntoNotes 5.0. (~2m Words)
• Also available: Chinese, Hindi/Urdu, Arabic.
• Full-text annotation (only verbs).
• Numbered arguments (semantic roles).
• Interpretation is specific to each verb.
Frameset for donate.01
Arg0: giver donate
the company over $35,000 to residents
Arg1: thing given d
Arg0 rel Arg1 Arg2
Arg2: entity given to
Proto Roles
(Dowty 1991)
• Proto-Agent
• Volitional involvement in event or state.
• Sentience (and/or perception)
• Causes an event or change of state in another participant
• Movement (relative to position of another participant)
• Proto-Patient
• Undergoes change of state
• Causally affected by another participant
• Stationary relative to movement of another participant
PropBank Roles
• Each frameset has numbered argument: Arg0, Arg1, Arg2,…
• Arg0:PROTO-AGENT
• Arg1:PROTO-PATIENT
• Arg2: usually: benefactive, instrument, attribute, or end state
• Arg3: usually: start point, benefactive, instrument, or attribute
• Arg4 the end point (Arg2-Arg5 are not really that consistent,
causes a problem for labeling)
PropBank FrameSets
• Different framesets correspond to different senses.
Frameset for tend.01, care for
Arg0: tender
Arg1: thing tended (to)
John tends to the needs of his patrons
Arg0 rel Arg1
Frameset for tend.02, have a tendency
Arg0: theme
Arg2: attribute
The cost, or premium tends to get fat in times of crisis
Arg0 rel Arg2
Another Example
Frameset for increase.01, go up incrementally
Arg0: causer of increase
Arg1: thing increasing
Arg2: amount increased by
Arg3: start point
Arg4: end point
[Arg0 Big Fruit Co.] increased [Arg1 the price of bananas]
[Arg1 The price of bananas] was increased again [Arg0 by Big Fruit Co.]
[Arg1 The price of bananas] increased [Arg2 5%]
Observations:
Syntax and semantics do not map 1:1. Generalize away from syntactic variations.
Semantic Role Labeling (SRL)
• Input: raw sentence.
• Goal: automatically produce PropBank or FrameNet-style
annotations ("frame-semantic parsing").
• Applications:
• Question Answering (Shen and Lapata 2007, Surdeanu et al.
2011)
• Machine Translation (Liu and Gildea 2010, Lo et al. 2013)
• Stock prediction, spoken dialog segmentation, ...
• How would you approach this problem?
Generic SRL Algorithm
Algorithm outline:
• Parse the sentence (dependence or constituency parse)
• Detect all potential targets (predicates / frame evoking
elements)
• For each predicate:
• For each node in the parse tree use supervised ML
classifiers to:
1. identify if it is an argument.
2. label the argument with a role.
Choosing Targets
• For PropBank:
• Choose all verbs.
• For FrameNet:
• Choose all lexical items (verbs, nouns, adjectives) that are in
the annotated FrameNet training data.
SRL Example
S
NP =ARG1 VP
VBD =TARGET
NP=ARGM-LOC SBAR
broke
DT NN WHPP S
The bed IN WDT NP=ARG0 VP
I I
on which PRP VBD =TARGET
I
I slept
I
SRL Example
subj
relcl
prep
det
pobj subj
The bed on which I slept broke
ARG1 ARG0 TARGET TARGET
ARGM-LOC
Selectional Restrictions and
Preferences
• Different semantic roles might have restrictions on the
semantic type of arguments they can take.
I want to eat someplace nearby
I want to eat Korean for lunch
• Food FE (or ARG1) needs to be edible.
• But what about:
...people realized you can't eat gold for lunch if you're hungry
• How could you model these?
Features
What features should we use for argument detection and
labeling?
• target predicate: broke
• headword (+POS): bed NN
• phrase type: NP S
• linear position: before or after the target
• argument structure of the verb. NP ??? VP
"NP broke"
• target voice: active VBD
=TARGET
• possibly semantic features
NP SBAR
broke
(named entity class,
WordNet synsets of head word, S
DT NN WHPP
...)
• first and last word of constituent The bed IN WDT NP VP
and their POS. I I
on which PRP VBD
• Parse tree path between target and node.
I
I
slept
(Features used in Gildea & Jurafsky 2000) I
Parse Tree Path
S
NP ??? VP
VBD
NP SBAR =TARGET
broke
DT NN WHPP S
The bed IN WDT NP VP
I I
on which PRP VBD
I
I slept
I
NP↑S↓VP↓VBD
Parse Tree Path
S
NP VP
VBD
NP SBAR
broke
DT NN WHPP S
The bed IN WDT NP ??? VP
I I
on which PRP VBD =TARGET
I
I slept
I
NP↑S↓VP↓VBD
Parse Tree Path
S
NP VP
VBD
NP ??? SBAR
broke
DT NN WHPP S
The bed IN WDT NP VP
I I
on which PRP VBD =TARGET
I
I slept
I
NP↑NP↓SBAR↓S↓VP↓VBD
Frequent Path Features
(from Palmer, Gildea, Xiu, 2010, SRL book)
Candidate Pruning (Xue and Palmer 2004)
• Algorithm looks at one target at a time. Very few phrases
can possibly be arguments.
• Difficult for classifiers to learn: Few positive samples
(phrases that are arguments), few positive samples.
• Syntax should tell us something about possible
arguments.
Pruning Heuristic (Xue and Palmer 2004)
• Add sisters of the predicate, then aunts, then great-aunts,
etc.
• Ignore nodes in the subtrees of the selected nodes.
• Ignore anything in coordinated structures.
FrameNet Parsing
• Slightly more complex: Need to decide on the frame first,
then use frame-specific classifiers for the semantic roles.
subj
relcl
prep
det
pobj subj
The bed on which I slept broke
Place Sleeper TARGET
Sleep
Frame Semantic Parsing Systems
More recent work uses Neural Networks (e.g. Swayamdipta et al. 2017)
Features used in FrameNet
Parsing
G&J J&N SEMAFOR
Syntactic Representation PS DepMST DepMST
Collins
Target Dependency Labels and ✔ ✔
Words
Target parent word / POS ✔ ✔
Target word/ POS ✔ ✔ ✔
Voice (for verb targets) ✔ ✔ ✔
Relative Position (before/after/on) ✔ ✔ ✔
Global Inference
• So far, classifier just decided on one argument at a time.
• But there are interactions between arguments!
• FEs may not overlap.
• Labeling one constituent as ARG0 should increase the
probability of another constituent to be ARG1.
• Some argument combinations are impossible.
• Solutions: Beam Search (Das et al. 2010/2014),
Dual Decomposition (Des et al. 2010/2014), DP algorithm
(Täckström et al. 2015)
Acknowledgments
• Some slides by Martha Palmer, Shumin Wu, Dan
Jurafsky, Nathan Schneider.
