<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="bassignana-plank-2022-mean">
    <titleInfo>
        <title>What Do You Mean by Relation Extraction? A Survey on Datasets and Study on Scientific Relation Classification</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Elisa</namePart>
        <namePart type="family">Bassignana</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Barbara</namePart>
        <namePart type="family">Plank</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Samuel</namePart>
            <namePart type="family">Louvan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andrea</namePart>
            <namePart type="family">Madotto</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Brielen</namePart>
            <namePart type="family">Madureira</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Dublin, Ireland</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Over the last five years, research on Relation Extraction (RE) witnessed extensive progress with many new dataset releases. At the same time, setup clarity has decreased, contributing to increased difficulty of reliable empirical evaluation (Taillé et al., 2020). In this paper, we provide a comprehensive survey of RE datasets, and revisit the task definition and its adoption by the community. We find that cross-dataset and cross-domain setups are particularly lacking. We present an empirical study on scientific Relation Classification across two datasets. Despite large data overlap, our analysis reveals substantial discrepancies in annotation. Annotation discrepancies strongly impact Relation Classification performance, explaining large drops in cross-dataset evaluations. Variation within further sub-domains exists but impacts Relation Classification only to limited degrees. Overall, our study calls for more rigour in reporting setups in RE and evaluation across multiple test sets.</abstract>
    <identifier type="citekey">bassignana-plank-2022-mean</identifier>
    <identifier type="doi">10.18653/v1/2022.acl-srw.7</identifier>
    <location>
        <url>https://aclanthology.org/2022.acl-srw.7/</url>
    </location>
    <part>
        <date>2022-05</date>
        <extent unit="page">
            <start>67</start>
            <end>83</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T What Do You Mean by Relation Extraction? A Survey on Datasets and Study on Scientific Relation Classification
%A Bassignana, Elisa
%A Plank, Barbara
%Y Louvan, Samuel
%Y Madotto, Andrea
%Y Madureira, Brielen
%S Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop
%D 2022
%8 May
%I Association for Computational Linguistics
%C Dublin, Ireland
%F bassignana-plank-2022-mean
%X Over the last five years, research on Relation Extraction (RE) witnessed extensive progress with many new dataset releases. At the same time, setup clarity has decreased, contributing to increased difficulty of reliable empirical evaluation (Taillé et al., 2020). In this paper, we provide a comprehensive survey of RE datasets, and revisit the task definition and its adoption by the community. We find that cross-dataset and cross-domain setups are particularly lacking. We present an empirical study on scientific Relation Classification across two datasets. Despite large data overlap, our analysis reveals substantial discrepancies in annotation. Annotation discrepancies strongly impact Relation Classification performance, explaining large drops in cross-dataset evaluations. Variation within further sub-domains exists but impacts Relation Classification only to limited degrees. Overall, our study calls for more rigour in reporting setups in RE and evaluation across multiple test sets.
%R 10.18653/v1/2022.acl-srw.7
%U https://aclanthology.org/2022.acl-srw.7/
%U https://doi.org/10.18653/v1/2022.acl-srw.7
%P 67-83

Markdown (Informal)

[What Do You Mean by Relation Extraction? A Survey on Datasets and Study on Scientific Relation Classification](https://aclanthology.org/2022.acl-srw.7/) (Bassignana & Plank, ACL 2022)

• What Do You Mean by Relation Extraction? A Survey on Datasets and Study on Scientific Relation Classification (Bassignana & Plank, ACL 2022)

ACL

• Elisa Bassignana and Barbara Plank. 2022. What Do You Mean by Relation Extraction? A Survey on Datasets and Study on Scientific Relation Classification. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 67–83, Dublin, Ireland. Association for Computational Linguistics.