<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="pinter-elhadad-2023-emptying">
    <titleInfo>
        <title>Emptying the Ocean with a Spoon: Should We Edit Models?</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yuval</namePart>
        <namePart type="family">Pinter</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Michael</namePart>
        <namePart type="family">Elhadad</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We call into question the recently popularized method of direct model editing as a means of correcting factual errors in LLM generations. We contrast model editing with three similar but distinct approaches that pursue better defined objectives: (1) retrieval-based architectures, which decouple factual memory from inference and linguistic capabilities embodied in LLMs; (2) concept erasure methods, which aim at preventing systemic bias in generated text; and (3) attribution methods, which aim at grounding generations into identified textual sources. We argue that direct model editing cannot be trusted as a systematic remedy for the disadvantages inherent to LLMs, and while it has proven potential in improving model explainability, it opens risks by reinforcing the notion that models can be trusted for factuality. We call for cautious promotion and application of model editing as part of the LLM deployment process, and for responsibly limiting the use cases of LLMs to those not relying on editing as a critical component.</abstract>
    <identifier type="citekey">pinter-elhadad-2023-emptying</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-emnlp.1012</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-emnlp.1012/</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>15164</start>
            <end>15172</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Emptying the Ocean with a Spoon: Should We Edit Models?
%A Pinter, Yuval
%A Elhadad, Michael
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Findings of the Association for Computational Linguistics: EMNLP 2023
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F pinter-elhadad-2023-emptying
%X We call into question the recently popularized method of direct model editing as a means of correcting factual errors in LLM generations. We contrast model editing with three similar but distinct approaches that pursue better defined objectives: (1) retrieval-based architectures, which decouple factual memory from inference and linguistic capabilities embodied in LLMs; (2) concept erasure methods, which aim at preventing systemic bias in generated text; and (3) attribution methods, which aim at grounding generations into identified textual sources. We argue that direct model editing cannot be trusted as a systematic remedy for the disadvantages inherent to LLMs, and while it has proven potential in improving model explainability, it opens risks by reinforcing the notion that models can be trusted for factuality. We call for cautious promotion and application of model editing as part of the LLM deployment process, and for responsibly limiting the use cases of LLMs to those not relying on editing as a critical component.
%R 10.18653/v1/2023.findings-emnlp.1012
%U https://aclanthology.org/2023.findings-emnlp.1012/
%U https://doi.org/10.18653/v1/2023.findings-emnlp.1012
%P 15164-15172

Markdown (Informal)

[Emptying the Ocean with a Spoon: Should We Edit Models?](https://aclanthology.org/2023.findings-emnlp.1012/) (Pinter & Elhadad, Findings 2023)

• Emptying the Ocean with a Spoon: Should We Edit Models? (Pinter & Elhadad, Findings 2023)

ACL

• Yuval Pinter and Michael Elhadad. 2023. Emptying the Ocean with a Spoon: Should We Edit Models?. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15164–15172, Singapore. Association for Computational Linguistics.