Statistically constrained shallow text marking: techniques, evaluation paradigm, and results

27 February 2007 Statistically-constrained shallow text marking: techniques, evaluation paradigm and results

Brian Murphy, Carl Vogel

Proceedings Volume 6505, Security, Steganography, and Watermarking of Multimedia Contents IX; 65050Z (2007) https://doi.org/10.1117/12.713355
Event: Electronic Imaging 2007, 2007, San Jose, CA, United States

Abstract

We present three natural language marking strategies based on fast and reliable shallow parsing techniques, and on widely available lexical resources: lexical substitution, adjective conjunction swaps, and relativiser switching. We test these techniques on a random sample of the British National Corpus. Individual candidate marks are checked for goodness of structural and semantic fit, using both lexical resources, and the web as a corpus. A representative sample of marks is given to 25 human judges to evaluate for acceptability and preservation of meaning. This establishes a correlation between corpus based felicity measures and perceived quality, and makes qualified predictions. Grammatical acceptability correlates with our automatic measure strongly (Pearson's r = 0.795, p = 0.001), allowing us to account for about two thirds of variability in human judgements. A moderate but statistically insignificant (Pearson's r = 0.422, p = 0.356) correlation is found with judgements of meaning preservation, indicating that the contextual window of five content words used for our automatic measure may need to be extended.

Citation Download Citation

Brian Murphy and Carl Vogel "Statistically-constrained shallow text marking: techniques, evaluation paradigm and results", Proc. SPIE 6505, Security, Steganography, and Watermarking of Multimedia Contents IX, 65050Z (27 February 2007); https://doi.org/10.1117/12.713355

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available