research-article

Intelligent assistance for conversational storytelling using story patterns

Authors:

Henry LiebermanAuthors Info & Claims

IUI '11: Proceedings of the 16th international conference on Intelligent user interfaces

Pages 217 - 226

https://doi.org/10.1145/1943403.1943438

Published: 13 February 2011 Publication History

Abstract

People who are not professional storytellers usually have difficulty composing travel photos and videos from a mundane slideshow into a coherent and engaging story, even when it is about their own experiences. However, consider putting the same person in a conversation with a friend - suddenly the story comes alive.

We present Raconteur 2, a system for conversational storytelling that encourages people to make coherent points, by instantiating large-scale story patterns and suggesting illustrative media. It performs natural language processing in real-time on a text chat between a storyteller and a viewer and recommends appropriate media items from a library. Each item is annotated with one or a few sentences in unrestricted English. A large commonsense knowledge base and a novel commonsense inference technique are used to identify story patterns such as problem and resolution or expectation violation. It uses a concept vector representation that goes beyond keyword matching or word co-occurrence based techniques. A small experiment shows that people find Raconteur's interaction design engaging, and suggestions helpful for real-time storytelling.

Supplementary Material

JPG File (fp118.jpg)

Download
9.77 KB

MP4 File (fp118.mp4)

Download
28.33 MB

References

[1]

BECKER, H., NAAMAN, M., AND GRAVANO, L. Learning Similarity Metrics for Event Identification in Social Media. In Proc. of WSDM '10: the 3rd Intl. Conf. on Web Search and Data Mining, ACM Press, New York, New York, USA, 2010.

Digital Library

[2]

BLACK, J. B. AND WILENSKY, R. An Evaluation of Story Grammars. In Cognitive Science, vol. 3 (3), pp. 213--230, 1979.

[3]

BIRD, S. KLEIN, E, LOPER, E. AND BALDRIDGE, J. Multidisciplinary Instruction with the Natural Language Toolkit. In Proc. of TeachCL '08: the 3rd Workshop on Issues in Teaching Computational Linguistics, Columbus, Ohio, USA, 2008.

Digital Library

[4]

BURKE, M., MARLOW, C., AND LENTO, T. Social Network Activity and Social Well-being. In Proc. of CHI 2010, ACM Press, Atlanta, Georgia, USA, 2010.

Digital Library

[5]

CASSELL, J. Embodied Conversational Agents: Representation and Intelligence in User Interfaces. In AI Magazine, vol. 22 (4), pp. 67--84, 2001.

Digital Library

[6]

CESAR, P., BULTERMAN, D., JANSEN, J., GEERTS, D., KNOCHE, H., AND SEAGER W. Fragment, Tag, Enrich, and Send: Enhancing Social Sharing of Video. In TOMCCAP: Transactions on Multimedia Computing, Communications, and Applications, vol. 5 (3), 2009.

Digital Library

[7]

CHI, P.-Y. AND LIEBERMAN, H. Raconteur: From Intent to Stories. In Proc. of IUI 2010, ACM Press, Hong Kong, China, 2010.

Digital Library

[8]

CHURCHILL, E., GOODMAN, E., AND O'SULLIVAN, J. Mapchat: Conversing in Place. In Ext. Abs. of CHI 2008, ACM Press, Florence, Italy, 2008.

Digital Library

[9]

FELLBAUM, C. WordNet: An Electronic Lexical Database, MIT Press, Cambridge, MA, USA, 1998.

[10]

GARFINKEL, H. Common Sense Knowledge of Social Structures: The Documentary Method of Interpretation in Lay and Professional Fact Finding. In Studies in Ethnomethodology, Polity Press, pp. 76--103, 1967.

[11]

GRAESSER, A.C., VANLEHN, K., ROSÉ, C.P., AND JORDAN, P.W. Intelligent Tutoring Systems with Conversational Dialogue. In AI Magazine, vol. 22 (4), pp. 39--51, 2001.

Digital Library

[12]

KIRK, D., SELLEN, A., ROTHER, C., AND WOOD, K. Understanding Photowork. In Proceedings of CHI 2006, ACM Press, Montréal, Québec, Canada, 2006.

Digital Library

[13]

KUIPERS, B. Commonsense Reasoning About Causality: Deriving Behavior from Structure. In Artificial Intelligence, vol. 24, pp. 169--203, 1984.

Digital Library

[14]

KUMAR, R, MAHDIAN, M., AND MCGLOHON, M. Dynamics of Conversations. In Proc. of KDD '10: the 16th ACM SIGKDD Intl. Conf. on Knowledge Discovery and Data Mining, New York, NY, USA, 2010.

Digital Library

[15]

KURLANDER, D., SKELLY, T., AND SALESIN, D. Comic Chat. In Proc. of SIGGRAPH 1996, ACM Press, New York, NY, USA, pp. 225--236, 1996.

Digital Library

[16]

LABOV, W. AND WALETZKY, J. Narrative Analysis: Oral Versions of Personal Experience. In Essays on the Verbal and Visual Arts, University of Washington Press, pp. 12--44, 1967.

[17]

LABOV, W. Some Further Steps in Narrative Analysis. In Journal of Narrative and Life History, vol. 7 (1-4), pp. 395--415, 1997.

[18]

LINDE, C. Life Stories: The Creation of Coherence, Oxford University Press, 1993.

[19]

LIU, H. AND SINGH, P. ConceptNet: a Practical Commonsense Reasoning Toolkit. In BT Technology Journal, vol 22 (4), pp. 211--226, 2004.

Digital Library

[20]

LIU Y., SHAFTON, P. SHAMMA, D., AND YANG, J. Zync: the Design of Synchronized Video Sharing. In Proc. of DUX '07: the 2007 Conf. on Designing for User Experiences, ACM Press, Chicago, IL, 2007.

Digital Library

[21]

POLANYI, L. Telling the American Story: A Structural and Cultural Analysis of Conversational Storytelling, MIT Press, 1989.

[22]

RAFFLE, H., BALLAGAS, R., REVELLE, G., HORII, H., FOLLMER, S., GO, J., REARDON, E., MORI, K., KAYE, J. AND SPASOJEVIC M. Family Story Play: Reading with Young Children (and Elmo) Over a Distance. In Proc. of CHI 2010, ACM Press, Atlanta, GA, USA, 2010.

Digital Library

[23]

SCHANK, R. C. Explanation Patterns: Understanding Mechanically and Creatively, Psychology Press, 1986.

Digital Library

[24]

SCHANK, R. C. Tell Me a Story: A New Look at Real and Artificial Intelligence, Northwestern University Press, 1991.

Digital Library

[25]

SHAMMA, D., SHAW, R., SHAFTON, P, AND LIU, Y. Watch What I Watch: Using Community Activity to Understand Content. In Proc. of MIR '07: the Intl. Workshop on Multimedia Information Retrieval, Ausburg, Bavaria, German, 2007.

Digital Library

[26]

SINGH, P., LIN, T., MUELLER, E. T., LIM, G., PERKINS, T. AND ZHU, W.-L. Open Mind Common Sense: Knowledge acquisition from the general public. In Proc. of the 1st Intl. Conf. on Ontologies, Databases, and Applications of Semantics for Large Scale Information Systems, Irvine, CA, USA, 2002.

Digital Library

[27]

SPEER, R., HAVASI, C., AND LIEBERMAN, H. AnalogySpace: Reducing the Dimensionality of Common Sense Knowledge. In Proc. of AAAI2008, AAAI Press, Chicago, IL, USA, 2008.

Digital Library

[28]

SPIERLING, U. AND IURGEL, I. Just Talking about Art-Creating Virtual Storytelling Experiences in Mixed Reality. In Proc. of ICVS '03: the 2nd Intl. Conf. on Virtual Storytelling, Toulouse, France, 2003.

[29]

STEIN, A., GULLA, J.A., MÜLLER, A., AND THIEL, U. Conversational Interaction for Semantic Access to Multimedia Information. In Intelligent Multimedia Information Retrieval, MIT Press, pp. 399--421, 1997.

Digital Library

Cited By

Huey HLeake MAneja DFisher MFan J(2024)How do video content creation goals impact which concepts people prioritize for generating B-roll imagery?Proceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3664252(542-549)Online publication date: 23-Jun-2024
https://dl.acm.org/doi/10.1145/3635636.3664252
Liu XKirilyuk VYuan XOlwal AChi PChen XDu R(2023)Visual Captions: Augmenting Verbal Communication with On-the-fly VisualsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581566(1-20)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581566
Kim HZhao SLiu CHara K(2020)LiveSnippets: Voice-based Live Authoring of Multimedia Articles about Experiences22nd International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/3379503.3403556(1-11)Online publication date: 5-Oct-2020
https://dl.acm.org/doi/10.1145/3379503.3403556
Show More Cited By

Index Terms

Intelligent assistance for conversational storytelling using story patterns
1. Human-centered computing

Recommendations

Raconteur: integrating authored and real-time social media
CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Social media enables people to share personal experiences, often through real-time media such as chat. People also record their life experiences in media collections, with photos and video. However, today's social media force a choice between real-time ...
Raconteur: from intent to stories
IUI '10: Proceedings of the 15th international conference on Intelligent user interfaces

When editing a story from a large collection of media, such as photos and video clips captured from daily life, it is not always easy to understand how particular scenes fit into the intent for the overall story. Especially for novice editors, there is ...
Story Envisioning Framework for Visualized Collective Storytelling in Conversation
Human Interface and the Management of Information. Visual Information and Knowledge Management
Abstract
In this paper, we introduce the method of story envisioning as an efficient approach to collective storytelling in daily conversation and training procedure. The goal of story envisioning is to provide intuitive, concrete and consistent ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '11: Proceedings of the 16th international conference on Intelligent user interfaces

February 2011

504 pages

ISBN:9781450304191

DOI:10.1145/1943403

General Chairs:
Pearl Pu
EPFL, Switzerland
,
Michael Pazzani
Rutgers University, USA
,
Program Chairs:
Elisabeth André
Augsburg University, Germany
,
Doug Riecken
CCLS Columbia University, USA

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 February 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IUI '11

Sponsor:

IUI '11: 16th International Conference on Intelligent User Interfaces

February 13 - 16, 2011

CA, Palo Alto, USA

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
644
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)1

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Huey HLeake MAneja DFisher MFan J(2024)How do video content creation goals impact which concepts people prioritize for generating B-roll imagery?Proceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3664252(542-549)Online publication date: 23-Jun-2024
https://dl.acm.org/doi/10.1145/3635636.3664252
Liu XKirilyuk VYuan XOlwal AChi PChen XDu R(2023)Visual Captions: Augmenting Verbal Communication with On-the-fly VisualsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581566(1-20)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581566
Kim HZhao SLiu CHara K(2020)LiveSnippets: Voice-based Live Authoring of Multimedia Articles about Experiences22nd International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/3379503.3403556(1-11)Online publication date: 5-Oct-2020
https://dl.acm.org/doi/10.1145/3379503.3403556
Völkel SSchneegass CEiband MBuschek DPaternò FOliver NConati CSpano LTintarev N(2020)What is "intelligent" in intelligent user interfaces?Proceedings of the 25th International Conference on Intelligent User Interfaces10.1145/3377325.3377500(477-487)Online publication date: 17-Mar-2020
https://dl.acm.org/doi/10.1145/3377325.3377500
Zancanaro MStock OSchiavo GCappelletti AGehrmann SCanetti DShaked OFachter SYifat RMimran RWeiss P(2019)Evaluating an automated mediator for joint narratives in a conflict situationBehaviour & Information Technology10.1080/0144929X.2019.163794039:9(1022-1037)Online publication date: 9-Jul-2019
https://doi.org/10.1080/0144929X.2019.1637940
Bennett PHinder HKozar SBowdler CMassung ECole TManchester HCater KBegole BKim JInkpen KWoo W(2015)TopoTilesProceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems10.1145/2702613.2732918(911-916)Online publication date: 18-Apr-2015
https://dl.acm.org/doi/10.1145/2702613.2732918
Kim JDontcheva MLi WBernstein MSteinsapir DBegole BKim JInkpen KWoo W(2015)MotifProceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems10.1145/2702123.2702507(1211-1220)Online publication date: 18-Apr-2015
https://dl.acm.org/doi/10.1145/2702123.2702507
Cambria EHussain ACambria EHussain A(2015)ConclusionSentic Computing10.1007/978-3-319-23654-4_5(155-160)Online publication date: 2015
https://doi.org/10.1007/978-3-319-23654-4_5
Cambria EHussain ACambria EHussain A(2015)Sentic ApplicationsSentic Computing10.1007/978-3-319-23654-4_4(107-153)Online publication date: 2015
https://doi.org/10.1007/978-3-319-23654-4_4
Cambria EHussain ACambria EHussain A(2015)Sentic PatternsSentic Computing10.1007/978-3-319-23654-4_3(73-106)Online publication date: 2015
https://doi.org/10.1007/978-3-319-23654-4_3
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents