[go: up one dir, main page]

Skip to main content

Intelligent Diagrammatic Interfaces: State of the Art

  • Chapter
Diagrammatic Representation and Reasoning

Abstract

One important practical role for diagrammatic reasoning is to provide the foundation for intelligent diagrammatic environments (IDEs), in which structured diagrams are used as a means of natural visual communication between the human user and the system. The present chapter provides a characterisation of IDEs and gives an overview of the state of the art. Rather than attempting an exhaustive survey, we discuss the subtasks that have to be addressed in a general IDE framework and show how they are handled in representative systems from the following IDE application areas:

  • Document understanding

  • Diagramming support

  • Diagrammatic query languages for spatial information systems

  • Diagrammatic theorem provers

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Ah-Soon, C. (1998). A constraint network for symbol detection in architectural drawings. In K. Tombre and A. Chhabra (Eds), Graphics Recognition: Algorithms and systems, Vol. 1389 of LNCS. Berlin: Springer, pp. 80–90.

    Google Scholar 

  2. Baird, H. (1990). Industrial applications. In H. Bunke and A. Sanfeliu (Eds), Syntactic and structural pattern recognition: Theory and applications. World Scientific, pp. 369–380.

    Google Scholar 

  3. Baker, S., Ireland, A. and Smaill, A. (1992). On the use of the constructive omega rule within automated deduction. In A. Voronkov (Ed.), Logic programming and automated reasoning. LNAI 624. Berlin: Springer, pp. 214–225.

    Chapter  Google Scholar 

  4. Barker-Plummer, D. and Bailin, S. (1992). Proving the diamond lemma with the GROVER theorem proving system. In AAAI symposium on reasoning with diagrammatic representations.

    Google Scholar 

  5. Barwise, J. and Etchemendy, J. (1996). Visual information and valid reasoning. In G. Allwein and J. Barwise (Eds), Logical reasoning with diagrams. Oxford: Oxford University Press, pp. 3–26.

    Google Scholar 

  6. Borning, A., Marriott, K., Stuckey, P. and Xiao, Y. (1997). Solving linear arithmetic constraints for user interface applications. In UIST’97: ACM symposium on user interface software and technology.

    Google Scholar 

  7. Brachman, R.J. and Schmolze, J.G. (1985). An overview of KL-ONE knowledge representation systems. Cognitive Science August:171–216.

    Google Scholar 

  8. Catarci, T. Costabile, M.F., Levialdi, S. and Batini, C. (1997). Visual query systems for databases: A survey. Journal of Visual Languages and Computing

    Google Scholar 

  9. Chok, S. and Marriott, K. (1995). Automatic construction of user interfaces from constraint multiset grammars. In IEEE symposium on visual languages. Los Alamitos, CA: IEEE Computer Society Press, pp. 242–250.

    Google Scholar 

  10. Chok, S. and Marriott, K. (1998a). Automatic construction of intelligent diagram editors. In Proceedings of the 11th ACM symposium on user interface software and technology, pp. 185–194.

    Google Scholar 

  11. Chok, S. and Marriott, K. (1998b). Automatic construction of intelligent diagram editors. In Proceedings of the 11th ACM symposium on user interface software and technology.

    Google Scholar 

  12. Clarke, B. (1981). A calculus of individuals based on connection. Notre Dame Journal of Formal Logic 23(3):204–218.

    Article  Google Scholar 

  13. Clarke, B. (1985). Individuals and points. Notre Dame Journal of Formal Logic 26(1):61–75.

    Article  MathSciNet  MATH  Google Scholar 

  14. Clementini, E., DiFelice, P. and van Oosterom, P. (1993). A small set of formal topological relationships suitable for end-user interaction. In D. Abel and B. Ooi (Eds), International symposium on advances in spatial databases (SSD’93), Singapore, June, pp. 277–295.

    Google Scholar 

  15. Cohn, A.G. (1997). Qualitative spatial representation and reasoning techniques. In Künstliche Intelligenz 1997 (KI’97). LNAI 1303. Springer, pp. 1–30.

    Google Scholar 

  16. Costagliola, G., De Lucia, A., Orefice, S. and Tortora, G. (1998). Positional grammars: Theory and practice in a visual language for interface modelling. In K. Marriott and B. Meyer (Eds), Visual language theory. New York: Springer, pp. 171–191.

    Chapter  Google Scholar 

  17. Dinesh, T. and Usküdarli, S. (1998). Input and output for specified visual languages. In K. Marriott and B. Meyer (Eds), Visual language theory. New York: Springer, pp. 325–351.

    Chapter  Google Scholar 

  18. Egenhofer, M. (1996). Spatial-query-by-sketch. In IEEE sypmosium on visual languages, pp. 170–177.

    Google Scholar 

  19. Egenhofer, M. and Franzosa, R. (1991). Point-set topological spatial relations. International Journal of Geographical Information Systems 5(2):161–174.

    Article  Google Scholar 

  20. Faloutsos, C., Barber, R., Flickner, M., Hafner, J., Niblack, W., Petrovic, D. and Equitz, W. (1994). Efficient and effective querying by image content. Journal of Intelligent Information Systems 3:231–262.

    Article  Google Scholar 

  21. Fumas, G. (1990). Formal models for imaginal deduction. In Twelfth annual conference of the Cognitive Science Society. Hillsdale, NJ: Erlbaum, pp. 662–669.

    Google Scholar 

  22. Fumas, G. (1991). New graphical reasoning models for understanding graphical interfaces. In Proceedings of the ACM conference on human factors in computing systems (CHI). New York: ACM Press, pp. 71–78.

    Google Scholar 

  23. Fumas, G. (1992). Reasoning with diagrams only. In AAAI spring symposium on reasoning with diagrammatic representations, Stanford.

    Google Scholar 

  24. Futrelle, R. and Nikolakis, N. (1995). Efficient analysis of complex diagrams using constraint-based parsing. In Proceedings of the third international conference on document analysis and recognition (ICDAR), pp. 782–790.

    Google Scholar 

  25. Gelernter, H. (1963). Realization of a geometry theorem-proving machine. In E. Feigenbaum and J. Feldman (Eds), Computers and thought. New York: McGraw-Hill, pp. 134–152..

    Google Scholar 

  26. Gooday, J. and Cohn, A. (1996). Using spatial logic to describe visual programming languages. Artificial Intelligence Review 10:171–186.

    Article  Google Scholar 

  27. Gross, M. (1994a). Recognizing and interpreting diagrams in design. In T. Catarci, M. Costabile, S. Levialdi and G. Santucci (Eds), Advanced visual interfaces (AVI). New York: ACM Press.

    Google Scholar 

  28. Gross, M. (1994b). Stretch-a-sketch: A dynamic diagrammer. In Proceedings of the 1994 IEEE symposium on visual languages. Los Alamitos, CA: IEEE Computer Society Press, pp. 232–238:

    Google Scholar 

  29. Gross, M. and Do, E.-L. (1996). Ambiguous intentions: a paper-like interface for creative design. In ACM 9th symposium on user interface software and technology (UIST). ACM Press, pp. 183–192.

    Google Scholar 

  30. Haarslev, V. (1998). A fully formalized theory for describing visual notations. In K. Marriott and B. Meyer (Eds), Visual language theory. New York: Springer, pp. 261–192g.

    Google Scholar 

  31. Haarslev, V. and Wessel, M. (1997). Querying GIS with animated spatial sketches. In IEEE sypmosium on visual languages, Capri, Italy, pp. 201–208.

    Google Scholar 

  32. Helm, R. and Marriott, K. (1986). Declarative graphics. In Proceedings of the 3rd international conference on logic programming, Vol. 225 of LNCS. New York: Springer, pp. 513–527.

    Google Scholar 

  33. Helm, R., Marriott, K., Huynh, T. and Vlissides, J. (1995). An object-oriented architecture for constraint-based graphical editing. In C. Laffra, E. Blake, V. de Mey and X. Pintado (Eds), Object-oriented programming for graphics. Berlin: Springer, pp. 217–238.

    Chapter  Google Scholar 

  34. Jamnik, M., Bundy, A. and Green, I. (1997). Automation of diagrammatic proofs in mathematics. In B. Kokinov (Ed.), Perspectives on cognitive science, Vol. 3. New Bulgarian University, pp. 168–175.

    Google Scholar 

  35. Karima, M., Sadhal, K. and McNeil, T. (1995). From paper drawings to computer aided design. IEEE Computer Graphics and Applications February:24–39.

    Google Scholar 

  36. Kasturi, R., Raman, R., Chennubthotla, C. and O’Gorman, L. (1992). An overview of techniques for graphics recognition. In H. Baird, H. Bunke, and K. Yamamoto (Eds), Structured document image analysis. Berlin: Springer, pp. 285–324.

    Chapter  Google Scholar 

  37. Kato, T. (1992). Cognitive view mechanism for multimedia information systems. In R. Cooper (Ed.), International workshop on interfaces to database systems. LNCS. New York: Springer.

    Google Scholar 

  38. Lakin, F. (1986). Spatial parsing for visual languages. In S.-K. Chang, T. Ichikawa and P.A. Ligomenides (Eds), Visual languages. New York: Plenum Press, pp. 35–85.

    Chapter  Google Scholar 

  39. Lakin, F. (1987). Visual grammars for visual languages. In AAAI-87, 7th national conference on AI, pp. 683–688.

    Google Scholar 

  40. Marriott, K. (1994). Constraint multiset grammars. In IEEE symposium on visual languages. Los Alamitos, CA: IEEE Computer Society Press, pp. 118–125.

    Google Scholar 

  41. Marriott, K., Meyer, B. and Wittenburg, K. (1998). A survey of visual language specification and recognition. In K. Marriott and B. Meyer (Eds), Visual language theory. New York: Springer, pp. 5–85.

    Chapter  Google Scholar 

  42. Meyer, B. (1992). Beyond icons: Towards new metaphors for visual query languages for spatial information systems. In R. Cooper (Ed.), Interfaces to database systems. Glasgow: Springer, pp. 113–135.

    Google Scholar 

  43. Meyer, B. (1994). Pictorial deduction in spatial information systems. In International IEEE workshop on visual languages (VL’94), St Louis, MO, October, pp. 23–30.

    Google Scholar 

  44. Meyer, B. (1997). Formalization of visual mathematical notations. In M. Anderson (Ed.), AAAI symposium on diagrammatic reasoning (DR-II), Boston, MA, November. AAAI Technical Report FS-97–02, AAAI Press, pp. 97–02.

    Google Scholar 

  45. Meyer, B. (1999). Constraint diagram reasoning. In Principles and practice of constraint programming: CP’99, Alexandria, VA, October. Berlin: Springer.

    Google Scholar 

  46. Meyer, B. (2000). A constraint-based framework for diagrammatic reasoning. In Applied Artificial Intelligence (special issue on constraint handling rules) 14.

    Google Scholar 

  47. Meyer, B. and Marriott, K. (1997). Specifying diagram animation with rewrite systems. In International workshop on theory of visual languages (TVL’97), Capri, Italy, September, pp. 85–96.

    Google Scholar 

  48. Meyer, B. and Zweckstetter, H. (1998). Interpretation of visual notations in the recopla editor generator. In FRVDR’98: AAAI fall symposium on formalizing reasoning with visual and diagrammatic representations, Orlando, FL, October. AAAI TR FS-98–04.

    Google Scholar 

  49. Meyer, B., Zweckstetter, H., Mandel, L. and Gassmann, Z. (1999). Automatic construction of intelligent diagrammatic environments. In HCI’99: 8th international conference on human-computer interaction, Munich, Germany, August. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  50. Minas, M. and Viehstaedt, G. (1995). Diagen: A generator for diagram editors providing direct manipulation and execution of diagrams. In IEEE workshop on visual languages. Los Alamitos, CA: IEEE Computer Society Press, pp. 203–210.

    Google Scholar 

  51. Nagy, G., Seth, S. and Viswanathan, M. (1992). A prototype document image analysis system for technical journals. IEEE Computer 25(7):10–22.

    Article  Google Scholar 

  52. Rozenberg, G. (Ed.)(1997). Handbook of graph grammars and computing by graph transformation. Singapore, World Scientific.

    Google Scholar 

  53. Schneider, M. (1999). Uncertainty management for spatial data in databases: Fuzzy spatial data types. In International symposium on advances in spatial databases (SSD’99), Hong Kong. LNCS 1651, pp. 330–351. LNCS 1651.

    Google Scholar 

  54. Weitzman, L. and Wittenburg, K. (1993). Relational grammars for interactive design. In IEEE symposium on visual languages, pp. 4–11.

    Google Scholar 

  55. Wessel, M. and Haarslev, V. (1998). VISCO: Bringing visual spatial querying to reality. In IEEE sypmosium on visual languages, Halifax, Canada, pp. 170–177.

    Google Scholar 

  56. Wittenburg, K. and Weitzman, L. (1990). Visual grammars and incremental parsing for interface languages. In IEEE symposium on visual languages. Los Alamitos, CA: IEEE Computer Society Press, pp. 111–118.

    Google Scholar 

  57. Yokokura, N. and Watanabe, T. (1998). Layout-based approach for extracting constructive elements of bar-charts. In K. Tombre, and A. Chhabra (Eds),

    Google Scholar 

Download references

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag London

About this chapter

Cite this chapter

Meyer, B., Marriott, K., Allwein, G. (2002). Intelligent Diagrammatic Interfaces: State of the Art. In: Anderson, M., Meyer, B., Olivier, P. (eds) Diagrammatic Representation and Reasoning. Springer, London. https://doi.org/10.1007/978-1-4471-0109-3_23

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-0109-3_23

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-85233-242-6

  • Online ISBN: 978-1-4471-0109-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics