[go: up one dir, main page]

Skip to main content

Historical Printed Ornaments: Dataset and Tasks

  • Conference paper
  • First Online:
Document Analysis and Recognition - ICDAR 2024 (ICDAR 2024)

Abstract

This paper aims to develop the study of historical printed ornaments with modern unsupervised computer vision. We highlight three complex tasks that are of critical interest to book historians: clustering, element discovery, and unsupervised change localization. For each of these tasks, we introduce an evaluation benchmark, and we adapt and evaluate state-of-the-art models. Our Rey’s Ornaments dataset is designed to be a representative example of a set of ornaments historians would be interested in. It focuses on an XVIIIth century bookseller, Marc-Michel Rey, providing a consistent set of ornaments with a wide diversity and representative challenges. Our results highlight the limitations of state-of-the-art models when faced with real data and show simple baselines such as k-means or congealing can outperform more sophisticated approaches on such data. Our dataset and code can be found at https://printed-ornaments.github.io/.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Akcay, S., Atapour-Abarghouei, A., Breckon, T.P.: GANomaly: semi-supervised anomaly detection via adversarial training. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11363, pp. 622–637. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20893-6_39

    Chapter  Google Scholar 

  2. Bahier-Porte, C., et al.: Regions of interest to investigate after learning the use of ornaments by marc-michelrey. In: Conference Abstracts of Digital Humanities 2022, Pannel Computer Vision for the Study of Printers’ Ornaments and Illustrations in European Hand-Press Books, pp. 66–67 (2022)

    Google Scholar 

  3. Bahier-Porte, C.: « l’expérience me l’apprend »: Marc michel rey et la presse en hollande. In: University of Toronto Quarterly, vol. 89 (4), pp. 731–746 (2021)

    Google Scholar 

  4. Baudrier, E., Busson, S., Corsini, S., Delalandre, M., Landré, J., , Morain-Nicolier, F.: Retrieval of the ornaments from the hand-press period: an overview. In: 10th International Conference on Document Analysis and Recognition, pp. 496–500. IEEE (2009)

    Google Scholar 

  5. Baur, C., Wiestler, B., Albarqouni, S., Navab, N.: Deep autoencoding models for unsupervised anomaly segmentation in brain MR images. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11383, pp. 161–169. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11723-8_16

    Chapter  Google Scholar 

  6. Bergel, G., Franklin, A., Heaney, M., Arandjelovic, R., Zisserman, A., Funke, D.: Content-based image recognition on printed broadside ballads: the Bodleian libraries’ imagematch tool. In: Proceedings of the IFLA World Library and Information Congress (2013)

    Google Scholar 

  7. Bergmann, P., Fauser, M., Sattlegger, D., Steger, C.: Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9592–9600 (2019)

    Google Scholar 

  8. Bigün, J., Bhattacharjee, S., Michel, S.: Orientation radiograms for image retrieval: an alternative to segmentation. In: Proceedings of the ICPR, pp. 346–350 (1996)

    Google Scholar 

  9. Bodleian ballads: database of woodcuts. http://balladsblog.bodleian.ox.ac.uk/blog/1069

  10. Burgess, C.P., et al.: MONet: Unsupervised Scene Decomposition and Representation. arXiv preprint arXiv:1901.11390 [cs, stat] (Jan 2019)

  11. Cao, L., Fei-Fei, L.: Spatially coherent latent topic model for concurrent object segmentation and classification. In: ICCV (2007)

    Google Scholar 

  12. Caron, M., Bojanowski, P., Joulin, A., Douze, M.: Deep clustering for unsupervised learning of visual features (ECCV). arXiv preprint arXiv:1807.05520 [Cs] (2018)

  13. Caron, M., et al.: Emerging properties in self-supervised vision transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9650–9660 (2021)

    Google Scholar 

  14. Chaki, S., Steinlin, S., Emonet, R., Fournel, T.: One-to-many pattern comparison combining fully-connected autoencoder with spatial transformer for ornament investigation (2023). https://doi.org/10.21203/rs.3.rs-3573134/v1

  15. Cho, M., Kwak, S., Schmid, C., Ponce, J.: Unsupervised object discovery and localization in the wild. In: CVPR (2015)

    Google Scholar 

  16. Chung, J.S., Arandjelovic, R., Bergel, G., Franklin, A., Zisserman, A.: Re-presentations of art collections. In: Workshop on Computer Vision for Art Analysis (Visart), ECCV (2014)

    Google Scholar 

  17. Compositor: database of ornaments. https://compositor.bham.ac.uk

  18. Corsini, S.: La preuve par les fleurons: analyse comparée du matériel ornemental des imprimeurs suisses romands, 1775–1785. In: Centre international d’étude du XVIIIe siècle (1999)

    Google Scholar 

  19. Corsini, S.: « passe-partout : banque internationale d’ornements d’imprimerie. Bulletin des bibliothèques de France 5, 73 (2001)

    Google Scholar 

  20. Corsini, S.: Vers un corpus des ornements typographiques lausannois du xviiie siècle : problèmes de définition et de méthode. In: Ornementation typographique et bibliographie historique. vol. Mons et Bruxelles, Van Balberghe, pp. 139–158 (1988)

    Google Scholar 

  21. Cox, M., Sridharan, S., Lucey, S., Cohn, J.: Least squares congealing for unsupervised alignment of images. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008). https://doi.org/10.1109/CVPR.2008.4587573

  22. Crawford, E., Pineau, J.: Spatially invariant unsupervised object detection with convolutional neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3412–3420 (2019)

    Google Scholar 

  23. Dutta, A., Bergel, G., Zisserman, A.: Visual analysis of chapbooks printed in Scotland. In: The 6th International Workshop on Historical Document Imaging and Processing, pp. 67–72 (2021)

    Google Scholar 

  24. Engelcke, M., Kosiorek, A.R., Jones, O.P., Posner, I.: GENESIS: generative scene inference and sampling with object-centric latent representations. In: ICLR (2020)

    Google Scholar 

  25. Engelcke, M., Parker Jones, O., Posner, I.: Genesis-v2: inferring unordered object representations without iterative refinement. Adv. Neural. Inf. Process. Syst. 34, 8085–8094 (2021)

    Google Scholar 

  26. Enschedé, J.: Proef van letteren, welke gegooten worden in de nieuwe haerlemsche lettergietery van j. enschedé (1768). https://gallica.bnf.fr/ark:/12148/bpt6k328783b

  27. Eslami, S., et al.: Attend, infer, repeat: Fast scene understanding with generative models. Adv. Neural Inf. Process. Syst. 29 (2016)

    Google Scholar 

  28. Everingham, M., Gool, L.V., Williams, C.K.I., Winn, J.M., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010). http://dblp.uni-trier.de/db/journals/ijcv/ijcv88.html

  29. Fiorucci, M., Khoroshiltseva, M., Pontil, M., Traviglia, A., Del Bue, A., James, S.: Machine learning for cultural heritage: a survey. Pattern Recogn. Lett. 133, 102–108 (2020). https://doi.org/10.1016/j.patrec.2020.02.017

  30. Fleuron: database of printing ornaments. https://db-prod-bcul.unil.ch/ornements/scripts/index.html

  31. Fournier, P.S.: Les caractéres de l’imprimerie (1764). https://gallica.bnf.fr/ark:/12148/bpt6k15021752/

  32. Frey, B., Jojic, N.: Transformation-invariant clustering using the em algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 25, 1– 17 (2003). https://doi.org/10.1109/TPAMI.2003.1159942

  33. Frey, B.J., Jojic, N.: Fast, large-scale transformation-invariant clustering. In: Dietterich, T., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems, vol. 14. MIT Press (2001). https://proceedings.neurips.cc/paper_files/paper/2001/file/95f6870ff3dcd442254e334a9033d349-Paper.pdf

  34. Goyal, K., Dyer, C., Warren, C.N., G’Sell, M.G., Berg-Kirkpatrick, T.: A probabilistic generative model for typographical analysis of early modern printing. In: Annual Meeting of the Association for Computational Linguistics (2020). https://api.semanticscholar.org/CorpusID:218486915

  35. Goyette, N., Jodoin, P.M., Porikli, F., Konrad, J., Ishwar, P.: Changedetection.net: a new change detection benchmark dataset. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–8. IEEE (2012)

    Google Scholar 

  36. Grauman, K., Darrell, T.: Unsupervised learning of categories from sets of partially matching image features. In: CVPR (2006)

    Google Scholar 

  37. Greff, K., et al.: Multi-object representation learning with iterative variational inference. In: ICML (2019)

    Google Scholar 

  38. Guo, X., Gao, L., Liu, X., Yin, J.: Improved deep embedded clustering with local structure preservation. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pp. 1753–1759 (2017). https://doi.org/10.24963/ijcai.2017/243

  39. He, S., Samara, P., Burgers, J., Schomaker, L.: A multiple-label guided clustering algorithm for historical document dating and localization. IEEE Trans. Image Process. 25, 5252–5265 (2016). https://api.semanticscholar.org/CorpusID:16772542

  40. Hsu, C.C., Lin, C.W.: CNN-based joint clustering and representation learning with feature drift compensation for large-scale image data. IEEE Trans. Multimedia 20(2), 421–429 (2017)

    Article  Google Scholar 

  41. Hu, W., Miyato, T., Tokui, S., Matsumoto, E., Sugiyama, M.: Learning discrete representations via information maximizing self-augmented training. In: International Conference on Machine Learning, pp. 1558–1567. PMLR (2017)

    Google Scholar 

  42. Huang, D., Chen, D., Chen, X., Wang, C., Lai, J.: Deepclue: enhanced image clustering via multi-layer ensembles in deep neural networks. arXiv preprint arXiv:2206.00359 (2022)

  43. Jaderberg, M., Simonyan, K., Zisserman, A.: Spatial transformer networks. Adv. Neural Inf. Process. Syst. 28 (2015)

    Google Scholar 

  44. Ji, X., Henriques, J.F., Vedaldi, A.: Invariant information clustering for unsupervised image classification and segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 9865–9874 (2019)

    Google Scholar 

  45. Jiang, J., Ahn, S.: Generative neurosymbolic machines. Adv. Neural. Inf. Process. Syst. 33, 12572–12582 (2020)

    Google Scholar 

  46. Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C.L., Girshick, R.: CLEVR: a diagnostic dataset for compositional language and elementary visual reasoning. In: CVPR (2017)

    Google Scholar 

  47. Joulin, A., Bach, F., Ponce, J.: Discriminative clustering for image co-segmentation. In: CVPR (2010)

    Google Scholar 

  48. Kabra, R., et al.: Multi-object datasets (2019). https://github.com/deepmind/multi_object_datasets/

  49. Karazija, L., Laina, I., Rupprecht, C.: Clevrtex: a texture-rich benchmark for unsupervised multi-object segmentation. In: Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) (2021)

    Google Scholar 

  50. Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: 2nd International Conference on Learning Representations, ICLR 2014, Banff, 14–16 April 2014, Conference Track Proceedings (2014)

    Google Scholar 

  51. Kosiorek, A.R., Sabour, S., Teh, Y.W., Hinton, G.E.: Stacked capsule autoencoders. Adv. Neural Inf. Process. Syst. 23 (2019)

    Google Scholar 

  52. Kuhn, H.W.: The Hungarian method for the assignment problem. Naval Res. Logist. (NRL) 52 (1955). https://api.semanticscholar.org/CorpusID:9426884

  53. Learned-Miller, E.G.: Data driven image models through continuous joint alignment. IEEE Trans. Pattern Anal. Mach. Intell. 28(2), 236–250 (2005)

    Article  Google Scholar 

  54. Lesjak, Ž., et al.: A novel public MR image dataset of multiple sclerosis patients with lesion segmentations based on multi-rater consensus. Neuroinformatics 16, 51–63 (2018)

    Google Scholar 

  55. Lin, Z., et al.: Space: unsupervised object-oriented scene representation via spatial attention and decomposition. In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=rkl03ySYDH

  56. Liu, W., et al.: Towards visually explaining variational autoencoders. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8642–8651 (2020)

    Google Scholar 

  57. MacQueen, J., et al.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Oakland, vol. 1, pp. 281–297 (1967)

    Google Scholar 

  58. Maguelone: database of typographical ornaments. http://maguelone.enssib.fr

  59. McKenna, A., Mori, G.: Claude-françois simon, imprimeur, dit « poppy », contrefacteur de marc michel rey (2019). https://mmrey.hypotheses.org/778

  60. Metaxas, I.M., Tzimiropoulos, G., Patras, I.: Divclust: Controlling diversity in deep clustering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3418–3428 (2023)

    Google Scholar 

  61. Monnier, T., Groueix, T., Aubry, M.: Deep transformation-invariant clustering. In: NeurIPS (2020)

    Google Scholar 

  62. Monnier, T., Vincent, E., Ponce, J., Aubry, M.: Unsupervised layered image decomposition into object prototypes. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8640–8650 (2021)

    Google Scholar 

  63. Naumann, A., Hertlein, F., Dörr, L., Furmans, K.: Tampar: visual tampering detection for parcel logistics in postal supply chains. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 8076–8086 (2024)

    Google Scholar 

  64. Pang, G., Shen, C., Cao, L., Hengel, A.V.D.: Deep learning for anomaly detection: a review. ACM Comput. Surv. 54(2), 1–38 (2021)

    Article  Google Scholar 

  65. Pimentel, M.A., Clifton, D.A., Clifton, L., Tarassenko, L.: A review of novelty detection. Signal Process. 99, 215–249 (2014)

    Article  Google Scholar 

  66. Radford, A., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning, pp. 8748–8763. PMLR (2021)

    Google Scholar 

  67. Rey database: database of publishing ornaments, ROIi. https://heurist.huma-num.fr/heurist/?db=ROIi

  68. Riffaud, A., Pantin, I.: Une archéologie du livre français moderne. Droz (2011)

    Google Scholar 

  69. Rosart, J.: Epreuves des caractères qui se gravent et se fondent dans la nouvelle fonderie de jacques françois rosart (1761)

    Google Scholar 

  70. Rubinstein, M., Joulin, A., Kopf, J., Liu, C.: Unsupervised joint object discovery and segmentation in internet images. In: CVPR (2013)

    Google Scholar 

  71. Ruff, L., et al.: A unifying review of deep and shallow anomaly detection. Proc. IEEE 109(5), 756–795 (2021)

    Google Scholar 

  72. Russell, B.C., Freeman, W.T., Efros, A.A., Sivic, J., Zisserman, A.: Using multiple segmentations to discover objects and their extent in image collections. In: CVPR (2006)

    Google Scholar 

  73. Sauvalle, B., de La Fortelle, A.: Unsupervised multi-object segmentation using attention and soft-argmax. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3267–3276 (2023)

    Google Scholar 

  74. Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., Langs, G.: Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: International Conference on Information Processing in Medical Imaging, pp. 146–157. Springer (2017)

    Google Scholar 

  75. Shen, L., et al.: S2looking: a satellite side-looking dataset for building change detection. Remote Sens. 13(24), 5094 (2021)

    Google Scholar 

  76. Shen, X., Efros, A.A., Aubry, M.: Discovering visual patterns in art collections with spatially-consistent feature learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)

    Google Scholar 

  77. Siméoni, O., et al.: Localizing objects with self-supervised transformers and no labels. In: Proceedings of the British Machine Vision Conference (BMVC) (2021)

    Google Scholar 

  78. Sivic, J., Russell, B.C., Zisserman, A., Freeman, W.T., Efros, A.A.: Unsupervised discovery of visual object class hierarchies. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)

    Google Scholar 

  79. Smirnov, D., Gharbi, M., Fisher, M., Guizilini, V., Efros, A.A., Solomon, J.: MarioNette: Self-Supervised Sprite Learning. arXiv preprint arXiv:2104.14553 [cs] (2021)

  80. Van Gansbeke, W., Vandenhende, S., Georgoulis, S., Proesmans, M., Van Gool, L.: Scan: learning to classify images without labels. In: Proceedings of the European Conference on Computer Vision (2020)

    Google Scholar 

  81. Venkataramanan, S., Peng, K.-C., Singh, R.V., Mahalanobis, A.: Attention guided anomaly localization in images. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12362, pp. 485–503. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58520-4_29

    Chapter  Google Scholar 

  82. Vo, H.V., et al.: Unsupervised image matching and object discovery as optimization. In: CVPR (2019)

    Google Scholar 

  83. Wilkinson, H., Briggs, J., Gorissen, D.: Computer vision and the creation of a database of printers’ ornaments. Digital Humanities Quarterly (2021)

    Google Scholar 

  84. Wilkinson, H.: ‘printers’ flowers as evidence in the identification of unknown printers: two examples from 1715. In: The Library, 7th Series, vol. 14, pp. 70–79 (2013)

    Google Scholar 

  85. Xie, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: International Conference on Machine Learning, pp. 478–487. PMLR (2016)

    Google Scholar 

  86. Yang, B., Fu, X., Sidiropoulos, N.D., Hong, M.: Towards k-means-friendly spaces: simultaneous deep learning and clustering. In: International Conference on Machine Learning, pp. 3861–3870. PMLR (2017)

    Google Scholar 

  87. Yang, Y., Yang, B.: Promising or elusive? Unsupervised object segmentation from real-world single images. Adv. Neural. Inf. Process. Syst. 35, 4722–4735 (2022)

    Google Scholar 

  88. Zenati, H., Foo, C.S., Lecouat, B., Manek, G., Chandrasekhar, V.R.: Efficient gan-based anomaly detection. arXiv preprint arXiv:1802.06222 (2018)

  89. Zhu, W., Shen, Y., Yu, L., Aguirre Sanchez, L.P.: Gmair: unsupervised object detection based on spatial attention and gaussian mixture. arXiv preprints, pp. arXiv–2106 (2021)

    Google Scholar 

Download references

Acknowledgement

This work was funded by ANR ROIi project ANR-20-CE38-0005. S. Baltaci, E. Vincent, and M. Aubry were supported by ERC project DISCOVER funded by the European Union’s Horizon Europe Research and Innovation program under grant agreement No. 101076028 and ANR VHS project ANR-21-CE38-0008. We thank Silya Ounoughi, Thomas Gautrais, and Vincent Ventresque for their work in the collection and annotation of the datasets, and Ségolène Albouy, Raphaël Baena, Syrine Kalleli, Ioannis Siglidis, Gurjeet Sangra Singh, Andrea Morales Garzón and Malamatenia Vlachou for valuable feedbacks.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Sayan Kumar Chaki or Zeynep Sonat Baltaci .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chaki, S.K. et al. (2024). Historical Printed Ornaments: Dataset and Tasks. In: Barney Smith, E.H., Liwicki, M., Peng, L. (eds) Document Analysis and Recognition - ICDAR 2024. ICDAR 2024. Lecture Notes in Computer Science, vol 14806. Springer, Cham. https://doi.org/10.1007/978-3-031-70543-4_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-70543-4_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-70542-7

  • Online ISBN: 978-3-031-70543-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics