Abstract
This paper aims to develop the study of historical printed ornaments with modern unsupervised computer vision. We highlight three complex tasks that are of critical interest to book historians: clustering, element discovery, and unsupervised change localization. For each of these tasks, we introduce an evaluation benchmark, and we adapt and evaluate state-of-the-art models. Our Rey’s Ornaments dataset is designed to be a representative example of a set of ornaments historians would be interested in. It focuses on an XVIIIth century bookseller, Marc-Michel Rey, providing a consistent set of ornaments with a wide diversity and representative challenges. Our results highlight the limitations of state-of-the-art models when faced with real data and show simple baselines such as k-means or congealing can outperform more sophisticated approaches on such data. Our dataset and code can be found at https://printed-ornaments.github.io/.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Akcay, S., Atapour-Abarghouei, A., Breckon, T.P.: GANomaly: semi-supervised anomaly detection via adversarial training. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11363, pp. 622–637. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20893-6_39
Bahier-Porte, C., et al.: Regions of interest to investigate after learning the use of ornaments by marc-michelrey. In: Conference Abstracts of Digital Humanities 2022, Pannel Computer Vision for the Study of Printers’ Ornaments and Illustrations in European Hand-Press Books, pp. 66–67 (2022)
Bahier-Porte, C.: « l’expérience me l’apprend »: Marc michel rey et la presse en hollande. In: University of Toronto Quarterly, vol. 89 (4), pp. 731–746 (2021)
Baudrier, E., Busson, S., Corsini, S., Delalandre, M., Landré, J., , Morain-Nicolier, F.: Retrieval of the ornaments from the hand-press period: an overview. In: 10th International Conference on Document Analysis and Recognition, pp. 496–500. IEEE (2009)
Baur, C., Wiestler, B., Albarqouni, S., Navab, N.: Deep autoencoding models for unsupervised anomaly segmentation in brain MR images. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11383, pp. 161–169. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11723-8_16
Bergel, G., Franklin, A., Heaney, M., Arandjelovic, R., Zisserman, A., Funke, D.: Content-based image recognition on printed broadside ballads: the Bodleian libraries’ imagematch tool. In: Proceedings of the IFLA World Library and Information Congress (2013)
Bergmann, P., Fauser, M., Sattlegger, D., Steger, C.: Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9592–9600 (2019)
Bigün, J., Bhattacharjee, S., Michel, S.: Orientation radiograms for image retrieval: an alternative to segmentation. In: Proceedings of the ICPR, pp. 346–350 (1996)
Bodleian ballads: database of woodcuts. http://balladsblog.bodleian.ox.ac.uk/blog/1069
Burgess, C.P., et al.: MONet: Unsupervised Scene Decomposition and Representation. arXiv preprint arXiv:1901.11390 [cs, stat] (Jan 2019)
Cao, L., Fei-Fei, L.: Spatially coherent latent topic model for concurrent object segmentation and classification. In: ICCV (2007)
Caron, M., Bojanowski, P., Joulin, A., Douze, M.: Deep clustering for unsupervised learning of visual features (ECCV). arXiv preprint arXiv:1807.05520 [Cs] (2018)
Caron, M., et al.: Emerging properties in self-supervised vision transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9650–9660 (2021)
Chaki, S., Steinlin, S., Emonet, R., Fournel, T.: One-to-many pattern comparison combining fully-connected autoencoder with spatial transformer for ornament investigation (2023). https://doi.org/10.21203/rs.3.rs-3573134/v1
Cho, M., Kwak, S., Schmid, C., Ponce, J.: Unsupervised object discovery and localization in the wild. In: CVPR (2015)
Chung, J.S., Arandjelovic, R., Bergel, G., Franklin, A., Zisserman, A.: Re-presentations of art collections. In: Workshop on Computer Vision for Art Analysis (Visart), ECCV (2014)
Compositor: database of ornaments. https://compositor.bham.ac.uk
Corsini, S.: La preuve par les fleurons: analyse comparée du matériel ornemental des imprimeurs suisses romands, 1775–1785. In: Centre international d’étude du XVIIIe siècle (1999)
Corsini, S.: « passe-partout : banque internationale d’ornements d’imprimerie. Bulletin des bibliothèques de France 5, 73 (2001)
Corsini, S.: Vers un corpus des ornements typographiques lausannois du xviiie siècle : problèmes de définition et de méthode. In: Ornementation typographique et bibliographie historique. vol. Mons et Bruxelles, Van Balberghe, pp. 139–158 (1988)
Cox, M., Sridharan, S., Lucey, S., Cohn, J.: Least squares congealing for unsupervised alignment of images. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008). https://doi.org/10.1109/CVPR.2008.4587573
Crawford, E., Pineau, J.: Spatially invariant unsupervised object detection with convolutional neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3412–3420 (2019)
Dutta, A., Bergel, G., Zisserman, A.: Visual analysis of chapbooks printed in Scotland. In: The 6th International Workshop on Historical Document Imaging and Processing, pp. 67–72 (2021)
Engelcke, M., Kosiorek, A.R., Jones, O.P., Posner, I.: GENESIS: generative scene inference and sampling with object-centric latent representations. In: ICLR (2020)
Engelcke, M., Parker Jones, O., Posner, I.: Genesis-v2: inferring unordered object representations without iterative refinement. Adv. Neural. Inf. Process. Syst. 34, 8085–8094 (2021)
Enschedé, J.: Proef van letteren, welke gegooten worden in de nieuwe haerlemsche lettergietery van j. enschedé (1768). https://gallica.bnf.fr/ark:/12148/bpt6k328783b
Eslami, S., et al.: Attend, infer, repeat: Fast scene understanding with generative models. Adv. Neural Inf. Process. Syst. 29 (2016)
Everingham, M., Gool, L.V., Williams, C.K.I., Winn, J.M., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010). http://dblp.uni-trier.de/db/journals/ijcv/ijcv88.html
Fiorucci, M., Khoroshiltseva, M., Pontil, M., Traviglia, A., Del Bue, A., James, S.: Machine learning for cultural heritage: a survey. Pattern Recogn. Lett. 133, 102–108 (2020). https://doi.org/10.1016/j.patrec.2020.02.017
Fleuron: database of printing ornaments. https://db-prod-bcul.unil.ch/ornements/scripts/index.html
Fournier, P.S.: Les caractéres de l’imprimerie (1764). https://gallica.bnf.fr/ark:/12148/bpt6k15021752/
Frey, B., Jojic, N.: Transformation-invariant clustering using the em algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 25, 1– 17 (2003). https://doi.org/10.1109/TPAMI.2003.1159942
Frey, B.J., Jojic, N.: Fast, large-scale transformation-invariant clustering. In: Dietterich, T., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems, vol. 14. MIT Press (2001). https://proceedings.neurips.cc/paper_files/paper/2001/file/95f6870ff3dcd442254e334a9033d349-Paper.pdf
Goyal, K., Dyer, C., Warren, C.N., G’Sell, M.G., Berg-Kirkpatrick, T.: A probabilistic generative model for typographical analysis of early modern printing. In: Annual Meeting of the Association for Computational Linguistics (2020). https://api.semanticscholar.org/CorpusID:218486915
Goyette, N., Jodoin, P.M., Porikli, F., Konrad, J., Ishwar, P.: Changedetection.net: a new change detection benchmark dataset. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–8. IEEE (2012)
Grauman, K., Darrell, T.: Unsupervised learning of categories from sets of partially matching image features. In: CVPR (2006)
Greff, K., et al.: Multi-object representation learning with iterative variational inference. In: ICML (2019)
Guo, X., Gao, L., Liu, X., Yin, J.: Improved deep embedded clustering with local structure preservation. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pp. 1753–1759 (2017). https://doi.org/10.24963/ijcai.2017/243
He, S., Samara, P., Burgers, J., Schomaker, L.: A multiple-label guided clustering algorithm for historical document dating and localization. IEEE Trans. Image Process. 25, 5252–5265 (2016). https://api.semanticscholar.org/CorpusID:16772542
Hsu, C.C., Lin, C.W.: CNN-based joint clustering and representation learning with feature drift compensation for large-scale image data. IEEE Trans. Multimedia 20(2), 421–429 (2017)
Hu, W., Miyato, T., Tokui, S., Matsumoto, E., Sugiyama, M.: Learning discrete representations via information maximizing self-augmented training. In: International Conference on Machine Learning, pp. 1558–1567. PMLR (2017)
Huang, D., Chen, D., Chen, X., Wang, C., Lai, J.: Deepclue: enhanced image clustering via multi-layer ensembles in deep neural networks. arXiv preprint arXiv:2206.00359 (2022)
Jaderberg, M., Simonyan, K., Zisserman, A.: Spatial transformer networks. Adv. Neural Inf. Process. Syst. 28 (2015)
Ji, X., Henriques, J.F., Vedaldi, A.: Invariant information clustering for unsupervised image classification and segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 9865–9874 (2019)
Jiang, J., Ahn, S.: Generative neurosymbolic machines. Adv. Neural. Inf. Process. Syst. 33, 12572–12582 (2020)
Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C.L., Girshick, R.: CLEVR: a diagnostic dataset for compositional language and elementary visual reasoning. In: CVPR (2017)
Joulin, A., Bach, F., Ponce, J.: Discriminative clustering for image co-segmentation. In: CVPR (2010)
Kabra, R., et al.: Multi-object datasets (2019). https://github.com/deepmind/multi_object_datasets/
Karazija, L., Laina, I., Rupprecht, C.: Clevrtex: a texture-rich benchmark for unsupervised multi-object segmentation. In: Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) (2021)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: 2nd International Conference on Learning Representations, ICLR 2014, Banff, 14–16 April 2014, Conference Track Proceedings (2014)
Kosiorek, A.R., Sabour, S., Teh, Y.W., Hinton, G.E.: Stacked capsule autoencoders. Adv. Neural Inf. Process. Syst. 23 (2019)
Kuhn, H.W.: The Hungarian method for the assignment problem. Naval Res. Logist. (NRL) 52 (1955). https://api.semanticscholar.org/CorpusID:9426884
Learned-Miller, E.G.: Data driven image models through continuous joint alignment. IEEE Trans. Pattern Anal. Mach. Intell. 28(2), 236–250 (2005)
Lesjak, Ž., et al.: A novel public MR image dataset of multiple sclerosis patients with lesion segmentations based on multi-rater consensus. Neuroinformatics 16, 51–63 (2018)
Lin, Z., et al.: Space: unsupervised object-oriented scene representation via spatial attention and decomposition. In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=rkl03ySYDH
Liu, W., et al.: Towards visually explaining variational autoencoders. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8642–8651 (2020)
MacQueen, J., et al.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Oakland, vol. 1, pp. 281–297 (1967)
Maguelone: database of typographical ornaments. http://maguelone.enssib.fr
McKenna, A., Mori, G.: Claude-françois simon, imprimeur, dit « poppy », contrefacteur de marc michel rey (2019). https://mmrey.hypotheses.org/778
Metaxas, I.M., Tzimiropoulos, G., Patras, I.: Divclust: Controlling diversity in deep clustering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3418–3428 (2023)
Monnier, T., Groueix, T., Aubry, M.: Deep transformation-invariant clustering. In: NeurIPS (2020)
Monnier, T., Vincent, E., Ponce, J., Aubry, M.: Unsupervised layered image decomposition into object prototypes. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8640–8650 (2021)
Naumann, A., Hertlein, F., Dörr, L., Furmans, K.: Tampar: visual tampering detection for parcel logistics in postal supply chains. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 8076–8086 (2024)
Pang, G., Shen, C., Cao, L., Hengel, A.V.D.: Deep learning for anomaly detection: a review. ACM Comput. Surv. 54(2), 1–38 (2021)
Pimentel, M.A., Clifton, D.A., Clifton, L., Tarassenko, L.: A review of novelty detection. Signal Process. 99, 215–249 (2014)
Radford, A., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning, pp. 8748–8763. PMLR (2021)
Rey database: database of publishing ornaments, ROIi. https://heurist.huma-num.fr/heurist/?db=ROIi
Riffaud, A., Pantin, I.: Une archéologie du livre français moderne. Droz (2011)
Rosart, J.: Epreuves des caractères qui se gravent et se fondent dans la nouvelle fonderie de jacques françois rosart (1761)
Rubinstein, M., Joulin, A., Kopf, J., Liu, C.: Unsupervised joint object discovery and segmentation in internet images. In: CVPR (2013)
Ruff, L., et al.: A unifying review of deep and shallow anomaly detection. Proc. IEEE 109(5), 756–795 (2021)
Russell, B.C., Freeman, W.T., Efros, A.A., Sivic, J., Zisserman, A.: Using multiple segmentations to discover objects and their extent in image collections. In: CVPR (2006)
Sauvalle, B., de La Fortelle, A.: Unsupervised multi-object segmentation using attention and soft-argmax. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3267–3276 (2023)
Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., Langs, G.: Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: International Conference on Information Processing in Medical Imaging, pp. 146–157. Springer (2017)
Shen, L., et al.: S2looking: a satellite side-looking dataset for building change detection. Remote Sens. 13(24), 5094 (2021)
Shen, X., Efros, A.A., Aubry, M.: Discovering visual patterns in art collections with spatially-consistent feature learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)
Siméoni, O., et al.: Localizing objects with self-supervised transformers and no labels. In: Proceedings of the British Machine Vision Conference (BMVC) (2021)
Sivic, J., Russell, B.C., Zisserman, A., Freeman, W.T., Efros, A.A.: Unsupervised discovery of visual object class hierarchies. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)
Smirnov, D., Gharbi, M., Fisher, M., Guizilini, V., Efros, A.A., Solomon, J.: MarioNette: Self-Supervised Sprite Learning. arXiv preprint arXiv:2104.14553 [cs] (2021)
Van Gansbeke, W., Vandenhende, S., Georgoulis, S., Proesmans, M., Van Gool, L.: Scan: learning to classify images without labels. In: Proceedings of the European Conference on Computer Vision (2020)
Venkataramanan, S., Peng, K.-C., Singh, R.V., Mahalanobis, A.: Attention guided anomaly localization in images. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12362, pp. 485–503. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58520-4_29
Vo, H.V., et al.: Unsupervised image matching and object discovery as optimization. In: CVPR (2019)
Wilkinson, H., Briggs, J., Gorissen, D.: Computer vision and the creation of a database of printers’ ornaments. Digital Humanities Quarterly (2021)
Wilkinson, H.: ‘printers’ flowers as evidence in the identification of unknown printers: two examples from 1715. In: The Library, 7th Series, vol. 14, pp. 70–79 (2013)
Xie, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: International Conference on Machine Learning, pp. 478–487. PMLR (2016)
Yang, B., Fu, X., Sidiropoulos, N.D., Hong, M.: Towards k-means-friendly spaces: simultaneous deep learning and clustering. In: International Conference on Machine Learning, pp. 3861–3870. PMLR (2017)
Yang, Y., Yang, B.: Promising or elusive? Unsupervised object segmentation from real-world single images. Adv. Neural. Inf. Process. Syst. 35, 4722–4735 (2022)
Zenati, H., Foo, C.S., Lecouat, B., Manek, G., Chandrasekhar, V.R.: Efficient gan-based anomaly detection. arXiv preprint arXiv:1802.06222 (2018)
Zhu, W., Shen, Y., Yu, L., Aguirre Sanchez, L.P.: Gmair: unsupervised object detection based on spatial attention and gaussian mixture. arXiv preprints, pp. arXiv–2106 (2021)
Acknowledgement
This work was funded by ANR ROIi project ANR-20-CE38-0005. S. Baltaci, E. Vincent, and M. Aubry were supported by ERC project DISCOVER funded by the European Union’s Horizon Europe Research and Innovation program under grant agreement No. 101076028 and ANR VHS project ANR-21-CE38-0008. We thank Silya Ounoughi, Thomas Gautrais, and Vincent Ventresque for their work in the collection and annotation of the datasets, and Ségolène Albouy, Raphaël Baena, Syrine Kalleli, Ioannis Siglidis, Gurjeet Sangra Singh, Andrea Morales Garzón and Malamatenia Vlachou for valuable feedbacks.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chaki, S.K. et al. (2024). Historical Printed Ornaments: Dataset and Tasks. In: Barney Smith, E.H., Liwicki, M., Peng, L. (eds) Document Analysis and Recognition - ICDAR 2024. ICDAR 2024. Lecture Notes in Computer Science, vol 14806. Springer, Cham. https://doi.org/10.1007/978-3-031-70543-4_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-70543-4_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-70542-7
Online ISBN: 978-3-031-70543-4
eBook Packages: Computer ScienceComputer Science (R0)