Papers by William Sethares
Heritage
This paper introduces the Watermark Imaging System (WImSy) which can be used to photograph, docum... more This paper introduces the Watermark Imaging System (WImSy) which can be used to photograph, document, and study sheets of paper. The WImSy provides surface images, raking light images, and transmitted light images of the paper, all in perfect alignment. We develop algorithms that exploit this alignment by combining several images together in a process that mimics both the “surface image removal” technique and the method of “high dynamic range” photographs. An improved optimization criterion and an automatic parameter selection procedure streamline the process and make it practical for art historians and conservators to extract the relevant information to study watermarks. The effectiveness of the method is demonstrated in several experiments on images taken with the WImSy at the Metropolitan Museum of Art in New York and at the Getty Museum in Los Angeles, and the results are compared with manually optimized images.
Bookmarks Related papers MentionsView impact
arXiv (Cornell University), Nov 28, 2022
Bookmarks Related papers MentionsView impact
ArXiv, 2018
This paper presents a new search algorithm called Target Image Search based on Local Features (TI... more This paper presents a new search algorithm called Target Image Search based on Local Features (TISLF) which compares target images and video source images using local features. TISLF can be used to locate frames where target images occur in a video, and by computing and comparing the matching probability matrix, estimates the time of appearance, the duration, and the time of disappearance of the target image from the video stream. The algorithm is applicable to a variety of applications such as tracking the appearance and duration of advertisements in the broadcast of a sports event, searching and labelling painting in documentaries, and searching landmarks of different cities in videos. The algorithm is compared to a deep learning method and shows competitive performance in experiments.
Bookmarks Related papers MentionsView impact
Version: Accepted Manuscript Link(s) to article on publisher’s website:
Bookmarks Related papers MentionsView impact
A primary esthetic in the performance practice of Balinese gamelan is the <i>ombak</i>... more A primary esthetic in the performance practice of Balinese gamelan is the <i>ombak</i> (Indonesian for <i>wave</i>), which is manifest in musical form, performance, and tuning. The <i>ombak</i> arises in a paired tuning system in which corresponding unisons of two instruments (or instrumental groups) are tuned to slightly different frequencies, one higher and one lower, to produce beats. Pitch classes are not necessarily tuned to octaves in an exact 2:1 frequency ratio; instead, octaves are often stretched or compressed. This paper discusses the relationship between the <i>ombak</i> rate and octave tempering, and demonstrates that the beating rate, combined with the octave tuning strategy chosen, can be modeled using a <i>tempering parameter</i> that determines the amount of stretching or compression. This model is then used to analyze tuning data of nine complete gamelan.
Bookmarks Related papers MentionsView impact
We present an application XronoMorph for the algorithmic generation of rhythms in the context of ... more We present an application XronoMorph for the algorithmic generation of rhythms in the context of creative composition and performance, and of musical analysis and education. XronoMorph makes use of visual and geometrical conceptualizations of rhythms, and allows the user to smoothly morph between rhythms. Sonification of the user generated geometrical constructs is possible using a built-in sampler, VST and AU plugins, or standalone synthesizers via MIDI. The algorithms are based on two underlying mathematical principles: perfect balance and well-formedness, both of which can be derived from coefficients of the discrete Fourier transform of the rhythm. The mathematical background, musical implications, and their implementation in the software are discussed.
Bookmarks Related papers MentionsView impact
Bookmarks Related papers MentionsView impact
Journal of Quantitative Description: Digital Media, 2021
Live-tweeting has emerged as a popular hybrid media activity during broadcasted media events. Thr... more Live-tweeting has emerged as a popular hybrid media activity during broadcasted media events. Through second screens, users are able to engage with one another and react in real time to the broadcasted content. These reactions are dynamic: they ebb and flow throughout the media event as users respond to and converse about different memorable moments. Using the first 2016 U.S. presidential debate between Hillary Clinton and Donald Trump as a case, this paper employs a temporal method for identifying resonant moments on social media during televised events by combining time series analysis, qualitative (human-in-the-loop) evaluation, and a novel natural language processing tool to identify discursive shifts before and after resonant moments. This analysis finds key differences in social media discourse about the two candidates. Notably, Trump received substantially more coverage than Clinton throughout the debate. However, a more in-depth analysis of these candidates’ resonant moments...
Bookmarks Related papers MentionsView impact
Handmade laid paper has the important quality that every sheet of paper formed on the same paperm... more Handmade laid paper has the important quality that every sheet of paper formed on the same papermaking mold retains a nearly identical imprint of the mold’s wire structure. These “moldmates” are identified by analyzing the recorded wire features, which are visible using transmitted light. When visual analysis is not sufficient to distinguish moldmates, three features of the mold’s wire mesh can be quantitatively analyzed using image processing techniques: watermark shape and placement, chain line intervals, and laid line density, for which a new method of analysis is introduced here. Using signal processing procedures, the frequency of the laid lines across a sheet of paper was found to fluctuate in a pattern unique to that mold. These quantitative methods were tested on a sample set of blank sheets from a 1536 edition of De re militari by Vegetius; computational analysis using any one of the three features was able to distinguish between four molds used in the group of papers. Thes...
Bookmarks Related papers MentionsView impact
This paper learns multi-modal embeddings from text, audio, and video views/modes of data in order... more This paper learns multi-modal embeddings from text, audio, and video views/modes of data in order to improve upon downstream sentiment classification. The experimental framework also allows investigation of the relative contributions of the individual views in the final multi-modal embedding. Individual features derived from the three views are combined into a multi-modal embedding using Deep Canonical Correlation Analysis (DCCA) in two ways i) One-Step DCCA and ii) TwoStep DCCA. This paper learns text embeddings using BERT, the current state-of-the-art in text encoders. We posit that this highly optimized algorithm dominates over the contribution of other views, though each view does contribute to the final result. Classification tasks are carried out on two benchmark data sets and on a new Debate Emotion data set, and together these demonstrate that the one-Step DCCA outperforms the current state-of-the-art in learning multi-modal embeddings.
Bookmarks Related papers MentionsView impact
We present a class of novel metrics for measuring the distance between any two periodic scales wh... more We present a class of novel metrics for measuring the distance between any two periodic scales whatever their precise tuning or cardinality. The metrics have some important applications: (1) finding effective lower-dimensional temperaments of higherdimensional tunings (such as just intonation); (2) finding simple scales that effectively approximate complex scales (e.g., using equally-tuned and well-formed scales to approximate Fokker periodicity blocks and pairwise well-formed scales); (3) finding ways to map the notes of any arbitrary scale to a button lattice controller (a generalized keyboard) so as to maximise geometrical consistency and playability; (4) comparing the distance of various scales (such as equal tunings with different numbers of notes per octave) for analytical, maybe even compositional, purposes; (5) a generalized method to determine the similarity of different “pitch class sets” that is not dependent on the use of a low cardinality equal tunings such 12-tone, thu...
Bookmarks Related papers MentionsView impact
Standard word embedding algorithms learn vector representations from large corpora of text docume... more Standard word embedding algorithms learn vector representations from large corpora of text documents in an unsupervised fashion. However, the quality of word embeddings learned from these algorithms is affected by the size of training data sets. Thus, applications of these algorithms in domains with only moderate amounts of available data is limited. In this paper we introduce an algorithm that learns word embeddings jointly with a classifier. Our algorithm is called SWESA (Supervised Word Embeddings for Sentiment Analysis). SWESA leverages document label information to learn vector representations of words from a modest corpus of text documents by solving an optimization problem that minimizes a cost function with respect to both word embeddings and the weight vector used for classification. Experiments on several real world data sets show that SWESA has superior performance on domains with limited data, when compared to previously suggested approaches to word embeddings and sentim...
Bookmarks Related papers MentionsView impact
ArXiv, 2019
The memory consumption of most Convolutional Neural Network (CNN) architectures grows rapidly wit... more The memory consumption of most Convolutional Neural Network (CNN) architectures grows rapidly with increasing depth of the network, which is a major constraint for efficient network training and inference on modern GPUs with limited memory. Several studies show that the feature maps (as generated after the convolutional layers) are the main bottleneck in this memory problem. Often, these feature maps mimic natural photographs in the sense that their energy is concentrated in the spectral domain. Although embedding CNN architectures in the spectral domain is widely exploited to accelerate the training process, we demonstrate that it is also possible to use the spectral domain to reduce the memory footprint by proposing a Spectral Domain Convolutional Neural Network (SpecNet) that performs both the convolution and the activation operations in the spectral domain. SpecNet exploits a configurable threshold to force small values in the feature maps to zero, allowing the feature maps to b...
Bookmarks Related papers MentionsView impact
Uniform strings have a harmonic sound; nonuniform strings have an inharmonic sound. This paper ex... more Uniform strings have a harmonic sound; nonuniform strings have an inharmonic sound. This paper experiments with musical instruments based on nonuniform/inharmonic strings. Given a precise description of the string, its spectrum can be calculated using standard techniques. Dissonance curves are used to motivate specific choices of spectrum. A particular inharmonic string consisting of three segments (two equal unwound segments surrounding a thicker wound portion) is used in the construction of the hyperpiano. A second experiment designs a string with overtones that lie on steps of the 10-tone equal tempered scale. The strings are sampled, and digital (software) versions of the instruments are made available along with a call for composers interested in writing for these new instruments.
Bookmarks Related papers MentionsView impact
Aerospace Structures and MaterialsAerospace Engineerin
Bookmarks Related papers MentionsView impact
The sieve-like molds used early paper manufacturing leave characteristic chain line patterns that... more The sieve-like molds used early paper manufacturing leave characteristic chain line patterns that can help determine if two pieces of paper were made using the same mold [1]. Such moldmates can help in establishing chronology, suggest paper preferences, and indicate periods of intense activity of an artist, and the study of Rembrandt’s prints has occupied a prominent place within this scholarship [2]. There are two basic steps to the automation of the analysis. First, the chain lines must be accurately marked, extracting points and lines from the x-ray images. Second, the markings from many x-rays must be compared to find matches. This paper presents improved algorithms for both of these tasks. Section 2 follows [5] in replacing the classic Radon transform approach of [3] (which requires that the chain lines be straight) with a local method that outputs marked grids of points. The method segments the images, corrects for tilt, filters, conducts a (local) Radon transform and then con...
Bookmarks Related papers MentionsView impact
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 2021
Bookmarks Related papers MentionsView impact
Journal of Historians of Netherlandish Art, 2021
Identifying, comparing, and matching watermarks in pre-machine-made papers has occupied scholars ... more Identifying, comparing, and matching watermarks in pre-machine-made papers has occupied scholars of prints and drawings for some time. One popular but arduous approach is to overlay, either manually or digitally, an image of the watermark in question with its presumed match from a known source. For example, a newly discovered watermark in a Rembrandt print might be compared to a similar one reproduced in Erik Hinterding’s Rembrandt as an Etcher (2006). Such an overlay can confirm the pair as identical, i.e., as moldmates, or reveal their differences. But creating an accurate overlay for two images with different scales, orientations, or resolutions using standard image-manipulation tools can be time consuming and, ultimately, unsuccessful. Part One of this article describes advances in the emerging field of computational art history, specifically the development of digital image processing software, that can be used to semi-automatically create a reliable animated overlay of two wat...
Bookmarks Related papers MentionsView impact
Sensors, 2021
Pyroelectric Infrared (PIR) sensors are low-cost, low-power, and highly reliable sensors that hav... more Pyroelectric Infrared (PIR) sensors are low-cost, low-power, and highly reliable sensors that have been widely used in smart environments. Indoor localization systems can be categorized as wearable and non-wearable systems, where the latter are also known as device-free localization systems. Since the binary PIR sensor detects only the presence of a human motion in its field of view (FOV) without any other information about the actual location, utilizing the information of overlapping FOV of multiple sensors can be useful for localization. In this study, a PIR detector and sensing signal processing algorithms were designed based on the characteristics of the PIR sensor. We applied the designed PIR detector as a sensor node to create a non-wearable cooperative indoor human localization system. To improve the system performance, signal processing algorithms and refinement schemes (i.e., the Kalman filter, a Transferable Belief Model, and a TBM-based hybrid approach (TBM + Kalman filte...
Bookmarks Related papers MentionsView impact
Journal of Mathematics and Music, 2020
Bookmarks Related papers MentionsView impact
Uploads
Papers by William Sethares
To achieve this aim, we embed pitch collections in novel multi-way expectation arrays, and show how metrics between such arrays can model the perceived dissimilarity of the pitch collections they embed. By modeling the uncertainties of human pitch perception, expectation arrays indicate the expected number of tones, ordered pairs of tones, ordered triples of tones and so forth, that are heard as having any given pitch, dyad of pitches, triad of pitches, and so forth. The pitches can be either absolute or relative (in which case the arrays are invariant with respect to transposition).
We provide a number of examples that show how the metrics accord well with musical intuition, and suggest some ways in which this work may be developed.