The Melody Triangle is an interface for the discovery of melodic materials, where the input – pos... more The Melody Triangle is an interface for the discovery of melodic materials, where the input – positions within a triangle – directly map to information theoretic properties of the output. A model of human expectation and surprise in the perception of music, information dynamics, is used to 'map out' a musical generative system's parameter space. This enables a user to explore the possibilities afforded by a generative algorithm, in this case Markov chains, not by directly selecting parameters, but by specifying the subjective predictability of the output sequence. We describe some of the relevant ideas from information dynamics and how the Melody Triangle is defined in terms of these. We describe its incarnation as a screen based performance tool and compositional aid for the generation of musical textures; the users control at the abstract level of randomness and predictability, and some pilot studies carried out with it. We also briefly outline a multi-user installatio...
This research focuses on real-time gesture learning and recognition. Events arrive in a continuou... more This research focuses on real-time gesture learning and recognition. Events arrive in a continuous stream without explicitly given boundaries. To obtain temporal accuracy, weneed to consider the lag between the detection of an eventand any effects we wish to trigger with it. Two methodsfor real time gesture recognition using a Nintendo Wii controller are presented. The first detects gestures similar to agiven template using either a Euclidean distance or a cosinesimilarity measure. The second method uses novel information theoretic methods to detect and categorize gestures inan unsupervised way. The role of supervision, detection lagand the importance of haptic feedback are discussed.
Measures such as entropy and mutual information can be used to characterise random processes. In ... more Measures such as entropy and mutual information can be used to characterise random processes. In this paper, we propose the use of several time-varying information measures, computed in the context of a probabilistic model which evolves as a sample of the process unfolds, as a way to characterise temporal structure in music. One such measure is a novel predictive information rate which we conjecture may provide an explanation for the ‘inverted-U ’ relationship often found between simple measures of randomness (e.g. entropy rate) and judgements of aesthetic value (1). We explore these ideas in the context of Markov chains using both artificially generated sequences and two pieces of minimalist music by Philip Glass, showing that even such a manifestly simplistic model (the Markov chain), when interpreted according to information dynamic principles, produces a structural analysis which largely agrees with that of an expert human listener.
COPYRIGHT All material supplied via Goldsmiths Library and Goldsmiths Research Online (GRO) is pr... more COPYRIGHT All material supplied via Goldsmiths Library and Goldsmiths Research Online (GRO) is protected by copyright and other intellectual property rights. You may use this copy for personal study or research, or for educational purposes, as defined by UK copyright law. Other specific conditions may apply to individual items. This copy has been supplied on the understanding that it is copyright material. Duplication or sale of all or part of any of the GRO Data Collections is not permitted, and no quotation or excerpt from the work may be published without the prior written consent of the copyright holder/s.
Several Prolog implementations include a facility for tabling, an alternative resolution strategy... more Several Prolog implementations include a facility for tabling, an alternative resolution strategy which uses memoisation to avoid redundant duplication of computations. Until relatively recently, tabling has required either low-level support in the underlying Prolog engine, or extensive program transormation (de Guzman et al., 2008). An alternative approach is to augment Prolog with low level support for continuation capturing control operators, particularly delimited continuations, which have been investigated in the field of functional programming and found to be capable of supporting a wide variety of computational effects within an otherwise declarative language. This technical report describes an implementation of tabling in SWI Prolog based on delimited control operators for Prolog recently introduced by Schrijvers et al. (2013). In comparison with a previous implementation of tabling for SWI Prolog using delimited control (Desouter et al., 2015), this approach, based on the f...
Memoisation, or tabling, is a well-known technique that yields large improvements in the performa... more Memoisation, or tabling, is a well-known technique that yields large improvements in the performance of some recursive computations. Tabled resolution in Prologs such as XSB and B-Prolog can transform so called left-recursive predicates from non-terminating computations into finite and well-behaved ones. In the functional programming literature, memoisation has usually been implemented in a way that does not handle left-recursion, requiring supplementary mechanisms to prevent non-termination. A notable exception is Johnson's (1995) continuation passing approach in Scheme. This, however, relies on mutation of a memo table data structure and coding in explicit continuation passing style. We show how Johnson's approach can be implemented purely functionally in a modern, strongly typed functional language (OCaml), presented via a monadic interface that hides the implementation details, yet providing a way to return a compact represention of the memo tables at the end of the comp...
We describe a method of visualising geometrically the dependency structure of a distributed repre... more We describe a method of visualising geometrically the dependency structure of a distributed representation. The mutual information between each pair of components is estimated using a nonlinear correlation coecient, in terms of which a distance measure is dened. Multidimensional scaling is then used to generate a spatial conguration that reproduces these distances, the end result being a spatial representation of the dependency between the components, from which an appropriate topology for the representation may be inferred. The method is applied to ICA representations of speech and music.
Music similarity has been widely studied through melodic and harmonic matching, clustering, and u... more Music similarity has been widely studied through melodic and harmonic matching, clustering, and using various metrics for measuring distance. Such analyses offer the musicologist a view of the ‘sameness’ of parts of a score. However, similarity alone does not necessarily allow exploitation of that sameness in reasoning about the music. In this paper, we present work in progress to investigate rhythm similarity at various scales, beginning at the smallest (single measures or groups of measures). We use normalised compression distance and variations thereof to derive similarity-based dependencies between parts of the music. Establishing such dependencies may allow software engineering dependence analysis techniques to be applied to music to, e.g. remove from focus aspects not relevant to a particular enquiry (‘slicing’), determine the sensitivity of later parts of the music on former parts (‘impact analysis’), and to find motivic processes and developments within the musical form. The...
We consider the problem of detecting note onsets in music under the hypothesis that the onsets, a... more We consider the problem of detecting note onsets in music under the hypothesis that the onsets, and events in general, are essentially surprising moments, and that event detection should therefore be based on an explicit probability model of the sensory input, which generates a moment-by-moment trace of the probability of each observation as it is made. Relatively unexpected events should thus appear as clear spikes. In this way, several well known methods of onset detection can be understood in terms of an implicit probability model. We apply ICA to the problem as an adaptive non-Gaussian model, and investigate the use of ICA as a conditional probability model. The results obtained using several methods on two extracts of piano music are presented and compared. Finally, we tentatively suggest an information theoretic interpretation of the approach.
Previous work has shown that various flavours of Independent Component Analysis, when applied to ... more Previous work has shown that various flavours of Independent Component Analysis, when applied to natural images, all result in broadly similar localised, oriented band-pass feature detectors, which have been likened to wavelets or edge detectors. In this paper, we present a similar analysis of ‘natural’ sounds drawn from two radio stations: one broadcasting mainly speech; the other mainly classical music. Many of the resulting basis vectors are quite wavelet-like, and can easily be characterised in terms of their position and spread in the time-frequency plane. Some of them, however, particularly from the set trained on music, do not fit that interpretation very well. The Wigner-Ville Distribution can be used to gain a clearer picture of time-frequency localisation of these basis vectors. We conclude by suggesting that these results be compared with other widely used auditory representations such as short-term Fourier transforms, wavelet transforms, and physiologically derived model...
Samer Abdallah, Aquiles Alencar-Brayner, Emmanouil Benetos, Stephen Cottrell Jason Dykes, Nicolas... more Samer Abdallah, Aquiles Alencar-Brayner, Emmanouil Benetos, Stephen Cottrell Jason Dykes, Nicolas Gold, Alexander Kachkaev, Mahendra Mahey, Dan Tidhar, Adam Tovell, Tillman Weyde, Daniel Wolff 1 Department of Computer Science, University College London, 2 The British Library, 3 Department of Computer Science, City University London, 4 Department of Music, City University London 5 Centre for Digital Music, Queen Mary University of London dml-owner@city.ac.ukhttp://dml.city.ac.uk
Automatic differentiation is a technique which allows a programmer to define a numerical computat... more Automatic differentiation is a technique which allows a programmer to define a numerical computation via compositions of a broad range of numeric and computational primitives and have the underlying system support the computation of partial derivatives of the result with respect to any of its inputs, without making any finite difference approximations, and without manipulating large symbolic expressions representing the computation. This note describes a novel approach to reverse mode automatic differentiation using constraint logic programmming, specifically, the constraint handling rules (CHR) library of SWI Prolog, resulting in a very small (50 lines of code) implementation. When applied to a differentiation-based implementation of the inside-outside algorithm for parameter learning in probabilistic grammars, the CHR based implementations outperformed two well-known frameworks for optimising differentiable functions, Theano and TensorFlow, by a large margin.
We present the StructureNet a recurrent neural network for inducing structure in machine-generate... more We present the StructureNet a recurrent neural network for inducing structure in machine-generated compositions. This model resides in a musical structure space and works in tandem with a probabilistic music generation model as a modifying agent. It favourably biases the probabilities of those notes that result in the occurrence of structural elements it has learnt from a dataset. It is extremely flexible in that it is able to work with any such probabilistic model, it works well when training data is limited, and the types of structure it can be made to induce are highly customisable. We demonstrate through our experiments on a subset of the Nottingham dataset that melodies generated by a recurrent neural network based melody model are indeed more structured in the presence of the StructureNet.
An Information-Theoretic Account of Musical Expectation and Memory Kat Agres (kathleen.agres@eecs... more An Information-Theoretic Account of Musical Expectation and Memory Kat Agres (kathleen.agres@eecs.qmul.ac.uk) Centre for Digital Music and Cognitive Science Research Group School of Electronic Engineering & Computer Science Queen Mary, University of London London E1 4NS, United Kingdom Samer Abdallah (samer.abdallah@eecs.qmul.ac.uk) Centre for Digital Music School of Electronic Engineering & Computer Science Queen Mary, University of London London E1 4NS, United Kingdom Marcus Pearce (marcus.pearce@eecs.qmul.ac.uk) Cognitive Science Research Group, Centre for Digital Music and Centre for Research in Psychology School of Electronic Engineering & Computer Science Queen Mary, University of London London E1 4NS, United Kingdom This paper examines the process of learning novel music over time, with a focus on mental anticipatory processing and musical structure. By using carefully constructed tone sequences, we are able to test how the statistical structure of music, as measured using in...
The Melody Triangle is an interface for the discovery of melodic materials, where the input – pos... more The Melody Triangle is an interface for the discovery of melodic materials, where the input – positions within a triangle – directly map to information theoretic properties of the output. A model of human expectation and surprise in the perception of music, information dynamics, is used to 'map out' a musical generative system's parameter space. This enables a user to explore the possibilities afforded by a generative algorithm, in this case Markov chains, not by directly selecting parameters, but by specifying the subjective predictability of the output sequence. We describe some of the relevant ideas from information dynamics and how the Melody Triangle is defined in terms of these. We describe its incarnation as a screen based performance tool and compositional aid for the generation of musical textures; the users control at the abstract level of randomness and predictability, and some pilot studies carried out with it. We also briefly outline a multi-user installatio...
This research focuses on real-time gesture learning and recognition. Events arrive in a continuou... more This research focuses on real-time gesture learning and recognition. Events arrive in a continuous stream without explicitly given boundaries. To obtain temporal accuracy, weneed to consider the lag between the detection of an eventand any effects we wish to trigger with it. Two methodsfor real time gesture recognition using a Nintendo Wii controller are presented. The first detects gestures similar to agiven template using either a Euclidean distance or a cosinesimilarity measure. The second method uses novel information theoretic methods to detect and categorize gestures inan unsupervised way. The role of supervision, detection lagand the importance of haptic feedback are discussed.
Measures such as entropy and mutual information can be used to characterise random processes. In ... more Measures such as entropy and mutual information can be used to characterise random processes. In this paper, we propose the use of several time-varying information measures, computed in the context of a probabilistic model which evolves as a sample of the process unfolds, as a way to characterise temporal structure in music. One such measure is a novel predictive information rate which we conjecture may provide an explanation for the ‘inverted-U ’ relationship often found between simple measures of randomness (e.g. entropy rate) and judgements of aesthetic value (1). We explore these ideas in the context of Markov chains using both artificially generated sequences and two pieces of minimalist music by Philip Glass, showing that even such a manifestly simplistic model (the Markov chain), when interpreted according to information dynamic principles, produces a structural analysis which largely agrees with that of an expert human listener.
COPYRIGHT All material supplied via Goldsmiths Library and Goldsmiths Research Online (GRO) is pr... more COPYRIGHT All material supplied via Goldsmiths Library and Goldsmiths Research Online (GRO) is protected by copyright and other intellectual property rights. You may use this copy for personal study or research, or for educational purposes, as defined by UK copyright law. Other specific conditions may apply to individual items. This copy has been supplied on the understanding that it is copyright material. Duplication or sale of all or part of any of the GRO Data Collections is not permitted, and no quotation or excerpt from the work may be published without the prior written consent of the copyright holder/s.
Several Prolog implementations include a facility for tabling, an alternative resolution strategy... more Several Prolog implementations include a facility for tabling, an alternative resolution strategy which uses memoisation to avoid redundant duplication of computations. Until relatively recently, tabling has required either low-level support in the underlying Prolog engine, or extensive program transormation (de Guzman et al., 2008). An alternative approach is to augment Prolog with low level support for continuation capturing control operators, particularly delimited continuations, which have been investigated in the field of functional programming and found to be capable of supporting a wide variety of computational effects within an otherwise declarative language. This technical report describes an implementation of tabling in SWI Prolog based on delimited control operators for Prolog recently introduced by Schrijvers et al. (2013). In comparison with a previous implementation of tabling for SWI Prolog using delimited control (Desouter et al., 2015), this approach, based on the f...
Memoisation, or tabling, is a well-known technique that yields large improvements in the performa... more Memoisation, or tabling, is a well-known technique that yields large improvements in the performance of some recursive computations. Tabled resolution in Prologs such as XSB and B-Prolog can transform so called left-recursive predicates from non-terminating computations into finite and well-behaved ones. In the functional programming literature, memoisation has usually been implemented in a way that does not handle left-recursion, requiring supplementary mechanisms to prevent non-termination. A notable exception is Johnson's (1995) continuation passing approach in Scheme. This, however, relies on mutation of a memo table data structure and coding in explicit continuation passing style. We show how Johnson's approach can be implemented purely functionally in a modern, strongly typed functional language (OCaml), presented via a monadic interface that hides the implementation details, yet providing a way to return a compact represention of the memo tables at the end of the comp...
We describe a method of visualising geometrically the dependency structure of a distributed repre... more We describe a method of visualising geometrically the dependency structure of a distributed representation. The mutual information between each pair of components is estimated using a nonlinear correlation coecient, in terms of which a distance measure is dened. Multidimensional scaling is then used to generate a spatial conguration that reproduces these distances, the end result being a spatial representation of the dependency between the components, from which an appropriate topology for the representation may be inferred. The method is applied to ICA representations of speech and music.
Music similarity has been widely studied through melodic and harmonic matching, clustering, and u... more Music similarity has been widely studied through melodic and harmonic matching, clustering, and using various metrics for measuring distance. Such analyses offer the musicologist a view of the ‘sameness’ of parts of a score. However, similarity alone does not necessarily allow exploitation of that sameness in reasoning about the music. In this paper, we present work in progress to investigate rhythm similarity at various scales, beginning at the smallest (single measures or groups of measures). We use normalised compression distance and variations thereof to derive similarity-based dependencies between parts of the music. Establishing such dependencies may allow software engineering dependence analysis techniques to be applied to music to, e.g. remove from focus aspects not relevant to a particular enquiry (‘slicing’), determine the sensitivity of later parts of the music on former parts (‘impact analysis’), and to find motivic processes and developments within the musical form. The...
We consider the problem of detecting note onsets in music under the hypothesis that the onsets, a... more We consider the problem of detecting note onsets in music under the hypothesis that the onsets, and events in general, are essentially surprising moments, and that event detection should therefore be based on an explicit probability model of the sensory input, which generates a moment-by-moment trace of the probability of each observation as it is made. Relatively unexpected events should thus appear as clear spikes. In this way, several well known methods of onset detection can be understood in terms of an implicit probability model. We apply ICA to the problem as an adaptive non-Gaussian model, and investigate the use of ICA as a conditional probability model. The results obtained using several methods on two extracts of piano music are presented and compared. Finally, we tentatively suggest an information theoretic interpretation of the approach.
Previous work has shown that various flavours of Independent Component Analysis, when applied to ... more Previous work has shown that various flavours of Independent Component Analysis, when applied to natural images, all result in broadly similar localised, oriented band-pass feature detectors, which have been likened to wavelets or edge detectors. In this paper, we present a similar analysis of ‘natural’ sounds drawn from two radio stations: one broadcasting mainly speech; the other mainly classical music. Many of the resulting basis vectors are quite wavelet-like, and can easily be characterised in terms of their position and spread in the time-frequency plane. Some of them, however, particularly from the set trained on music, do not fit that interpretation very well. The Wigner-Ville Distribution can be used to gain a clearer picture of time-frequency localisation of these basis vectors. We conclude by suggesting that these results be compared with other widely used auditory representations such as short-term Fourier transforms, wavelet transforms, and physiologically derived model...
Samer Abdallah, Aquiles Alencar-Brayner, Emmanouil Benetos, Stephen Cottrell Jason Dykes, Nicolas... more Samer Abdallah, Aquiles Alencar-Brayner, Emmanouil Benetos, Stephen Cottrell Jason Dykes, Nicolas Gold, Alexander Kachkaev, Mahendra Mahey, Dan Tidhar, Adam Tovell, Tillman Weyde, Daniel Wolff 1 Department of Computer Science, University College London, 2 The British Library, 3 Department of Computer Science, City University London, 4 Department of Music, City University London 5 Centre for Digital Music, Queen Mary University of London dml-owner@city.ac.ukhttp://dml.city.ac.uk
Automatic differentiation is a technique which allows a programmer to define a numerical computat... more Automatic differentiation is a technique which allows a programmer to define a numerical computation via compositions of a broad range of numeric and computational primitives and have the underlying system support the computation of partial derivatives of the result with respect to any of its inputs, without making any finite difference approximations, and without manipulating large symbolic expressions representing the computation. This note describes a novel approach to reverse mode automatic differentiation using constraint logic programmming, specifically, the constraint handling rules (CHR) library of SWI Prolog, resulting in a very small (50 lines of code) implementation. When applied to a differentiation-based implementation of the inside-outside algorithm for parameter learning in probabilistic grammars, the CHR based implementations outperformed two well-known frameworks for optimising differentiable functions, Theano and TensorFlow, by a large margin.
We present the StructureNet a recurrent neural network for inducing structure in machine-generate... more We present the StructureNet a recurrent neural network for inducing structure in machine-generated compositions. This model resides in a musical structure space and works in tandem with a probabilistic music generation model as a modifying agent. It favourably biases the probabilities of those notes that result in the occurrence of structural elements it has learnt from a dataset. It is extremely flexible in that it is able to work with any such probabilistic model, it works well when training data is limited, and the types of structure it can be made to induce are highly customisable. We demonstrate through our experiments on a subset of the Nottingham dataset that melodies generated by a recurrent neural network based melody model are indeed more structured in the presence of the StructureNet.
An Information-Theoretic Account of Musical Expectation and Memory Kat Agres (kathleen.agres@eecs... more An Information-Theoretic Account of Musical Expectation and Memory Kat Agres (kathleen.agres@eecs.qmul.ac.uk) Centre for Digital Music and Cognitive Science Research Group School of Electronic Engineering & Computer Science Queen Mary, University of London London E1 4NS, United Kingdom Samer Abdallah (samer.abdallah@eecs.qmul.ac.uk) Centre for Digital Music School of Electronic Engineering & Computer Science Queen Mary, University of London London E1 4NS, United Kingdom Marcus Pearce (marcus.pearce@eecs.qmul.ac.uk) Cognitive Science Research Group, Centre for Digital Music and Centre for Research in Psychology School of Electronic Engineering & Computer Science Queen Mary, University of London London E1 4NS, United Kingdom This paper examines the process of learning novel music over time, with a focus on mental anticipatory processing and musical structure. By using carefully constructed tone sequences, we are able to test how the statistical structure of music, as measured using in...
Uploads
Papers by Samer Abdallah