Papers by Federico Visi
Lecture Notes in Computer Science, 2016
Bookmarks Related papers MentionsView impact
Lecture Notes in Computer Science, 2016
Bookmarks Related papers MentionsView impact
Bookmarks Related papers MentionsView impact
Bookmarks Related papers MentionsView impact
Bookmarks Related papers MentionsView impact
We have reviewed and assessed the reliability of a dead reckoning and drift correction algorithm ... more We have reviewed and assessed the reliability of a dead reckoning and drift correction algorithm for the estimation of spatial gait parameters using Inertial Measurement Units (IMUs). In particular , we are interested in obtaining accurate stride lengths measurements in order to assess the effects of a wearable haptic cueing device designed to assist people with neurological health conditions during gait rehabilitation. To assess the accuracy of the stride lengths estimates, we compared the output of the algorithm with measurements obtained using a high-end marker-based motion capture system, here adopted as a gold standard. In addition, we introduce an alternative method for detecting initial impact events (i.e. the instants at which one foot contacts the ground, here used for delimiting strides) using accelerometer data. Our method, based on a kinematic feature we named 'jerkage', has proved more robust than detecting peaks on raw accelerometer data. We argue that the resulting measurements of stride lengths are accurate enough to provide trend data needed to support worthwhile gait rehabilitation applications. This approach has potential to assist physiotherapists and patients without access to fully-equipped movement labs. More specifically, it has applications for collecting data to guide and assess gait rehabilitation both outdoors and at home.
Bookmarks Related papers MentionsView impact
Music is a complex multimodal medium experienced not only via sounds but also through body moveme... more Music is a complex multimodal medium experienced not only via sounds but also through body movement. Musical instruments can be seen as technological objects coupled with a repertoire of gestures. We present technical and conceptual issues related to the digital representation and mediation of body movement in musical performance. The paper reports on a case study of a musical performance where motion sensor technologies tracked the movements of the musicians while they played their instruments. Motion data were used to control the electronic elements of the piece in real time. It is suggested that computable motion descriptors and machine learning techniques are useful tools for interpreting motion data in a meaningful manner. However, qualitative insights regarding how human body movement is understood and experienced are necessary to inform further development of motion-capture technologies for expressive purposes. Thus, musical performances provide an effective test bed for new modalities of human–computer interaction.
Bookmarks Related papers MentionsView impact
This paper presents a knowledge-based, data-driven method for using data describing action-sound ... more This paper presents a knowledge-based, data-driven method for using data describing action-sound couplings collected from a group of people to generate multiple complex map-pings between the performance movements of a musician and sound synthesis. This is done by using a database of multimodal motion data collected from multiple subjects coupled with sound synthesis parameters. A series of sound stimuli is synthesised using the sound engine that will be used in performance. Multimodal motion data is collected by asking each participant to listen to each sound stimulus and move as if they were producing the sound using a musical instrument they are given. Multimodal data is recorded during each performance, and paired with the synthesis parameters used for generating the sound stimulus. The dataset created using this method is then used to build a topological representation of the performance movements of the subjects. This representation is then used to interactively generate training data for machine learning algorithms , and define mappings for real-time performance. To better illustrate each step of the procedure, we describe an implementation involving clarinet, motion capture, wear-able sensor armbands, and waveguide synthesis.
Bookmarks Related papers MentionsView impact
This work describes a new approach to gesture mapping in a performance with a traditional musical... more This work describes a new approach to gesture mapping in a performance with a traditional musical instrument and live electronics inspired by theories of embodied music cognition (EMC) and musical gestures. Considerations on EMC and how gestures affect the experience of music inform different mapping strategies. Our intent is to enhance the expressiveness and the liveness of performance by tracking gestures via a multimodal motion capture system and to use motion data to control several features of the music. We then describe an application of such approach to a performance with electric guitar and live electronics, focusing both on aspects of meaning formation and motion capturing.
Bookmarks Related papers MentionsView impact
CMMR 2015 - LNCS9617 , 2015
A machine system is designed to analyze the musical aspects during the live performance, allowing... more A machine system is designed to analyze the musical aspects during the live performance, allowing an interactive and dynamic flow of new expressions and also opening new compositional forms and multimodal methods.
The focus of this approach is to measure the expressiveness from distinct characters during the performance of the musical piece while decisions are made by the machine.
This multimodal approach is motivated by the Three Chamber Micro Songs composition, where measures of loudness and body motion are used by the algorithm to choose a particular musical end.
Bookmarks Related papers MentionsView impact
Proceedings of the 2014 International Workshop on Movement and Computing - MOCO '14, 2014
This paper describes the implementation of gestural mapping strategies for performance with a tra... more This paper describes the implementation of gestural mapping strategies for performance with a traditional musical instrument and electronics. The approach adopted is informed by embodied music cognition and functional categories of musical gestures. Within this framework, gestures are not seen as means of control subordinated to the resulting musical sounds but rather as signi cant elements contributing to the formation of musical meaning similarly to auditory features. Moreover, the ecological knowledge of the gestural repertoire of the instrument is taken into account as it de nes the action-sound relationships between the instrument and the performer and contributes to form expectations in the listeners. Subsequently, mapping strategies from a case study of electric guitar performance will be illustrated describing what motivated the choice of a multimodal motion capture system and how di erent solutions have been adopted considering both gestural meaning formation and technical constraints.
Bookmarks Related papers MentionsView impact
Body movement and embodied knowledge play an impor- tant part in how we express and understand mu... more Body movement and embodied knowledge play an impor- tant part in how we express and understand music. The gestures of a musician playing an instrument are part of a shared knowledge that con- tributes to musical expressivity by building expectations and influencing perception. In this study, we investigate the extent in which the move- ment vocabulary of violin performance is part of the embodied knowl- edge of individuals with no experience in playing the instrument. We asked people who cannot play the violin to mime a performance along an audio excerpt recorded by an expert. They do so by using a silent violin, specifically modified to be more accessible to neophytes. Prelimi- nary motion data analyses suggest that, despite the individuality of each performance, there is a certain consistency among participants in terms of overall rhythmic resonance with the music and movement in response to melodic phrasing. Individualities and commonalities are then analysed using Functional Principal Component Analysis.
Bookmarks Related papers MentionsView impact
Bookmarks Related papers MentionsView impact
This paper describes an exploratory study of different gestures
and body movements of a viola pl... more This paper describes an exploratory study of different gestures
and body movements of a viola player resulting from
the variation of bow strokes length and quantity. Within
the theoretical framework of embodied music cognition
and the study of musical gestures, we aim to observe how
the variation of a musical feature within the piece affects
the body movements of the performer. Two brief pieces
were performed in four different versions, each one with
different directions regarding the bow strokes. The performances
were recorded using a multimodal recording platform
that included audio, video and motion capture data
obtained from high-speed tracking of reflective markers
placed on the body of the performer and on the instrument.
We extracted measurements of quantity of motion
and velocity of different parts of the body, the bow and the
viola. Results indicate that an increased activity in soundproducing
and instrumental gestures does not always resonate
proportionally in the rest of the body and the outcome
in terms of ancillary gestures may vary across upper
body and lower body.
Bookmarks Related papers MentionsView impact
Unfolding | Clusters is a music and visual media installation
modeled from published scientific d... more Unfolding | Clusters is a music and visual media installation
modeled from published scientific data related to the
pathophysiology of amyotrophic lateral sclerosis (ALS). The
work aims to create an engaging multimodal experience useful
for raising awareness in the greater public about the disease and
its scientific process. This paper describes the motivation
behind the adoption of a musification approach and the musical
criteria applied to the data mapping process. Details regarding
the mapping structure are illustrated in relation to the different
phases of the progress of the disease. The results are then
discussed, noting that adopting a musification approach not
only helped in obtaining a more engaging audience experience
but also in providing expressive solutions that would be useful
for modeling other complex biomedical data and processes.
Bookmarks Related papers MentionsView impact
This paper describes the implementation of gestural mapping
strategies for performance with a tr... more This paper describes the implementation of gestural mapping
strategies for performance with a traditional musical
instrument and electronics. The approach adopted is informed
by embodied music cognition and functional categories
of musical gestures. Within this framework, gestures
are not seen as means of control subordinated to the resulting
musical sounds but rather as signicant elements contributing
to the formation of musical meaning similarly to
auditory features. Moreover, the ecological knowledge of
the gestural repertoire of the instrument is taken into account
as it denes the action-sound relationships between
the instrument and the performer and contributes to form
expectations in the listeners. Subsequently, mapping strategies
from a case study of electric guitar performance will be
illustrated describing what motivated the choice of a multimodal
motion capture system and how dierent solutions
have been adopted considering both gestural meaning formation
and technical constraints.
Bookmarks Related papers MentionsView impact
This work describes a new approach to gesture mapping in a
performance with a traditional musica... more This work describes a new approach to gesture mapping in a
performance with a traditional musical instrument and live
electronics inspired by theories of embodied music cognition
(EMC) and musical gestures. Considerations on EMC and
how gestures aect the experience of music inform dierent
mapping strategies. Our intent is to enhance the expressiveness
and the liveness of performance by tracking gestures
via a multimodal motion capture system and to use motion
data to control several features of the music. We then
describe an application of such approach to a performance
with electric guitar and live electronics, focusing both on
aspects of meaning formation and motion capturing.
Bookmarks Related papers MentionsView impact
Amyotrophic lateral sclerosis (ALS), also referred to
as Motor Neurone Disease in the UK, is the... more Amyotrophic lateral sclerosis (ALS), also referred to
as Motor Neurone Disease in the UK, is the most
common of the five motor neuron diseases.
Communicating about the behaviour of ALS with a
wider audience remains one of the main goals of
charities and associated organisations driving
medical research and providing care for sufferers of
ALS. This work presents a way of employing
immersive media in order to illustrate the disease
processes that characterise the progress of ALS
and thus help raise awareness in the greater
public.
Bookmarks Related papers MentionsView impact
Sample papers (see http://www.duncanwilliams.info) by Federico Visi
Amyotrophic lateral sclerosis (ALS) also referred to as Motor Neurone Disease in the UK, is the m... more Amyotrophic lateral sclerosis (ALS) also referred to as Motor Neurone Disease in the UK, is the most common of the five motor neuron diseases. Communicating about the behaviour of ALS with a wider audience remains one of the main goals of charities and associated organisations driving medical research and providing care for sufferers of ALS. This work presents a way of employing immersive media in order to illustrate the disease processes that characterise the progress of ALS and thus help raise awareness in the greater public.
Bookmarks Related papers MentionsView impact
Uploads
Papers by Federico Visi
The focus of this approach is to measure the expressiveness from distinct characters during the performance of the musical piece while decisions are made by the machine.
This multimodal approach is motivated by the Three Chamber Micro Songs composition, where measures of loudness and body motion are used by the algorithm to choose a particular musical end.
and body movements of a viola player resulting from
the variation of bow strokes length and quantity. Within
the theoretical framework of embodied music cognition
and the study of musical gestures, we aim to observe how
the variation of a musical feature within the piece affects
the body movements of the performer. Two brief pieces
were performed in four different versions, each one with
different directions regarding the bow strokes. The performances
were recorded using a multimodal recording platform
that included audio, video and motion capture data
obtained from high-speed tracking of reflective markers
placed on the body of the performer and on the instrument.
We extracted measurements of quantity of motion
and velocity of different parts of the body, the bow and the
viola. Results indicate that an increased activity in soundproducing
and instrumental gestures does not always resonate
proportionally in the rest of the body and the outcome
in terms of ancillary gestures may vary across upper
body and lower body.
modeled from published scientific data related to the
pathophysiology of amyotrophic lateral sclerosis (ALS). The
work aims to create an engaging multimodal experience useful
for raising awareness in the greater public about the disease and
its scientific process. This paper describes the motivation
behind the adoption of a musification approach and the musical
criteria applied to the data mapping process. Details regarding
the mapping structure are illustrated in relation to the different
phases of the progress of the disease. The results are then
discussed, noting that adopting a musification approach not
only helped in obtaining a more engaging audience experience
but also in providing expressive solutions that would be useful
for modeling other complex biomedical data and processes.
strategies for performance with a traditional musical
instrument and electronics. The approach adopted is informed
by embodied music cognition and functional categories
of musical gestures. Within this framework, gestures
are not seen as means of control subordinated to the resulting
musical sounds but rather as signicant elements contributing
to the formation of musical meaning similarly to
auditory features. Moreover, the ecological knowledge of
the gestural repertoire of the instrument is taken into account
as it denes the action-sound relationships between
the instrument and the performer and contributes to form
expectations in the listeners. Subsequently, mapping strategies
from a case study of electric guitar performance will be
illustrated describing what motivated the choice of a multimodal
motion capture system and how dierent solutions
have been adopted considering both gestural meaning formation
and technical constraints.
performance with a traditional musical instrument and live
electronics inspired by theories of embodied music cognition
(EMC) and musical gestures. Considerations on EMC and
how gestures aect the experience of music inform dierent
mapping strategies. Our intent is to enhance the expressiveness
and the liveness of performance by tracking gestures
via a multimodal motion capture system and to use motion
data to control several features of the music. We then
describe an application of such approach to a performance
with electric guitar and live electronics, focusing both on
aspects of meaning formation and motion capturing.
as Motor Neurone Disease in the UK, is the most
common of the five motor neuron diseases.
Communicating about the behaviour of ALS with a
wider audience remains one of the main goals of
charities and associated organisations driving
medical research and providing care for sufferers of
ALS. This work presents a way of employing
immersive media in order to illustrate the disease
processes that characterise the progress of ALS
and thus help raise awareness in the greater
public.
Sample papers (see http://www.duncanwilliams.info) by Federico Visi
The focus of this approach is to measure the expressiveness from distinct characters during the performance of the musical piece while decisions are made by the machine.
This multimodal approach is motivated by the Three Chamber Micro Songs composition, where measures of loudness and body motion are used by the algorithm to choose a particular musical end.
and body movements of a viola player resulting from
the variation of bow strokes length and quantity. Within
the theoretical framework of embodied music cognition
and the study of musical gestures, we aim to observe how
the variation of a musical feature within the piece affects
the body movements of the performer. Two brief pieces
were performed in four different versions, each one with
different directions regarding the bow strokes. The performances
were recorded using a multimodal recording platform
that included audio, video and motion capture data
obtained from high-speed tracking of reflective markers
placed on the body of the performer and on the instrument.
We extracted measurements of quantity of motion
and velocity of different parts of the body, the bow and the
viola. Results indicate that an increased activity in soundproducing
and instrumental gestures does not always resonate
proportionally in the rest of the body and the outcome
in terms of ancillary gestures may vary across upper
body and lower body.
modeled from published scientific data related to the
pathophysiology of amyotrophic lateral sclerosis (ALS). The
work aims to create an engaging multimodal experience useful
for raising awareness in the greater public about the disease and
its scientific process. This paper describes the motivation
behind the adoption of a musification approach and the musical
criteria applied to the data mapping process. Details regarding
the mapping structure are illustrated in relation to the different
phases of the progress of the disease. The results are then
discussed, noting that adopting a musification approach not
only helped in obtaining a more engaging audience experience
but also in providing expressive solutions that would be useful
for modeling other complex biomedical data and processes.
strategies for performance with a traditional musical
instrument and electronics. The approach adopted is informed
by embodied music cognition and functional categories
of musical gestures. Within this framework, gestures
are not seen as means of control subordinated to the resulting
musical sounds but rather as signicant elements contributing
to the formation of musical meaning similarly to
auditory features. Moreover, the ecological knowledge of
the gestural repertoire of the instrument is taken into account
as it denes the action-sound relationships between
the instrument and the performer and contributes to form
expectations in the listeners. Subsequently, mapping strategies
from a case study of electric guitar performance will be
illustrated describing what motivated the choice of a multimodal
motion capture system and how dierent solutions
have been adopted considering both gestural meaning formation
and technical constraints.
performance with a traditional musical instrument and live
electronics inspired by theories of embodied music cognition
(EMC) and musical gestures. Considerations on EMC and
how gestures aect the experience of music inform dierent
mapping strategies. Our intent is to enhance the expressiveness
and the liveness of performance by tracking gestures
via a multimodal motion capture system and to use motion
data to control several features of the music. We then
describe an application of such approach to a performance
with electric guitar and live electronics, focusing both on
aspects of meaning formation and motion capturing.
as Motor Neurone Disease in the UK, is the most
common of the five motor neuron diseases.
Communicating about the behaviour of ALS with a
wider audience remains one of the main goals of
charities and associated organisations driving
medical research and providing care for sufferers of
ALS. This work presents a way of employing
immersive media in order to illustrate the disease
processes that characterise the progress of ALS
and thus help raise awareness in the greater
public.