JP2008216486A - Music reproduction system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To propose a music reproduction system which adaptively selects a music piece suitable for a user according to user's feeling without requesting the user to perform complicated operation, by using a sound feature amount. <P>SOLUTION: The feature amount is extracted from each music piece as preprocessing, and feature amount space is held based on the music feature (step S0). When reproduction is started, a music piece is selected from a music piece group, and reproduced (step S1). Basically the music piece is reproduced in a music score order, but it is reproduced at random when it comes to reproduction from the beginning to when the music score is updated. When a user skips " a music piece to which the user does not like to listen" (step S2), the music score which is a priority degree of the music piece is calculated according to music selection algorithm, while referring to the feature amount space of the music piece, and updates it while succeeding the music score (step S3). A play list of the music pieces are dynamically updated by repeating the steps S1 to S3. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a music reproduction system for automatically selecting and reproducing music from a stored music group.

In recent years, with the increase in hard disk capacity and network speed, we have been able to use PCs (personal computers) as audio players to obtain, store and play back large amounts of music. In addition, with the spread of portable hard disk players, it has become possible to easily take a large amount of music anywhere. In the future, such changes in the style of music reproduction will continue to progress.
Junichi Kyokumoto, “Media playback mechanism that dynamically adapts to user preferences (UniversalPlaylist)”, Interaction 2005, 2005

  When it becomes possible to play while handling a large amount of music in this way, “I do not want to select the song to listen to myself, but I want to play some appropriate music”, “Random playback and unexpected music selection I want to enjoy it, but I want to omit songs that do n’t fit my mood. ” However, the current music playback interface has changed little compared to the conventional one, and it is hard to say that these requirements are fully met. Looking at the current music playback interface, as a function to respond to such demands, the “playlist function” in which a group of songs that the user wants to listen to is designated and held in advance, or the music is completely random "Random playback function" that plays in random order is provided. However, in the “playlist function”, the user must first manually register the music in the playlist, and since the playlist does not change, there is no surprise, and if the user gets bored with the playlist, It takes time and effort to create another playlist manually. On the other hand, the “Random Playback Function” does not require the user to specify the music, but the music on the hard disk is randomly selected regardless of the genre or the tone of the artist. It tends to be a bad song selection, and the song that the user wants to listen to is not always played.

  As related research on automatic music selection, multiple playlists of users are treated as a set, playlist evaluation weight is dynamically changed according to user's preference, and the priority of each song is calculated from the playlist weight There are researches on algorithms to perform (Non-Patent Document 1). However, the selection of music by metadata, etc. can make detailed settings such as age, genre, artist, experience, etc., but the user is required to perform complicated operations such as thinking a lot and adding data. . In addition, there is a problem that it takes a long time until sufficient metadata is stored and can be used effectively, and music selection according to mood changes or music selection that gives unexpectedness cannot be performed. In addition, there is a playback method in which music is ranked by the user, but in the first place, adding metadata is an operation that places a heavy burden on the user.

  This music selection problem can be regarded as a kind of music classification / identification problem by acoustic features. Research to deal with such problems, such as research on music genre classification based on information extracted from the power spectrum, research on evaluation of similarity and subjective similarity based on acoustic information extracted from the power spectrum, etc. Has been done. However, it cannot be said that a system and a playback interface that make effective use of acoustic features and select music according to the changing user's mood have been sufficiently developed.

  Therefore, an object of the present invention is to propose a music playback system that uses acoustic features and adaptively selects music suitable for the user according to the user's mood without requiring a complicated operation from the user.

  According to a first aspect of the present invention, there is provided a music reproduction system for reproducing music in accordance with a playlist indicating the reproduction order of music, an operation input means operable by a user, and music data storage means for storing and holding a plurality of music data A feature amount space generating means for extracting a predetermined feature amount from the song data and generating a feature amount space representing a similarity relationship between the songs, and a song designated by a predetermined operation input from the operation input means A playlist generation means for determining a priority for reproduction of each piece of music data by obtaining a distance between the data and each piece of music data in the feature amount space, and updating the playlist based on the priority; ing.

  In this way, there is no need for data addition by the user, so no complicated operation is required from the user, and a playlist that adapts to the user's mood is automatically and automatically generated while keeping the unexpectedness of music selection. can do.

  In the music reproduction system according to claim 2 of the present invention, a first information processing apparatus including the operation input unit, the music data storage unit, and the playlist generation unit, and a feature amount space generation unit includes the feature amount space generation unit. 2 information processing apparatuses.

  In this way, it can be mounted on a relatively simple device such as a portable music player.

  The music reproduction system according to claim 3 of the present invention is characterized in that the feature quantity space generating means generates the feature quantity space using beat information of music as an element of the feature quantity.

  The music reproduction system according to claim 4 of the present invention is characterized in that the feature amount space generating means generates the feature amount space using timbre information of music as the element of the feature amount.

  The music reproduction system according to claim 5 of the present invention is characterized in that the feature quantity space generating means generates the feature quantity space using music power information as an element of the feature quantity.

  In this way, it is possible to capture the subjective and objective features of the music well, and to make the feature quantities separated and distributed according to the user's mood in the feature quantity space.

  The music playback system according to claim 6 of the present invention is characterized in that the predetermined operation input is a skip operation.

  In this way, it is possible to specify a song that you do not want to listen to by a natural operation performed by the user when a song that you do not want to listen to is played.

  The music playback system according to claim 7 of the present invention is characterized in that the playlist generation means sets the priority higher as the distance in the feature amount space is longer.

  In this way, music that is completely different from the music that the user does not want to listen to is preferentially reproduced, so that music suitable for the user's mood is easily reproduced.

  The music reproduction system according to claim 8 of the present invention is characterized in that the playlist generation means sets the priority higher as the distance in the feature amount space is shorter.

  In this way, music similar to the music specified by the user is reproduced, so that a music recommendation system can be realized.

  According to the present invention, it is possible to propose a music playback system that uses acoustic feature amounts and adaptively selects music suitable for a user according to the user's mood without requiring a complicated operation from the user.

  The present invention relates to a music playback system that automatically selects music according to a user's preference without requiring annoying operation by the user, and is based on the relationship between subjective preference and acoustic characteristics of musical sound. We propose a method for adaptively selecting music that matches the user's mood through online learning during music playback.

  Specifically, when the user decides that he / she does not want to listen to a certain song, the system recognizes the operation of skipping this “unwanted song” and applies an algorithm for calculating the priority. However, the playlist is created in the order of the music that the user wants to listen to. As the algorithm, an algorithm for selecting a music piece that is far from those music pieces in a feature amount space that represents a similar relationship between the music pieces is applied.

  As an advantage of the present invention, since an acoustic feature quantity is first used, the user is not required to perform any operation such as adding metadata. Furthermore, it is a natural operation for the user to perform an operation of skipping “a song that he / she does not want to listen to”, and the system can be used as in the conventional operation method. Therefore, it can be mounted on small devices such as portable players. In addition, the user skips the song by his / her own intention, but can leave the unexpectedness of the song selection because he does not know how the playlist is calculated. Since the calculation is performed every time the user skips “Songs that he / she does not want to listen to” and dynamically creates a playlist, there are as many playlists as the number of combinations of songs. Furthermore, when the user adds music, the feature space also changes, and the music selection changes accordingly.

  In the present invention, it is assumed that a music file of about several thousand songs stored in, for example, a user's music player or an HDD (hard disk drive) of a PC is handled as an object. Based on the premise that the music that enters the HDD is the music that the user wants to listen to, a system that can select music that matches the mood of the user during playback is proposed.

  It is assumed that the user is a general music player and can only perform standard operations such as “play” and “next song”. When a user plays a series of music, it makes a simple decision as usual, “Skip this song because I don't want to listen to it now.” Aiming to avoid selecting songs that have a high degree of similarity to the “songs you do not want to listen to”. Since the user's selection is sequentially reflected in the music selection, even when the music that the user does not want to listen to changes depending on the mood or the situation, this can be reflected in the music selection. In addition, since this system performs identification based on the acoustic characteristics of music, music selection such as artist and genre can be performed without depending on metadata attached to the music.

  It is a practical advantage that this method can be realized without changing the conventional interface even in a limited user interface such as a portable player. In addition, as an application example of the present invention, it can be expected that the present invention is applied to the realization of a music recommendation system by analyzing an acoustic signal and music search.

  Of course, it goes without saying that the present invention is not limited to application under the general conditions described above, and has various expandability.

  FIG. 1 schematically shows the flow of processing in this system. A more detailed processing flow will be described later.

  Music data is a general term for data handled by this system. At least, a music group that is a set of music (audio data) to be played, music characteristics for determining the similarity of each music, and music selection And the music score that is the priority of. As preprocessing, feature quantities such as beat information, timbre information, and power configuration information are extracted from the waveform signals of each song in the song group, and a feature quantity space is held based on the song features (step S0). When the reproduction is started, the music is selected from the music group and reproduced (step S1). At this time, playback is basically performed in the order of the music score, but since the initial value of the music score is the same for each music as will be described later, the playback from the beginning until the music score is updated is random. Is selected. When the user skips “a song that you do not want to listen to” (step S2), the music score, which is the priority of the music, is calculated according to the music selection algorithm while referring to the music feature space, and the music score is inherited. Update (step S3). By repeating steps S1 to S3, the music playlist is dynamically changed.

  As described above, as a pre-process of the present system, it is necessary to first extract the feature amount of the acoustic signal necessary for the music selection algorithm. There are many candidates for the feature quantity used for acoustic analysis, but the music selection algorithm of the present invention performs calculations using the distribution in the feature quantity space, so it is very important to select the feature quantity appropriately. . The feature quantity effective for improving the performance is considered to be a feature quantity that captures the subjective and objective features of the music well and is separated and distributed in the feature quantity space according to the user's mood.

  Rhythm, tempo, chord progression, pitch information, music structure information, etc. can be considered as the characteristics of the music, but such characteristics are complicated and it is very difficult to derive by calculation. However, it is considered that the characteristics of the music can be appropriately expressed by using feature amounts related to those features.

  There are the following three types of feature quantities suitable for implementing the present invention. (1) Beat information (beat spectrum) such as rhythm and tempo, (2) Tone information (average MFCC), (3) Power information (power histogram).

  Details of each feature amount are shown below.

  The extraction of the first beat information will be described. As an effective method for extracting rhythm information, there is a method for calculating a feature amount called a beat spectrum, which is rhythm information, using autocorrelation of a short-time feature amount of an acoustic signal. This method calculates the beat using the similarity in the music, and does not limit the bandwidth, so the music does not include rhythm instruments such as drums and bass, and the volume is low (including the silence) It can be widely applied to music containing a lot of music. Therefore, it is used for research to clarify the similarity of rhythm between songs, research to search for musical sounds using rhythmic similarity, and research to visualize the distribution of musical sounds based on acoustic features.

  The procedure for calculating the beat spectrum is as follows. (1) Parameterization of acoustic signal, (2) Calculation of frame similarity, (3) Creation of distance matrix, (4) Derivation of beat spectrum.

  The beat spectrum calculation procedure will be specifically described below with reference to FIGS.

(1) Parameterization of acoustic signal There are various methods for parameterizing an acoustic signal. Here, a logarithmic power spectrum is used. As shown in FIG. 2, the acoustic signal is converted into a logarithmic power spectrum. As conditions, the window length was 256 points and the shift size was 128 points. The acoustic signal used here is a signal quantized at 16 bits and sampled at 22 kHz. Therefore, the frame length is about 11 ms. The beat spectrum is extracted every 10 seconds with a time window of 10 seconds.

(2) The similarity is obtained for all combinations of the frames of the signal for which the frame similarity is obtained. As shown in FIG. 3, the Eugrid distance in a linear space is used as a distance measure.

The dependence on the magnitude can be smoothed by taking the cosine of the feature vector.

(3) Creating a Distance Matrix As shown in FIG. 4, a distance matrix is created in which the similarity between frames enters each element. At this time, the element in the i-th row and j-th column has a similarity between v _i and v _j .

(4) A simple calculation method of the beat spectrum for obtaining the beat spectrum is as follows.

The beat spectrum is obtained over one piece of music, and the average value is set as a one-dimensional feature amount.

  FIG. 5 shows a beat spectrum of a rock music of a certain time as an example of the above calculation result. In the figure, the shape of the beat spectrum has a repetitive structure, which is considered to capture the characteristic of relatively strong beat repetition of rock.

  The second long-term MFCC will be described. A long-term MFCC obtained by averaging MFCC (mel-frequency cepstral coefficients), which is a feature amount representing a spectral envelope, over one song is used as a feature amount representing the tone color of the song. The calculation method of long-term MFCC is as follows. First, the MFCC of all frames is calculated for a whole musical piece with a window length of 4096 points and a shift width of 1280 points. The number of filter banks at the time of calculation of MFCC is 40, and the required number of dimensions is 13 dimensions. An average of all the calculated MFCCs over one song is obtained. Of these, the 12th dimension excluding the first dimension, which is a direct current component, is the long-term MFCC of the music.

  The third power histogram will be described with reference to FIGS. A short-time power histogram was used as a feature value representing the composition of the power of music. This is a feature value obtained by making the power for each hour into a histogram, which can express the intensity of the swell of the music, and can be considered to capture the structural characteristics of the music. The calculation method of the power histogram is as follows.

  First, regarding the waveform data W (t) (t is time) of the music shown in FIG. 6, the short-time power is calculated at regular time intervals throughout the music. That is, the power is obtained while moving the window for dividing the time by the width size seconds by shift seconds. Size and shift may be set to appropriate values such as 0.8 seconds (no overlap). As shown in FIG. 7, the power of each window is obtained. The formula for calculating the power of the i-th window is as follows.

Here, the calculation is simply performed by the sum of squares, but it is also conceivable to obtain the power in a logarithm (decibel) to match the human hearing.

  Next, as shown in FIG. 8, the Max value, which is the largest value of the short-time power, is defined and divided into 10 equal parts from the maximum value (Max value) of the short-time power to the minimum value. Create a 10-segment histogram. At this time, the boundary value of the histogram is defined from the Max value. Specifically, 0 is set as the first boundary value, and a total of 11 boundary values are created.

  Two methods for creating a histogram are exemplified below.

  A method for creating the first histogram will be described. The number of frames whose power value is greater than the i-th boundary value is defined as the i-dimensional value of the histogram. Therefore, since the first dimension is the number of frames having power greater than 0, the value of the first dimension is equal to the total number of frames with short-time power. Each obtained dimension from 1 to 11 is divided by the total number of frames. Since it is divided by the total number of frames, the value of the first dimension is always 1. In other words, normalizing the number of frames included in a song to 1 prevents the shape of the histogram from changing because the time length differs for each song (time normalization). Then, 2 to 11 dimensions excluding the first dimension, which is all 1 to 1 to 11 dimensions, are set as 10-dimensional feature values. As an example of the calculation result using the first histogram creation method, FIG. 9 shows a histogram created for a certain classical music piece, and FIG. 10 shows a histogram created for a certain pop music piece.

  A method for creating the second histogram will be described. A total of 10 bins are created with 0-1 / 10 × Max as the first bin, 1/10 × Max as the second bin, and so on. The number of frames in the music whose power value is within the range of the bin power value is taken as the bin value. Divide each bin by the total number of frames (time normalization). Then, the values of the 1st to 10th bins are used as 10-dimensional feature values. As an example of a calculation result using the second histogram creation method, FIG. 11 shows a histogram created for a certain classical music piece, and FIG. 12 shows a histogram created for a pop music piece.

  The 10-dimensional feature amount obtained in this way is used. In this feature quantity, it is considered that the feature quantity also changes because the power value changes depending on the window length for taking power.

  Hereinafter, the processing procedure of the music selection algorithm will be described in detail with reference to the flowcharts of FIGS. Here, x (i) is a feature vector of music i, and s (i, j) is a score in j-th music selection of music i.

  As pre-processing of a music selection algorithm (not shown), the above-described three feature quantities are extracted for all music pieces, normalized, and held for each music piece.

  In FIG. 13, at the time of starting the system, the initial value s (i, 1) (i = 1, 2,... N) of the same music score is given to all n pieces of music (step S10).

  Of the n songs, one is randomly selected and played (step S11). If the reproduced song is a song that the user wants to listen to now, the user listens to the randomly reproduced song without any operation. This is repeated until there is a user input in step S13 (steps S11 and S12). When a music piece that the user does not want to listen to appears at the j-th music selection, the user performs a skip operation (steps S12 and S13), and the process proceeds to step S14.

  The music score is updated in step S14, and the playlist is reconfigured in step S15 based on the new music score. Here, these playlist reconstruction processes will be described in detail with reference to FIG. At the time of the skip operation in step S13, the music is designated as “I do not want to listen”, the music index at that time is set to “dislike”, and the music index of the first N songs in the playlist is set to i (step S20). For all N songs, the distance d () between the position vector x (dislike) on the feature amount space of the song that you do not want to hear and the position vector x (i) of each other song specified in step S13. i) is calculated, and the reciprocal number is subtracted from the music score according to the following equation (steps S21 to S24).

Steps S20 to S24 so far correspond to step S14. Based on the calculation result of the music score in step S14, the playlist is rearranged so as to be played in order from maxs (i, j) having the highest music score among the music not selected. Is updated (step S25). If there are a plurality of such songs, the playback order is determined randomly. This step S25 corresponds to step S15. Note that the music score calculation method is not limited to Equation 4, and a function that monotonously increases with the value of d (i) is F (d), and s (i, j-1)-(1 / F (d) ) May be used for calculation. Further, for example, various modifications are possible, such as adding d to the initial value of the music score and rearranging the playlist so that the music score is reproduced in order from the min (i, j).

  Returning again to FIG. 13, the next song is played back in the order of the reconfigured playlist (step S16). If the reproduced music is a song that the user wants to listen to now, the user listens to the reproduced music in the order of the playlist without performing any operation. This is repeated until there is a user input in step S18 (steps S16 and S17). When a music piece that the user does not want to listen to appears in the j-th music selection, the user performs a skip operation (steps S17 and S18), whereby the music piece is designated as "I do not want to listen". The music index at that time is set to “dislike”, and thereafter steps S14 to S18 are repeated.

  An experiment conducted based on this music selection algorithm will be described.

  As the music data for analysis, each subject asks each subject to listen to music (class 1) when he / she wants to calm down from the music data held in his / her own PC or portable music player (class 2). ), 30 songs were listed for each of the 3 classes of songs (class 3) that I wanted to listen to when I wanted to raise my mood.

  For subjects A and B, no genre or artist was specified. The genre of the song was not specified, but it was chosen freely. As a result, Subject A played Western rock, pop, techno, etc., or instrumental. Selected instrumental music including. Subject C has 90 songs of the same artist whose genre is rock.

  Note that all the songs used here are 16 bit quantization, 22 kHz sampling frequency, and monaural.

  An analysis was performed to examine what kind of characteristics each extracted feature quantity shows. The data used for the analysis is for one subject B.

  First, the beat spectrum was analyzed. Since the beat spectrum obtained in the above feature quantity extraction is a high-order feature quantity, there is a problem of how it is reduced. This time, several reduction methods were tried. Among them, since the beat spectrum is a technique that uses the similarity in the same music, the average value of all beat spectra is used as the feature value under the idea that the beat spectrum value itself indicates the characteristics of the beat structure. This method was adopted because it gave the best results. FIG. 15 shows a histogram of the feature value for each class. The feature quantity is normalized so that the average is 0 and the variance is 1. FIG. 15 shows that although there are overlapping portions between classes, they are generally distributed by class.

  Second, a long time MFCC analysis was performed. In the long-time MFCC obtained this time, a total of 12 dimensions from the 2nd dimension to the 13th dimension were used as feature values except for the 1st dimension which is a direct current component. FIG. 16 shows a result of performing principal component analysis on a 12-dimensional feature amount for each class and representing the value of the first principal component in a histogram. The feature values are normalized so that the average is 0 and the variance is 1 for each dimension. Looking at the table, it can be seen that class1 and class2 have large overlapping parts, but class3 is still different in value from other classes.

  Third, power histogram analysis was performed. Principal component analysis was performed on the total 10-dimensional power histogram obtained this time, and the distribution of the first principal component for each class was shown as a histogram (FIG. 17). The feature values are normalized so that the average is 0 and the variance is 1 for each dimension. Looking at the table, it is not possible to completely separate, but you can see that the overlap of histograms for each class is small.

  Then, all feature values were integrated. When applying the features to the system, the beat spectrum and power histogram were normalized for each dimension so that the average was 0 and the variance was 1. However, it is considered that such normalization is not appropriate for the long-term MFCC because the importance of features represented in each dimension is different. Therefore, the weight w was determined experimentally in the above experimental procedure. While changing w for subjects A, B, and C, the area between the reference line and the overall evaluation curve was calculated in the results of each subject, and w = 14 was set at which the sum of them was maximized.

  A principal component analysis was performed on the feature amount set for the subject B, and the distribution was examined with the first principal component on the horizontal axis and the second principal component on the vertical axis. The results are shown in FIG. Looking at this distribution, it can be seen that although there are overlapping portions in each class, a distribution for each class is roughly formed. Therefore, the feature quantity used in this system is a total of 23 dimensions, that is, a normalized one-dimensional beat spectrum, a long-term MFCC 12-dimensional with weight w, and a normalized power histogram 10-dimensional. FIG. 19 shows an image diagram when the music selection algorithm is applied in the feature amount space shown in FIG. For a piece of music that is not desired to be listened to (for example, a feature amount of class 1 in the lower right corner), a piece of music at a long distance is a piece of music that the user may listen to. The music score of each music for the music that you do not want to listen to is represented by S- (1 / d). S is the initial value of the music score, and d is the distance from the music that you do not want to listen to. In FIG. 19, the principal component analysis is performed so that it can be expressed in two dimensions so that it can be easily imaged. However, as described above, the actual distance calculation is performed in the 23-dimensional space.

  An experiment was conducted to investigate the performance of the integrated feature set when applied to a music selection algorithm.

  First, data for one subject is prepared. In the music selection algorithm described above, random playback is performed in (1), and the music selection is sequentially adapted according to the music selected by the user as “I do not want to listen”. However, in this experiment, performance is evaluated by the class added to the music.

  In order to reproduce the actual usage situation, the music selection algorithm is applied on the assumption that one of the three classes described above is a song that you do not want to listen to now.

  30 sets of trials are performed for each subject and each class. Each trial starts with one of the “I don't want to listen” classes and continues until all 90 classes selected by the subject have been played. In this 30 times x 3 sets of music selection, the average number of cumulative appearances of songs that you don't want to listen to at each music selection position is averaged, and this is the overall evaluation of the three classes of performance of the music selection algorithm. Compare the straight line when 30 songs appear with the same appearance rate to the reference line, that is, the straight line in the case of completely random music selection. In addition, only class 1, 2 and 3 were also investigated.

  The experimental results of each subject are shown in FIGS. The horizontal axis is the number of times a song has been selected, and the vertical axis is the cumulative number of songs that belong to the class designated as not to be listened to. In each figure, overall evaluation (all), evaluation that averaged only class 1 (class 1), evaluation that averaged only class 2 (class 2), and evaluation that averaged only class 3 (class 3) are shown. The reference line (base), which is the case where 30 songs appearing uniformly at the same rate of appearance, and the music of the class that you don't want to listen to only by the judgment of the first song, does not come to the end. The best result (best) is shown, assuming the worst result that all the songs that you don't want to listen to first appear and then other songs appear (worst). As an evaluation, the lower the evaluation line is, the lower the evaluation line is from the reference line that is assumed to be completely random. It can be seen from the results that subjects A, B, and C are all below the reference line in almost the entire area. It can be said that subject B has the highest evaluation, and subjects A and C have the same degree. In subject B, fairly high performance was obtained. Furthermore, especially in the subject C, the fact that the music of the same artist, which is almost similar music, was below the reference line can be said to show the effectiveness of the present technique. Looking at the evaluation for each class, it can be seen that the performance for class 3 is slightly better, but there was no clear difference. Subject A has a wide variety of types of music, and even if the mood in one type of music and the mood in other types of music are expressed in the same language, they do not necessarily match. The formation of a quantity space may have been difficult.

  Hereinafter, preferred embodiments of the music playback system according to the present invention will be described with reference to FIGS.

  FIG. 23 is a block diagram showing a configuration of a music playback system according to the present invention. This system mainly includes a playback function configuration unit 1 that plays back music and generates a playlist, and a feature calculation function configuration unit 2 that calculates the feature amount of each song. For example, data such as music data and feature space is exchanged between the playback function configuration unit 1 and the feature calculation function configuration unit 2.

  The playback function configuration unit 1 includes an operation input unit 3 such as buttons and input keys that can be operated by a user, an output unit 4 such as a speaker that outputs sound, and a hard disk or memory that stores a large number of music data. From the music data storage unit 5, the music data read from the music data storage unit 5, converted into an electric signal in a predetermined format and transmitted to the output unit 4, and the feature calculation function configuration unit 2 The feature amount space storage unit 7 that stores and holds the received feature amount space, the playlist generation unit 8 that calculates a music score by referring to the feature amount space and generates a playlist, and controls each component unit, for example, And a control unit 9 that executes information processing necessary for music reproduction, such as a music selection algorithm. The feature calculation function configuration unit 2 includes a feature amount space generation unit 10 that extracts a feature amount from each piece of music stored in the song data storage unit 5 and generates a feature amount space.

  FIG. 24 is a block diagram illustrating a configuration of the feature amount space generation unit 10. The feature space generation unit 10 includes a short-time power histogram extraction unit 11 that extracts power information from music data stored in the music data storage unit 5 in accordance with the above-described power histogram calculation procedure, and the above-described long-time MFCC calculation procedure. In accordance with the long-term MFCC extraction unit 12 for extracting timbre information from the music data stored in the music data storage unit 5 and the beat information from the music data stored in the music data storage unit 5 in accordance with the beat spectrum calculation procedure described above. A beat information extraction unit 13 for integrating the three feature amounts extracted by the extraction units 11, 12, and 13 to generate the feature amount space described above.

  FIG. 25 is a block diagram illustrating an aspect of an actual system configuration example. Reference numeral 20 denotes an information processing apparatus such as a computer, for example, by installing a music playback program in which the music selection algorithm described above is installed. The feature calculation function configuration unit 2 is realized on the same device.

  FIG. 26 is a block diagram illustrating another aspect of an actual system configuration example. 22 is an information processing apparatus such as a computer, for example, similar to the information processing apparatus 20. Each of the information processing apparatuses 20 is installed with a feature calculation program in which the power histogram, long-time MFCC, beat spectrum calculation algorithm, etc. are installed. Only the feature calculation function configuration unit 2 is realized on the device by organically connecting and coordinating hardware resources. 20 is an information processing device such as a portable music player, for example, by installing a music playback program in which the music selection algorithm described above is installed, etc. Only part 1 is implemented on the device.

  The information processing apparatus 21 and the information processing apparatus 22 are configured to be able to perform data communication. For example, data such as music data and feature amount space is exchanged. That is, when the information processing apparatus 21 is a client, the information processing apparatus 22 corresponds to a server.

  FIG. 27 shows an example of an operation screen displayed on the display device of the information processing device 20 or the information processing device 21. In the operation screen 30 shown in the figure, 31 is a display unit for a feature amount space as shown in FIG. 18, 32 is a play list display window, and 33 is a “Play” button for performing a music playback operation. 34 is a “Stop” button for performing a song stop operation, 35 is a “Skip” button for performing a song skip operation, and 36 is “I do not want to listen to” the song. A “dislike!” Button for performing a designated operation, 37 is a “Reset” button for performing a reset operation for returning the current playlist to the initial state, and 38 is a shuffle of playlist (random sorting). `` Shuffle '' button, 39 is the `` Sreset '' button for performing the es reset operation to return only the score to the initial state (initial value) without changing the song order (playlist), 40 Music data is stored This is a pull-down box for specifying a specified folder. In the screen shown in the figure, a “dislike!” Button 36 is provided separately from the “Skip” button 35 for designating that the currently reproduced music is “I do not want to listen”.

  As described above, in this embodiment, the music playback system plays back music in accordance with the playlist indicating the playback order of the music, and the operation input unit 3 that can be operated by the user and the music data that stores and holds a plurality of music data. By a predetermined operation input from the storage unit 5, a feature amount space generation unit 10 that extracts a predetermined feature amount from the music data and generates a feature amount space representing a similar relationship between the pieces of music, and a predetermined operation input from the operation input unit 3 Play list generation for determining a priority for reproduction of each piece of music data by obtaining a distance in the feature amount space between the designated piece of music data and each other piece of music data, and updating the playlist based on the priority Part 8.

  In this way, there is no need for data addition by the user, so no complicated operation is required from the user, and a playlist that adapts to the user's mood is automatically and automatically generated while keeping the unexpectedness of music selection. can do.

  In the music playback system of the present embodiment, the first information processing apparatus 21 including the operation input unit 3, the music data storage unit 5, and the playlist generation unit 8, and the second including the feature amount space generation unit 10. The information processing apparatus 22 is configured.

  In this way, it can be mounted on a relatively simple device such as a portable music player.

  Furthermore, in the music reproduction system of the present embodiment, the feature amount space generation unit 10 generates the feature amount space using the beat information, timbre information, and power information of the music as elements of the feature amount. .

  In this way, it is possible to capture the subjective and objective features of the music well, and to make the feature quantities separated and distributed according to the user's mood in the feature quantity space.

  In the music playback system of the present embodiment, the predetermined operation input is a skip operation.

  In this way, it is possible to specify a song that you do not want to listen to by a natural operation performed by the user when a song that you do not want to listen to is played.

  Furthermore, in the music reproduction system of the present embodiment, the playlist generation unit 8 sets the priority higher as the distance in the feature amount space is longer.

  In this way, music that is completely different from the music that the user does not want to listen to is preferentially reproduced, so that music suitable for the user's mood is easily reproduced.

  In the music playback system of the present embodiment, the playlist generation unit 8 sets the priority higher as the distance in the feature amount space is shorter.

  In this way, music similar to the music specified by the user is reproduced, so that a music recommendation system can be realized.

  In addition, this invention is not limited to the said Example, It can change in the range which does not deviate from the meaning of this invention.

It is a flowchart which shows roughly the flow of a process of the music reproduction system in this invention. It is a figure which shows the logarithmic power spectrum converted from the acoustic signal. It is explanatory drawing which shows the procedure which calculates | requires a frame similarity from the power spectrum figure of FIG. It is a figure which shows the distance matrix produced from the frame similarity of FIG. It is a figure which shows the beat spectrum of the rock music of a certain time. It is a wave form diagram which shows the waveform data of a music. It is a figure which shows the short time power calculated | required from the waveform shown in FIG. FIG. 8 is an explanatory diagram illustrating a procedure for defining a boundary value of a histogram in the diagram of FIG. It is a figure which shows the histogram of the classical music created according to the 1st histogram creation method. It is a figure which shows the histogram of a pop music music same as the above. It is a figure which shows the histogram of the classical music created according to the 2nd histogram preparation method. It is a figure which shows the histogram of a pop music music same as the above. It is a flowchart which shows the music selection algorithm of the music reproduction system in this invention. It is a flowchart which shows the play list production | generation process of a music selection algorithm in detail same as the above. It is a figure which shows the average value of the value of the beat spectrum calculated | required by the experiment conducted based on the music selection algorithm same as the above. It is a figure which shows the histogram of the value of the 1st main component of the MFCC12 dimension calculated | required by the experiment conducted based on the music selection algorithm same as the above. It is a figure which shows the histogram of the value of the 1st main component of the power histogram calculated | required by the experiment conducted based on the music selection algorithm same as the above. It is a figure which shows distribution of the music in the feature-value space calculated | required by the experiment conducted based on the music selection algorithm same as the above. It is explanatory drawing which shows the process image of a music selection algorithm in FIG. It is a figure which shows the experimental result with respect to the test subject A of the music reproduction system in this invention. It is a figure which shows the experimental result with respect to the test subject B of the music reproduction system in this invention. It is a figure which shows the experimental result with respect to the test subject C of the music reproduction system in this invention. It is a block diagram which shows the structure of the music reproduction system in this invention. It is a block diagram which shows the structure of a feature-value space generation part same as the above. It is a block diagram which shows the one aspect | mode of an actual system configuration example same as the above. It is a block diagram which shows another aspect of an actual system configuration example same as the above. The above shows an example of the operation screen.

Explanation of symbols

3 Operation input unit 5 Music data storage unit 8 Playlist generation unit
10 Feature space generator
21, 22 Information processing equipment

Claims (8)

A music playback system for playing music according to a playlist indicating a playback order of music pieces,
Operation input means that can be operated by the user, music data storage means for storing and holding a plurality of music data, and a feature quantity space that represents a similar relationship between each music by generating a predetermined feature quantity from the music data is generated. A feature space generation means;
A priority for reproduction of each piece of music data is determined by obtaining a distance in the feature amount space between the piece of music data designated by a predetermined operation input from the operation input unit and each piece of music data, and the priority And a playlist generating means for updating the playlist based on the music playback system. A first information processing device including the operation input unit, the music data storage unit, and the playlist generation unit; and a second information processing device including the feature space generation unit. The music playback system according to claim 1, wherein 3. The music reproducing system according to claim 1, wherein the feature amount space generating unit generates the feature amount space using beat information of music as an element of the feature amount. The music reproduction system according to any one of claims 1 to 3, wherein the feature amount space generation unit generates the feature amount space using timbre information of music as an element of the feature amount. . The music reproduction system according to any one of claims 1 to 4, wherein the feature amount space generation unit generates the feature amount space by using music power information as an element of the feature amount. . The music reproduction system according to claim 1, wherein the predetermined operation input is a skip operation. The music playback system according to any one of claims 1 to 6, wherein the playlist generation means sets the priority higher as the distance in the feature space is longer. . The music playback system according to any one of claims 1 to 7, wherein the playlist generation unit sets the priority higher as the distance in the feature amount space is shorter. .