JP2006317872A - Portable terminal device and music expression method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a portable terminal device and a music expression method capable of easily recognizing how music data being reproduced is evaluated.
SOLUTION: Impression degree data representing an impression of music data is recorded together with music data in a terminal music database 32, and a light emission color determining unit 38 impresses the degree of impression of the music data output by the audio output unit 36. A light emission color is determined based on the data, and the light emitting unit 37 that emits light with a plurality of light emission colors is caused to emit light with the light emission color determined by the light emission color determining unit. The impression degree data recorded in the terminal music database 32 is composed of a plurality of evaluation items, and the luminescent color determination unit 38 extracts the most characteristic evaluation item from the impression degree data, and stores the extracted evaluation items in advance. The light emitting unit 37 is caused to emit light with the assigned emission color.
[Selection] Figure 1

Description

  The present invention relates to a mobile terminal device and a music expression method for reproducing a plurality of music data stored in a terminal music database, and in particular, a mobile terminal device including a terminal display unit for displaying information of music data to be reproduced and music selection Regarding the method.

  In recent years, small and large-capacity storage means such as HDDs have been developed, and portable audio playback devices (hereinafter referred to as portable terminal devices) such as portable audio capable of storing a large amount of music data have been proposed. ing. Since portable portable terminal devices are miniaturized, input space and display space for operation are limited, and management of a large amount of music data stored in the storage means can be connected to the portable terminal device. It is performed by a configured personal computer.

  As management performed by the personal computer, for example, in order to make it possible to search for a song desired by the user based on a subjective impression of the song, the user's subjective requirements for the song desired to be searched are input and numerical values are entered. The predicted impression value obtained by quantifying the impression of the music to be searched is calculated from the output, and the sound signal of the plurality of music and the impression of the music are quantified using the calculated predicted impression value as a key. By searching a music database storing impression values, a desired music is searched based on a subjective image of the user's music, and the searched music data is transferred to the mobile terminal device (for example, , See Patent Document 1).

However, since the conventional technology is configured to manage a large amount of stored music data with a personal computer, the impression value of the music data is not transferred to the mobile terminal device, and the music data being played back There was a problem that it was not possible to easily recognize whether it was evaluated.
JP 2002-278547 A

  The present invention has been made in view of such problems, and an object of the present invention is to provide a portable terminal device and a music expression that can easily recognize how the music data being reproduced is evaluated. The point is to provide a method.

In order to solve the above problems, the present invention has the following configurations.
The portable terminal device of the present invention is a portable terminal device that records music data and reproduces the recorded music data, and the impression data representing the impression of the music data together with the music data. Based on the terminal music database to be recorded, audio output means for reading out and outputting the music data recorded in the terminal music database, and the impression degree data of the music data being audio-output by the audio output means A light emission color determining means for determining a light emission color and a light emission means for emitting light by switching a plurality of light emission colors, wherein the light emission means emits light with the light emission color determined by the light emission color determination means. To do.

  Furthermore, in the portable terminal device of the present invention, the impression degree data recorded in the terminal music database includes a plurality of evaluation items, and the emission color determination means is the most characteristic of the impression degree data. The evaluation item is extracted, and the light emitting means is caused to emit light with a light emission color assigned in advance to the extracted evaluation item.

  Furthermore, in the mobile terminal device of the present invention, the emission color determination means determines the most characteristic evaluation by determining which of the plurality of evaluation items of the impression degree data is closest to the minimum value or the maximum value. It is characterized by extracting items.

Furthermore, the portable terminal device of the present invention comprises terminal display means for displaying the bibliographic data of the music data that has been voice output by the voice output means,
The light emitting means is disposed at an end portion of the terminal display means, and projects light from the lateral direction with respect to the display surface of the terminal display means to cause the display surface of the terminal display means to develop color.

  The music expression method of the present invention is a music expression method for recording an image of the music data being reproduced in a portable terminal device that records the music data and reproduces the recorded music data. The impression data representing the impression of the song data is recorded together with the song data in the terminal song database, the song data recorded in the terminal song database is read out and output as voice, and the voice is output. A light emission color is determined based on the impression degree data of the music data, and a light emitting unit that emits light by switching a plurality of light emission colors is caused to emit light in the determined light emission color.

  Further, in the music expression method of the present invention, the impression degree data is composed of a plurality of evaluation items, and the most characteristic evaluation item is extracted from the impression degree data, and assigned to the extracted evaluation items in advance. The light emitting means is caused to emit light with a light emission color.

  Furthermore, the music expression method of the present invention is characterized in that the most characteristic evaluation item is extracted by determining which of the plurality of evaluation items of the impression degree data is closest to the minimum value or the maximum value. And

  Furthermore, the music expression method of the present invention is characterized in that the display surface display surface of the terminal display means for displaying the bibliographic data of the music data output as audio is colored with the determined emission color.

  According to the portable terminal device and the music selection method of the present invention, the impression data representing the impression of the song data is recorded together with the song data in the terminal song database, and the emission color is based on the impression degree data of the song data being output as sound. By making the light emission means that emits light with a plurality of light emission colors to emit light with the determined light emission color, it is possible to easily grasp the impression degree data representing the impression of the music data being reproduced by the light emission color Therefore, it is possible to easily recognize how the music data being reproduced is evaluated.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an embodiment of a music search system according to the present invention, FIG. 2 is a front view showing a configuration of the mobile terminal device shown in FIG. 1, and FIG. It is a block diagram which shows the structure of the neural network learning apparatus which learns in advance the neural network used for the music registration apparatus shown in FIG.

  In the present embodiment, referring to FIG. 1, the music registration device 10 and the mobile terminal device 30 are connected by a data transmission path 23 such as a USB, and the mobile terminal device 30 is separated from the music registration device 10. It is configured to be portable.

  The music registration apparatus 10 is an information processing apparatus that operates under program control such as a personal computer. Referring to FIG. 1, a music data input unit 11, a compression processing unit 12, a feature data extraction unit 13, and impression degree data. Conversion unit 14, music database 15, music mapping unit 16, music map storage unit 17, music search unit 18, PC operation unit 19, PC display unit 20, transmission / reception unit 21, and audio output unit 22 It consists of.

  The music data input unit 11 has a function of reading a storage medium in which music data such as a CD and a DVD is stored. The music data input unit 11 inputs music data from a storage medium such as a CD and a DVD, and extracts a compression processing unit 12 and feature data. To the unit 13. You may comprise so that the music data (delivery data) via networks, such as the internet, other than storage media, such as CD and DVD, may be input. When compressed music data is input, the compressed music data is decompressed and output to the feature data extraction unit 13.

  The compression processing unit 12 compresses the music data input from the music data input unit 11 in a compression format such as MP3 or ATRAC (Adaptive Transform Acoustic Coding) at the time of music registration, and the compressed music data includes an artist name, a music title, and the like. Are stored in the music database 15 together with the bibliographic data.

  The feature data extraction unit 13 extracts feature data composed of fluctuation information from the music data input from the music data input unit 11 and outputs the extracted feature data to the impression degree data conversion unit 14.

  The impression degree data conversion unit 14 determines the feature data input from the feature data extraction unit 13 based on human sensitivity, using a hierarchical neural network previously learned by the neural network learning device 40 shown in FIG. The impression data is converted to the impression data, the converted impression data is output to the music mapping unit 16, and the feature data input from the feature data extraction unit 13 and the converted impression data are associated with the music data to the music database. 15 to register.

  The music database 15 is a large-capacity storage unit such as an HDD, and the music data and bibliographic data compressed by the compression processing unit 12, the feature data extracted by the feature data extraction unit 13, and the impression degree data conversion unit 14. Is stored in association with the impression degree data converted by.

  The music mapping unit 16 maps the music data to a music map that is an arbitrary space based on the impression data input from the impression data conversion unit 14, and the music map in which the music data is mapped is stored in the music map storage unit 17. A search table for searching for music data mapped to the music map is created and stored, and the created search table is stored in the music database 15. As the music map, a self-organizing map that has been previously learned by the neural network learning device 40 shown in FIG. 3 is used.

  The music map storage unit 17 is a large-capacity storage unit such as an HDD, and stores a music map to which music data is mapped by the music mapping unit 16.

  The music search unit 18 searches the music database 15 based on the bibliographic data from the PC operation unit 19, displays the search result on the PC display unit 20, and plays music based on the representative music selected by the PC operation unit 19. The map storage unit 17 is searched, and the representative song search result is displayed on the PC display unit 20 and stored in the song database 15 as a search list.

  The PC operation unit 19 is input means such as a keyboard and a mouse, and inputs search conditions for searching for music data stored in the music database 15 and the music map storage unit 17 and music data to be output to the mobile terminal device 30. An input to select is made.

  The PC display unit 20 is a display unit such as a liquid crystal display, for example, displays the mapping status of the song data stored in the song map storage unit 17, and the song data stored in the song database 15 and the song map storage unit 17. The search conditions for searching for music, the searched music data (search results), etc. are displayed.

  The transmission / reception unit 21 is configured to be connectable to the transmission / reception unit 31 of the portable terminal device 30 by a data transmission path 23 such as a USB, and corresponds to the music data stored in the music database 15 and the music data. The neuron coordinates in the musical score map, the search table, and the search list are output to the transmission / reception unit 31 of the mobile terminal device 30.

The audio output unit 22 is an audio player that expands and reproduces music data stored in the music database 15, and outputs the expanded music data from the connected speaker 24 as audio.

  The portable terminal device 30 is an audio playback device such as a portable audio device having a large capacity storage means such as an HDD. Referring to FIG. 1, a transmission / reception unit 31, a terminal music database 32, a music search unit 33, and a terminal An operation unit 34, a terminal display unit 35, an audio output unit 36, a light emitting unit 37, and a light emission color determining unit 38 are included.

  The transmission / reception unit 31 is configured to be connectable to the transmission / reception unit 21 of the music registration device 10 via a data transmission path 23 such as a USB, and the music data input from the transmission / reception unit 21 of the music registration device 10 is used as the terminal music. Store in the database 32.

  The terminal song database 32 is a large-capacity storage unit such as an HDD, and the song data stored in the song database 15, impression degree data corresponding to the song data, the coordinates of neurons in the song map, a search table, The search list is stored.

  The music search unit 33 searches the terminal music database 32 based on the bibliographic data from the terminal operation unit 34, displays the search result on the terminal display unit 35, and based on the representative music specified by the terminal operation unit 34. The search table is searched, and the search result is displayed on the terminal display unit 35.

  The terminal operation unit 34 is an input unit for performing input related to the reproduction of music data. Referring to FIG. 2, input for selecting music data to be reproduced, input for selecting a search method, input for volume control, and the like are performed. It consists of a cross key 341 and an enter key 342 for performing an input for instructing output of the selected music data, an input for instructing execution of the selected search method, and the like.

  The terminal display unit 35 is a display unit such as a liquid crystal display, for example, and displays search conditions for searching for music data stored in the terminal music database 32 and displays searched music data (search results). .

  The audio output unit 36 is an audio player that expands and reproduces music data that is compressed and stored in the terminal music database 32, and outputs audio data of the expanded music data from the connected headphones 25.

  The light emitting unit 37 is composed of light emitting diodes of R (red: Red), G (green: Green), and B (blue: Blue), and is a light emitting unit that can emit light by switching a plurality of light emitting colors. The light emitting unit 37 is disposed at an end of the terminal display unit 35 and is configured to emit light from the lateral direction with respect to the display surface of the terminal display unit 35 to cause the display surface of the terminal display unit 35 to color.

  The light emission color determination unit 38 determines the light emission color of the light emission unit 37 based on the impression data of the music data reproduced by the audio output unit 36, and causes the light emission unit 37 to emit light with the determined light emission color.

  The neural network learning device 40 is a device that learns the hierarchical neural network used in the impression degree data conversion unit 14 and the music map used in the music mapping unit 16. Referring to FIG. 3, the music data input unit 41, voice output unit 42, feature data extraction unit 43, impression degree data input unit 44, combination weight value learning unit 45, music map learning unit 46, combination weight value output unit 47, and feature vector output Part 48.

  The music data input unit 41 has a function of reading a storage medium in which music data such as a CD and a DVD is stored. The music data input unit 41 inputs music data from a storage medium such as a CD and a DVD, and extracts an audio output unit 42 and feature data. To the unit 43. You may comprise so that the music data (delivery data) via networks, such as the internet, other than storage media, such as CD and DVD, may be input. When compressed music data is input, the compressed music data is decompressed and output to the audio output unit 42 and the feature data extraction unit 43.

  The audio output unit 42 is an audio player that expands and reproduces the music data input from the music data input unit 41, and outputs the audio data of the expanded music data from the connected speaker 24.

  The feature data extraction unit 43 extracts feature data composed of fluctuation information from the song data input from the song data input unit 41 and outputs the extracted feature data to the connection weight value learning unit 45.

  The impression degree data input unit 44 accepts input of impression degree data by the evaluator based on the audio output from the audio output unit 42, and uses the received impression degree data as a joint signal as a teacher signal used for learning of the hierarchical neural network. The value is output to the value learning unit 45 and is also output to the music map learning unit 46 as an input vector to the self-organizing map.

  The connection weight value learning unit 45 performs learning on the hierarchical neural network based on the feature data input from the feature data extraction unit 43 and the impression degree data input from the impression degree data input unit 44, and The connection weight value is updated, and the updated connection weight value is output via the connection weight value output unit 47. The learned hierarchical neural network (updated connection weight value) is transplanted to the impression degree data conversion unit 14 of the music registration device 10.

  The music map learning unit 46 learns the self-organizing map using the impression degree data input from the impression degree data input unit 44 as an input vector to the self-organizing map, updates the feature vector of each neuron, and The updated feature vector is output via the output unit 48. The learned self-organizing map (updated feature vector) is stored in the music map storage unit 17 of the music registration device 10 as a music map.

First, the music registration operation in the music registration apparatus 10 will be described in detail with reference to FIGS.
4 is a flowchart for explaining the music registration operation in the music registration apparatus shown in FIG. 1, and FIG. 5 is a flowchart for explaining the feature data extraction operation in the feature data extraction unit shown in FIG. FIG. 6 is a flowchart for explaining the tempo determination operation in the feature data extraction unit shown in FIG. 1, and FIG. 7 is an explanatory diagram showing an example of a hierarchical neural network used in the impression degree data conversion unit shown in FIG. 8 is an explanatory diagram illustrating an example of a music map in which the music mapping unit illustrated in FIG. 1 maps music data, and FIG. 9 illustrates an example of a search table stored in the music database illustrated in FIG. It is.

  A storage medium storing music data such as CD and DVD is set in the music data input section 11 and music data is input from the music data input section 11 (step A1).

  The compression processing unit 12 compresses the music data input from the music data input unit 11 (step A2), and stores the compressed music data in the music database 15 together with the bibliographic data such as artist name and music name (step A3). .

  The feature data extraction unit 13 extracts feature data including fluctuation information from the music data input from the music data input unit 11 (step A4).

  With reference to FIGS. 4 to 6, the feature data extraction operation in the feature data extraction unit 13 is performed when music data is input to the music data input unit 11 from a music playback device such as a CD player or a network such as the Internet (step). B1) The music data input unit 11 downsamples the music data input to the music data input unit 11 from 44.1 kHz to 22.05 kHz for the purpose of speeding up, and the downsampled music data is a feature data extraction unit. 13 is output.

  Next, the feature data extraction unit 13 performs FFT processing for a certain frame length from a predetermined data analysis start point (30 s from the beginning of the song) of the song data, and calculates a power spectrum (step B2). ). In the present embodiment, the tempo period is extracted as the tempo of the music, which is one of the feature data, and the sampling period is set to 22 .2 assuming that the tempo period is in the range of 0.3 to 1 s. It was configured to perform 1024-point FFT processing on music data at 05 kHz. That is, the frame length for performing the FFT processing is set to 1024 / 22.05 kHz≈46 ms, which is shorter than the minimum value of the tempo cycle in the assumed music.

  Next, the feature data extraction unit 13 sets frequency bands of Low (0 to 200 Hz), Middle (200 to 600 Hz), and High (600 to 11050 Hz) in advance, and the power of the three bands of Low, Middle, and High. The spectrum is integrated, the average power is calculated (step B3), and it is determined whether or not the number of frames for which the processing operations of steps B2 to B3 have been performed has reached a predetermined set value (2048) (step B4). ) If the number of frames that have undergone the processing operations in steps B2 to B3 has not reached a predetermined set value, the data analysis start point is shifted (step B5) while the processing in steps B2 to B3 is performed. Repeat the operation. As a result, the processing operations of step B2 to step B3 are performed for a set value of a predetermined number of frames, and time series data of average power in the Low, Middle, and High3 bands can be respectively obtained. In this embodiment, the analysis time length is 60 s, the data analysis start point is shifted by 60 s * 22.05 kHz / 2048≈646 points, and FFT processing is performed to obtain time series data of 2048 points and 60 s average power. Configured to create.

  Next, the feature data extraction unit 13 performs FFT on the time series data of the average power of Low, Middle, and High calculated by the processing operations of Step B2 to Step B5, and calculates fluctuation information (Step B6). In this embodiment, 2048-point FFT processing is performed on time series data of average power.

  Next, the feature data extraction unit 13 calculates an approximate straight line in a graph with the logarithmic frequency on the horizontal axis and the logarithmic power spectrum on the vertical axis from the FFT analysis results in Low, Middle, and High by the least square method or the like (step B7 ), The inclination of the approximate straight line and the Y intercept of the approximate straight line are obtained (step B8), and the slope and Y intercept of the approximate straight line in each of Low, Middle, and High are extracted as feature data.

  Further, the feature data extraction unit 13 calculates, as difference information, the difference between the fluctuation information calculated in Step B6 and the calculated approximate straight line calculated in Step B7 in the FFT analysis result in Low (Step B9). By analyzing the difference information calculated based on a predetermined logic, the tempo cycle is determined (step B10), the determined tempo is extracted as feature data, and approximate straight lines in each of Low, Middle, and High Are output to the impression degree data conversion unit 14 together with the inclination and the Y intercept.

  Note that the tempo determination operation in the feature data extraction unit 13 is, as shown in FIG. 6, first separated from the approximate straight line by 1.25 dB or more in the period 0.3 to 2 s by analyzing the difference information calculated in step B9. It is determined whether there is a frequency component that is more than 1.25 dB away from the approximate line within the frequency range of 1 / 0.3 to 1/2 Hz (step C1). If there is a frequency component separated from the straight line by 1.25 dB or more, the frequency component having the longest period among the frequency components separated from the approximate line by 1.25 dB or more is set as a candidate A (step C2).

  Next, the feature data extraction unit 13 determines whether or not the candidate A is 1 s or more (step C3). If the candidate A is 1 s or more, the candidate A / 2 is determined as the tempo period ( Step C4) If the candidate A is not 1 s or longer, the candidate A is determined as a tempo period (step C5).

  If there is no frequency component separated from the approximate line by 1.25 dB or more in step C1, the feature data extraction unit 13 determines whether there is a component separated from the approximate line by 0.7 dB or more in the period of 0.3 to 2s. That is, it is determined whether or not there is a frequency component that is separated from the approximate line by 0.7 dB or more within the frequency range of 1 / 0.3 to 1/2 Hz (step C6). If there is a frequency component that is present, among the frequency components that are separated from the approximate line by 0.7 dB or more, the cycle of the frequency component with the longest cycle is set as a candidate A (step C7).

  Next, the feature data extraction unit 13 determines whether or not the candidate A is 1 s or longer (step C8), and when the candidate A is 1 s or longer, determines the candidate A / 2 as a tempo period ( Step C9) If the candidate A is not 1 s or longer, it is determined whether the candidate A is 0.6 s or longer (Step C10). When the candidate A is 0.6 s or more, it is determined whether there are two or more frequency components apart from the approximate line by 0.7 dB or more other than the candidate A (step C11). If there are two or more frequency components separated from the straight line by 0.7 dB or more, the candidate A / 2 is determined as the tempo period (step C9). If candidate A is not 0.6 s or more in step C10, and if there are no more than two frequency components apart from the approximate line by 0.7 dB or more other than candidate A in step C11, candidate A is determined as the tempo period. (Step C12).

  If there is no frequency component separated by 0.7 dB or more from the approximate line in step C11, the feature data extraction unit 13 determines whether there is a component separated from the approximate line by 0.6 dB or more in the period of 0.3 to 2 s. That is, it is determined whether or not there is a frequency component that is separated from the approximate line by 0.6 dB or more within the frequency range of 1 / 0.3 to 1/2 Hz (step C13), and the approximate line is separated by 0.6 dB or more. If there is a frequency component that is present, among the frequency components that are separated from the approximate line by 0.6 dB or more, the cycle of the frequency component with the longest cycle is set as a candidate A (step C7).

  Next, the feature data extraction unit 13 determines whether or not the candidate A is 1 s or longer (step C8), and when the candidate A is 1 s or longer, determines the candidate A / 2 as a tempo period ( Step C9) If the candidate A is not 1 s or longer, it is determined whether the candidate A is 0.6 s or longer (Step C10). If the candidate A is 0.6 s or more, it is determined whether there are two or more frequency components apart from the approximate line by 0.6 dB or more other than the candidate A (step C11). When there are two or more frequency components separated from the straight line by 0.6 dB or more, the candidate A / 2 is determined as the tempo period (step C9), and when the candidate A is not 0.6s or more in step C10, If there are no more than two frequency components apart from the approximate straight line by 0.6B or more other than candidate A in step C11, candidate A is determined as the tempo period (step C12).

  If there is no frequency component separated by 0.6 dB or more from the approximate line in step C13, the feature data extraction unit 13 determines whether there is a component separated from the approximate line by 0.6 dB or more in the period 3 to 4s. That is, it is determined whether or not there is a frequency component that is separated from the approximate line by 0.6 dB or more within the frequency range of 1/3 to 1/4 Hz (step C14), and the frequency component that is separated from the approximate line by 0.6 dB or more. If there is a frequency component that is 0.6 dB or more away from the approximate line, the cycle of the frequency component with the longest cycle is set as the candidate A (step C15), and the candidate A / 4 is determined as the cycle of the tempo (step S15). Step C16).

  If there is no frequency component separated by 0.6 dB or more from the approximate line in step C14, the feature data extraction unit 13 has a component separated from the approximate line by 0.7 dB or more in the period of 0.1 to 0.3 s. Whether there is a frequency component that is more than 0.7 dB away from the approximate line within the frequency range of 1 / 0.1 to 1 / 0.3 Hz (step C17). When there is no frequency component separated by .7 dB or more, 1s is determined as the tempo period (step C18), and when there is a frequency component separated by 0.7 dB or more from the approximate line, 0.3 s is set. The tempo period is determined (step C19).

  In the present embodiment, the tempo period is extracted as the tempo of the music, but it may be converted into BPM (beat per minutes) generally used as a unit representing the tempo. You may make it process a tempo period with a fixed processing method.

Next, referring to FIG. 4, the impression degree data conversion unit 14 is a hierarchical type composed of an input layer (first layer), an intermediate layer (n-th layer), and an output layer (N-th layer) as shown in FIG. By inputting the feature data extracted by the feature data extraction unit 13 into the input layer (first layer) using a neural network, impression level data is output from the output layer (Nth layer), that is, the feature data is converted into the impression level. The data is converted into data (step A5), the impression degree data output from the output layer (Nth layer) is output to the music mapping unit 16, and the feature data input from the feature data extraction unit 13 and the output layer ( The impression degree data output from the (Nth layer) is stored in the music database 15 together with the music data. Note that the connection weight value w of each neuron in the intermediate layer (nth layer) is learned in advance by the evaluator. In the case of this embodiment, the feature data input to the input layer (first layer), that is, the feature data items extracted by the feature data extraction unit 13 are Low, Middle, High as described above. There are seven items, the slope of the approximate straight line and the Y-intercept and the tempo, and the evaluation items of the impression degree data are “bright and dark”, “clear and dark” judged by human sensitivity, Three items of “Intense and Calm” were set, and each evaluation item was set so as to be expressed by a seven-step evaluation. Accordingly, the number of neurons L ₁ in the input layer (first layer) is 7, and the number of neurons L _{N in} the output layer (Nth layer) is 3, and the intermediate layer (nth layer: n = 2). ,..., N-1) is set appropriately.

  The music mapping unit 16 maps the music data input from the music data input unit 11 to a corresponding portion of the music map stored in the music map storage unit 17 (step A6). The music map which is an arbitrary space used for the mapping operation in the music mapping unit 16 is divided into a plurality of partial spaces, and the music data is arranged in any of the plurality of partial spaces. As the music map, for example, a self-organizing map (SOM) in which neurons are regularly arranged in two dimensions (9 * 9 square in the example shown in FIG. 8) can be used. A neuron corresponds to multiple subspaces.

  In this embodiment, a two-dimensional SOM in which neurons are arranged in a square of 100 * 100 is used as a music map, and coordinates (1, 1) to (100, 100) are assigned to each neuron of the music map. The music mapping unit 16 creates a search table for searching for music data mapped to the music map, and uses the created search table as a music database. 15 and the coordinates of the neuron in which the music data is arranged are stored in the music database 15 together with the music data. The search table is for searching for the corresponding neuron and specifying the music data arranged in the searched neuron. As shown in FIG. 9, the coordinates of each neuron in the music map and each neuron It is composed of arranged music data (numbers specifying music data, etc.). In this embodiment, a two-dimensional SOM in which neurons are arranged in a square is used as the music map. However, the arrangement of neurons may be a rectangle or a honeycomb, and a three-dimensional SOM is used. You may do it.

In addition, the music map used for the mapping operation in the music mapping unit 16 is learned in advance, and each neuron is given a pre-learned n-dimensional feature vector m _i (t) εR ^n. cage, music mapping unit 16, and impression data that has been converted by the impression-data-conversion unit 14, and a tempo extracted by the feature data extraction unit 13 and the input vector x _j, nearest neuron to the input vector x _j, That is, the input music data is arranged in a neuron that minimizes the Euclidean distance ‖x _j −m _i 、, and the music map to which the music data is mapped is stored in the music map storage unit 17. Note that R indicates the number of evaluation stages of each evaluation item of impression degree data, and n indicates the number of items of impression degree data.

Next, the learning operation of the hierarchical neural network used for the conversion operation (step A5) in the impression degree data conversion unit 14 will be described in detail with reference to FIG.
FIG. 10 is a flowchart for explaining the learning operation of the hierarchical neural network in the neural network learning apparatus shown in FIG.

  Learning of the hierarchical neural network (connection weight value w) by the evaluator is performed using, for example, the neural network learning device 40 shown in FIG. 2. First, the hierarchical neural network (connection weight value w) is pre-learned. The pre-learning data (feature data feature data + impression degree data) is input.

  A music medium such as a CD or DVD is set in the music data input unit 41, music data is input from the music data input unit 41 (step D1), and the feature data extraction unit 43 inputs the music data. Feature data is extracted from the music data input from the unit 41 (step D2). The feature data extracted by the feature data extraction unit 43 is the same as the feature data extracted by the feature data extraction unit 13 of the music registration device 10.

  The audio output unit 42 outputs the music data input from the music data input unit 41 as audio (step D3), and the evaluator listens to the audio output from the audio output unit 42 to determine the impression level of the music. Evaluation is performed based on sensitivity, and the evaluation result is input as impression degree data from the impression degree data input unit 44 (step D4). The combined weight value learning unit 45 uses the impression degree data input from the impression degree data input unit 44 as a teacher signal. Accept as. In the present embodiment, as the evaluation items for impression degree, three items of “bright, dark”, “clear, fuzzy”, and “violent, gentle” determined by human sensitivity are set. A seven-stage evaluation for the evaluation item is configured to be received by the impression degree data input unit 44 as impression degree data.

Next, it is determined whether a learning data consisting of characteristic data and the inputted impression data reaches the number of samples T ₁ for a predetermined (step D5), until the learning data reaches the number of samples T ₁ Steps D1 to D4 are repeated.

Learning of the hierarchical neural network in the connection weight value learning unit 45, that is, the update of the connection weight value w of each neuron is performed using an error back propagation learning method.
First, as an initial value, the connection weight value w of all the neurons of the intermediate layer (nth layer) is set to a small value in the range of about −0.1 to 0.1 by a random number, and the connection weight value learning unit 45 inputs the feature data extracted by the feature data extraction unit 43 to the input layer (first layer) as an input signal x _j (j = 1, 2,..., 8), and from the input layer (first layer). The output of each neuron is calculated toward the output layer (Nth layer).

Next, the combined weight value learning unit 45 uses the impression degree data input from the impression degree data input unit 44 as a teacher signal y _j (j = 1, 2,..., 8), and outputs the output layer (Nth layer). The learning rule δ _j ^N is calculated from the error between the output out _j ^N and the teacher signal y _j by the following equation.

Next, the joint weight value learning unit 45 calculates the error signal δ _j ⁿ of the intermediate layer (nth layer) using the learning rule δ _j ^N by the following equation.

  In Equation 2, w represents a connection weight value between the n-th layer j-th neuron and the (n −1) -th layer k-th neuron.

Next, the connection weight value learning unit 45 calculates the amount of change Δw of the connection weight value w of each neuron using the following equation using the error signal δ _j ⁿ of the intermediate layer (nth layer), and the connection weight of each neuron. The value w is updated (step D6). In the following equation, η represents a learning rate, and is set to η ₁ (0 <η ₁ ≦ 1) in learning by the evaluator.

At step D6, each of the pre-training data sample number T ₁ learning is performed, then, or smaller or not than the reference value E ₁ for pre-learning is square error E shown in the following equation predetermined There is judged (step D7), the operation of step D6 to the square error E is smaller than the reference value E ₁ is repeated. Incidentally, the learning iterations S squared error E is assumed to be smaller than the reference value E ₁ is set in advance, may be the operation of step D6 to repeat the learning iterations S times.

In the case of the square error E is determined to be smaller than the reference value E ₁ Step D7, connection weights learning unit 45, the connection weights output unit 47 the coupling weight value w for each neuron is prior learning The connection weight value w of each neuron outputted and output from the connection weight value output unit 47 is stored in the impression degree data conversion unit 14.

Next, the music map learning operation used in the mapping operation (step A6) in the music mapping unit 16 will be described in detail with reference to FIG.
FIG. 11 is a flowchart for explaining a music map learning operation in the neural network learning apparatus shown in FIG.

  A music medium such as a CD or DVD is set in the music data input unit 41, music data is input from the music data input unit 41 (step E1), and the feature data extraction unit 43 inputs the music data. The tempo is extracted as feature data from the music data input from the unit 41 (step E2), and the extracted tempo is output to the music map learning unit 46. Note that the tempo extracted by the feature data extraction unit 43 is the same as the tempo extracted by the feature data extraction unit 13 of the music registration device 10.

  Next, the audio output unit 42 outputs the music data input from the music data input unit 41 as audio (step E3), and the evaluator listens to the audio output from the audio output unit 42 to thereby determine the degree of impression of the music. Is evaluated by sensitivity, and the evaluation result is input as impression degree data from the impression degree data input unit 44 (step E4).

  The music map learning unit 46 receives the tempo input from the feature data extraction unit 43 and the impression level data input from the impression level data input unit 44 as input vectors to the self-organizing map. In the present embodiment, as the evaluation items for impression degree, three items of “bright, dark”, “clear, fuzzy”, and “violent, gentle” determined by human sensitivity are set. A seven-stage evaluation for the evaluation item is configured to be received by the impression degree data input unit 44 as impression degree data.

The music map learning unit 46 uses the tempo input from the feature data extraction unit 43 and the impression level data input from the impression level data input unit 44 as an input vector x _j (t) εR ^n, and features of each neuron. Train the vector m _i (t) εR ⁿ . Note that t represents the number of learning times, a preset value T that determines the number of learning times is set in advance, and learning is performed for the learning number t = 0, 1,. Note that R represents the evaluation stage of each evaluation item, and n represents the number of items of impression degree data.

First, as an initial value, feature vectors m _c (0) of all neurons are set at random in the range of 0 to 1, and the music map learning unit 46 determines that the neuron c, which is closest to x _j (t), That is, a winner neuron c that minimizes ‖x _j (t) −m _c (t) ‖ is obtained, and a feature vector m _c (t) of the winner neuron c and a set Nc of neighboring neurons i in the vicinity of the winner neuron c Each feature vector m _i (t) (iεNc) is updated according to the following equation (step E5). It is assumed that the neighborhood radius for determining the neighborhood neuron i is set in advance.

In Equation 5, h _ci (t) represents a learning rate and is obtained by the following equation.

Α _init is an initial value of the learning rate, and R ² (t) is a monotonically decreasing linear function or exponential function.

Next, the music map learning unit 46 determines whether or not the learning count t has reached the set value T (step E6), and performs the processing operations of steps E1 to E5 until the learning count t reaches the set value T. Repeatingly, when the learning count t reaches the set value T, the same processing operation is performed again from the first sample. When the number of iterations reaches a predetermined number of times S, the feature vector m _i (T) εR ⁿ learned through the feature vector output unit 48 is output (step E7). The output feature vector m _i (T) of each neuron i is stored as a music map in the music map storage unit 17 of the music registration device 10.

Next, a music search operation in the music registration apparatus 10 will be described in detail with reference to FIGS.
12 is a flowchart for explaining a music search operation in the music registration device shown in FIG. 1, and FIG. 13 is a diagram showing an example of a search condition input screen displayed on the PC display unit shown in FIG. 14 is a diagram showing an example of a condition search result display screen displayed on the PC display unit shown in FIG. 1, and FIG. 15 is an example of a representative song search result display screen displayed on the PC display unit shown in FIG. 16 is a diagram showing an example of a music list display screen displayed on the PC display unit shown in FIG. 1, and FIG. 17 is a music map display screen displayed on the PC display unit shown in FIG. It is a figure which shows an example.

  The music search unit 18 displays a search condition input screen 51 for inputting search conditions on the PC display unit 20, and accepts user input from the PC operation unit 19. As shown in FIG. 13, the search condition input screen 51 includes a bibliographic data input area 511 for inputting bibliographic data as a search condition, a condition search execution button 512 for instructing execution of a search, an all song list display button 513, A map display button 514 is provided, and the user inputs bibliographic data as a search condition from the PC operation unit 19 (step F1), and clicks the condition search execution button 512 to perform a search based on the bibliographic data. To instruct.

  The music search unit 18 searches the music database 15 based on the bibliographic data input from the PC operation unit 19 (step F2), and causes the PC display unit 20 to display a condition search result display screen 52 that displays the condition search results. (Step F3). Referring to FIG. 14, the conditional search result display screen 52 has a representative song selection area 521 for selecting a representative song from the conditional search results, and a representative song search execution button 522 for instructing a search based on the selected representative song. And a registration button 523 for registering a condition search result as a search list.

  Next, the user selects a representative song from the search results displayed in the representative song selection area 521 (step F4), and clicks the representative song search execution button 522 to perform a search based on the representative song. The search unit 18 is instructed. When the registration button 523 is clicked at this time, the music search unit 18 stores the condition search results displayed in the search result display area 52 in the music database 15 as a search list.

  The music search unit 18 searches the music database 15 based on the selected representative music, thereby specifying the coordinates of a neuron (hereinafter referred to as a representative neuron) in which the representative music is arranged in the music map (step F5). ).

  Next, the music search unit 18 sets 0 to the variable n (step F6), and refers to the search table shown in FIG. 9 stored in the music map storage unit 17, thereby identifying the representative neuron specified in step F5. Is searched for music data arranged in a neuron at a distance n (step F7), and it is determined whether or not the number of searched music is greater than or equal to a predetermined number of searches (step F8).

  If the number of songs searched in step F8 is less than the predetermined search number, the song search unit 18 increments the variable n (step F9), returns to step F7, and the number of searched songs is preset. Steps F7 and F8 are repeated until the number of searches exceeds a predetermined number.

  If the number of songs searched in step F8 is equal to or greater than the predetermined number of searches, the music search unit 18 narrows down the searched music data to the predetermined number of searches (step F10) and then narrows down. The representative song search result display screen 53 is displayed on the PC display unit 20 with the song data as the representative song search result (step F11). As shown in FIG. 15, the representative song search result display screen 53 includes an output song selection area 531 for selecting song data to be output, and an output button 532 for instructing output of the selected song data.

In step F10, the searched music data is narrowed down by using the representative neuron feature vector or the impression data converted by the representative music impression data converter 14 as the input vector _xj, and the impression data of the searched music data. When the search target vector X _j, the input vectors x _j closer search target vector X _j, i.e. is performed by searching to the Euclidean distance ‖X _j -x _j searches that ‖ is sequentially predetermined ones less. It should be noted that the music data searched in step F10 is limited only to music data arranged in a neuron farthest from the representative neuron, that is, a neuron separated from the representative neuron by the maximum value of the variable n used for the search. By doing so, the number of objects for which distance calculation is performed can be reduced, so that the search time (calculation time) can be shortened. Further, all of the searched music data may be displayed as the representative music search result on the representative music search result display screen 53 without narrowing down the searched music data in step F10. In this case, Since the search can be performed without calculating the distance, the search time (calculation time) can be further shortened.

  Next, when the output button 532 is clicked, the music search unit 18 stores the representative music search result displayed in the output music selection area 531 as a search list in the music database 15 (step F12).

  When the all song list display button 513 is clicked on the search condition input screen 51, the song search unit 18 displays a song list display screen 54 that displays a song list of all stored song data on the PC display unit 20. Display. Referring to FIG. 16, the song list display screen 54 is a representative song selection area 541 for selecting a representative song from the song list of all song data, and a representative song search execution button for instructing a search based on the selected representative song. 542.

  When the user selects a representative song from the song list displayed in the representative song selection area 541 (step F4) and clicks the representative song search execution button 542, a search based on the selected representative song is performed. Instructed by the unit 18, the music search operation in steps F5 to F12 is performed.

  Further, when the music map display button 514 is clicked on the search condition input screen 51, the music search unit 18 causes the PC display unit 20 to display a music map display screen 55 that displays neurons of the music map as points. Referring to FIG. 17, the music map display screen 55 includes a neuron selection area 551 for selecting a neuron as a search center and a map search execution button 552 for instructing a search based on the selected neuron.

  When the user selects a neuron as a search center from the neurons displayed in the neuron selection area 551 and clicks the map search execution button 552, the neuron is selected instead of the representative neuron specified in step F5. A search based on the neuron selected in the region 551 is instructed to the music search unit 18, and the music search operation in steps F 6 to F 12 is performed thereafter.

  In the neuron selection area 551, only the neuron where the music data is arranged is displayed, and when the cursor is moved to the neuron to which the music is mapped, the music data of the music data arranged in the neuron where the cursor is located A name pop-up display 553 is performed.

Next, the music data transfer operation from the music registration device 10 to the mobile terminal device 30 will be described in detail.
FIG. 18 is an explanatory diagram for explaining an operation of transferring music data from the music registration device shown in FIG. 1 to the mobile terminal device.

  In the music database 15 of the music registration device 10, as shown in FIG. 18, music data, feature data corresponding to the music data (extracted by the feature data extraction unit 13), bibliographic data, impression data (impression data) Conversion unit 14), neuron coordinates in the music map (mapped by music mapping unit 16), search table (created by music mapping unit 16), and search list (searched by music search unit 18) are stored. The music data, the bibliographic data corresponding to the music data, the impression data, and the neuron coordinates in the music map, the search table, and the search list are transferred to the terminal music database 32 of the mobile terminal device 30. .

  When the portable terminal device 30 is connected to the transmission / reception unit 21 of the music registration device 10 and the transmission / reception unit 31 of the portable terminal device 30 via the data transmission path 23, the stored contents of the music database 15 of the music registration device 10 and the portable The content stored in the terminal music database 32 of the terminal device 30 is compared, and the content stored in the music database 15 of the music registration device 10 is reflected in the terminal music database 32 of the mobile terminal device 30.

Next, the music search operation in the mobile terminal device 30 will be described in detail with reference to FIG.
FIG. 19 is a diagram illustrating a display example at the time of music search of the terminal display unit of the mobile terminal device illustrated in FIG. 2.

  When the mobile terminal device 30 is turned on by a power key (not shown), the music search unit 33 searches for music data to be reproduced as “search by bibliographic data”, “search by impression data”, “search list”. A search method selection screen 351 for selecting any one of “search by” is displayed on the terminal display unit 35, and the user selects the search method displayed on the search method selection screen 351 with the up and down of the cross key 341, Pressing the enter key 342 instructs the music search unit 33 to search for music data based on the selected search method.

  When “search by bibliographic data” is selected on the search method selection screen 351, the music search unit 33 selects a list display for selecting one of “artist”, “album”, and “title”. The screen 352 is displayed on the terminal display unit 35, and the user selects one of “artist”, “album”, and “title” displayed on the list display selection screen 352 with the up and down keys of the cross key 341, and presses the enter key. Pressing 342 instructs the music search unit 33 to display the selected list.

  When “artist” or “album” is selected on the list display selection screen 352, the music search unit 33 searches for the artist name or album name from the bibliographic data corresponding to all music data stored in the terminal music database 32. And an artist / album list screen 353 for displaying a list of artist names or album names is displayed on the terminal display unit 35. The user selects either the artist name or the album name displayed on the artist / album list screen 353 with the up and down keys of the cross key 341 and presses the enter key 342 to search for the artist name or album name. The search unit 33 is instructed, and the music search unit 33 searches the terminal music database 32 based on the selected artist name or album name, and displays a title list of the corresponding music data on the terminal display unit 35 as a search result screen 354. indicate. When “title” is selected on the list display selection screen 352, the music search unit 33 reads the titles corresponding to all the music data stored in the terminal music database 32 and searches the read title list. The result screen 354 is displayed on the terminal display unit 35.

When “Search by impression degree data” is selected on the search method selection screen 351, the music search unit 33 displays an impression degree data input screen 355 for inputting impression degree data as a search condition on the terminal display unit 35. The user selects the value of each item of the impression degree data with the up / down / left / right directions of the cross key 341 and presses the enter key 342 to instruct the music search unit 33 to perform a search based on the impression degree data. When the search based on the impression degree data is instructed, the music search unit 33 searches the music data stored in the terminal music database 32 based on the impression degree data input as the search condition, and the searched music data The title list is displayed on the terminal display unit 35 as a search result screen 354. Music retrieval data based on the input impression data as a search condition, with the input vector x _j the input impression data as a search condition, corresponding to all music data stored in the terminal song database 32 Each impression degree data is set as a search target vector m _i (t) εR ^n, and a predetermined number is searched from the one having a smaller Euclidean distance ‖x _j −m _i ‖. Note that R indicates the number of evaluation stages of each evaluation item of impression degree data, and n indicates the number of items of impression degree data.

  If “search by search list” is selected on the search method selection screen 351, the music search unit 33 uses the search list list 356 as a search list list stored in the terminal music database 32, and the terminal display unit 35. The user selects one of the search lists displayed on the search list list screen 356 with the up and down keys of the cross key 341 and presses the enter key 342 to display the display of the selected search list. The music search unit 33 displays the title list of the selected search list on the terminal display unit 35 as a search result screen 354.

  When the right side of the cross key 341 is pressed while the search result screen 354 is displayed, the music search unit 33 sends music data corresponding to the title list displayed on the search result screen 354 from the terminal music database 32. Sequentially read out, and the read music data is output to the audio output unit 22 for audio output. When the search result screen 354 is displayed and the left of the cross key 341 is pressed after selecting one of the titles with the up and down keys of the cross key 341, the music search unit 33 selects the selected title. Only the music data corresponding to is read from the terminal music database 32, and the read music data is output to the audio output unit 22 for audio output.

  Further, when the search result screen 354 is displayed and one of the titles is selected with the up and down keys of the cross key 341 and then the enter key 342 is pressed, the music search unit 33 selects the selected title. A representative song search as a representative song is performed, and the representative song search result is displayed on the terminal display unit 35 as a search result screen 354. The representative music search performed by the music search unit 33 is performed based on the neuron coordinates and the search table stored in the terminal music database 32, and the representative music search performed by the music search unit 18 of the music registration apparatus 10 described above. The procedure is the same as the song search.

Next, a display method during music reproduction in the mobile terminal device 30 will be described in detail with reference to FIGS. 20 and 21. FIG.
20 is a diagram illustrating a display example during music reproduction of the terminal display unit of the mobile terminal device illustrated in FIG. 2, and FIG. 21 is a diagram for explaining a light emission color determination method in the light emission color determination unit illustrated in FIG. 1. It is explanatory drawing.

  At the time of music reproduction, the light emission color determination unit 38 extracts the most characteristic evaluation items from the impression data in the music data being reproduced, that is, the music data output by the audio output unit 22, and the extracted evaluation is performed. The light emitting unit 37 is caused to emit light with the light emission color assigned in advance to the item, and the display surface of the terminal display unit 35 is colored as shown in FIG.

  Each piece of music data is evaluated as 1 to 7 for each of the three evaluation items of “bright, dark”, “clear, dark”, and “violent, gentle” as impression data, and the minimum value (1 ) Or the closer to the maximum value (7), the more characteristic evaluation items. For example, in the case of the evaluation item “bright, dark”, music data that gives a “bright” impression as it approaches the minimum value (1), and music data that gives a “dark” impression as it approaches the maximum value (7). Accordingly, the luminescent color determination unit 38 determines which item is one of the three items, “bright, dark”, “clear, dull”, and “violent, gentle”, which are evaluation items of impression degree data in the music data being played back. The most characteristic evaluation item is extracted by judging whether it is closest to the minimum value (1) or the maximum value (7).

  As shown in FIG. 21, the emission colors pre-assigned to each evaluation item are yellow and gray for the evaluation item “bright and dark”, and sky blue and blue for the evaluation item “clear and dark”. Brown is assigned red and beige to the evaluation item “Intense and Gentle”, respectively.

  When the evaluation item “bright, dark” is extracted as the most characteristic evaluation item because it is close to the minimum value (1), the light emission color determination unit 38 causes the light emission unit 37 to emit light with the yellow light emission color. When the evaluation item “bright, dark” is extracted as the most characteristic evaluation item by being close to the value (7), the light emitting unit 37 is caused to emit light in a gray emission color. In addition, when the evaluation item “clear” is extracted as the most characteristic evaluation item because the emission color determination unit 38 is close to the minimum value (1), the light emission unit 37 has the sky blue emission color. When the evaluation item “clear” is extracted as the most characteristic evaluation item by being close to the maximum value (7), the light emitting unit 37 is caused to emit light in the brown emission color. Further, when the evaluation item “violent and gentle” is extracted as the most characteristic evaluation item because the emission color determination unit 38 is close to the minimum value (1), the light emission unit 37 emits light with the red emission color. When the evaluation item “violent and gentle” is extracted as the most characteristic evaluation item by being close to the maximum value (7), the light emitting unit 37 is caused to emit light in a beige emission color.

  In addition to playback that specifies only one song, that is, when the title list displayed on the search result screen 354 is played back sequentially or randomly, or all the song data recorded in the terminal song database 32 is played back sequentially or randomly. In this case, the music search unit 33 displays the bibliographic data of the music data being reproduced on the terminal display unit 35, and the next music data to be reproduced next time, that is, after the reproduction of the music data being reproduced is completed. Bibliographic data of music data to be reproduced is read from the terminal music database 32, and the read bibliographic data is displayed on the terminal display unit 35 as next music information as shown in FIG.

  In the state where the next song information is displayed on the terminal display unit 35, when the left side of the cross key 341 is pressed, the song search unit 33 skips the song data to be reproduced next time and creates new song data. (Sequential playback for sequential playback, and random selection for random playback) to select new next music data of the selected music data, and the bibliographic data of the music data from the terminal music database 32 The read bibliographic data is displayed on the terminal display unit 35 as next music information.

  As described above, according to the present embodiment, the impression data representing the impression of the music data is recorded in the terminal music database 32 together with the music data, and the emission color determining unit 38 is recorded by the audio output unit 36. By determining the light emission color based on the impression degree data of the music data that is output as audio, and configuring the light emitting unit 37 that emits light with a plurality of light emission colors to emit light with the light emission color determined by the light emission color determining unit 38 Since the impression degree data representing the impression of the music data being played can be easily grasped by the emission color, it is possible to easily recognize how the music data being played is evaluated Play.

  In the present embodiment, the light emitting unit 37 is configured to emit light with the light emission color determined by the light emission color determining unit 38. However, when the terminal display unit 35 is a color display, the terminal display unit 35 emits light. The background screen or the like may be displayed in the emission color determined by the emission color determination unit 38 as the unit 37.

  Note that the present invention is not limited to the above-described embodiments, and it is obvious that the embodiments can be appropriately changed within the scope of the technical idea of the present invention. In addition, the number, position, shape, and the like of the constituent members are not limited to the above-described embodiment, and can be set to a suitable number, position, shape, and the like in practicing the present invention. In each figure, the same numerals are given to the same component.

It is a block diagram which shows the structure of embodiment of the music search system which concerns on this invention. It is a front view which shows the structure of the portable terminal device shown in FIG. It is a block diagram which shows the structure of the neural network learning apparatus which learns in advance the neural network used for the music registration apparatus shown in FIG. It is a flowchart for demonstrating the music registration operation | movement in the music registration apparatus shown in FIG. It is a flowchart for demonstrating the feature data extraction operation | movement in the feature data extraction part shown in FIG. It is a flowchart for demonstrating the tempo determination operation | movement in the feature data extraction part shown in FIG. It is explanatory drawing which shows the hierarchical neural network example used by the impression degree data conversion part shown in FIG. It is explanatory drawing which shows the example of a music map which a music mapping part shown in FIG. 1 maps music data. It is a figure which shows the example of a search table memorize | stored in the music database shown in FIG. 3 is a flowchart for explaining a learning operation of a hierarchical neural network in the neural network learning apparatus shown in FIG. 3 is a flowchart for explaining a music map learning operation in the neural network learning apparatus shown in FIG. 2. It is a flowchart for demonstrating the music search operation | movement in the music registration apparatus shown in FIG. It is a figure which shows the example of a search condition input screen displayed on the PC display part shown in FIG. It is a figure which shows the example of a condition search result display screen displayed on the PC display part shown in FIG. It is a figure which shows the example of a representative music search result display screen displayed on the PC display part shown in FIG. It is a figure which shows the example of a music list display screen displayed on the PC display part shown in FIG. It is a figure which shows the example of a music map display screen displayed on the PC display part shown in FIG. It is explanatory drawing for demonstrating the transfer operation | movement of the music data from the music registration apparatus shown in FIG. 1 to a portable terminal device. It is a figure which shows the example of a display at the time of the music search of the terminal display part of the portable terminal device shown in FIG. It is a figure which shows the example of a display at the time of the music reproduction of the terminal display part of the portable terminal device shown in FIG. It is explanatory drawing for demonstrating the luminescent color determination method in the luminescent color determination part shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Music registration apparatus 11 Music data input part 12 Compression processing part 13 Feature data extraction part 14 Impression degree data conversion part 15 Music database 16 Music mapping part 17 Music map memory | storage part 18 Music search part 19 PC operation part 20 PC display part 21 Transmission / reception Unit 22 audio output unit 23 data transmission path 24 speaker 25 headphones 30 portable terminal device 31 transmission / reception unit 32 terminal music database 33 music search unit 34 terminal operation unit 35 terminal display unit 36 audio output unit 37 light emission unit 38 light emission color determination unit 40 neural Network learning device 41 Music data input unit 42 Voice output unit 43 Feature data extraction unit 44 Impression degree data input unit 45 Bond weight value learning unit 46 Music map learning unit 47 Bond weight value output unit 48 Feature vector output unit 51 Search condition input screen 52 conditions Search result display screen 53 Representative song search result display screen 54 Music list display screen 55 Music map display screen 341 Four-way key 342 Enter key 351 Search method selection screen 352 List display selection screen 353 Artist / album list screen 354 Search result screen 355 Impression degree Data input screen 356 Search list list screen 511 Bibliographic data input area 512 Conditional search execution button 513 All song list display button 514 Song map display button 521 Representative song selection area 522 Representative song search execution button 523 Registration button 531 Output song selection area 532 Output button 541 Representative song selection area 542 Representative song search execution button 551 Neuron selection area 552 Map search execution button 553 Pop-up display

Claims (8)

A portable terminal device that records music data and reproduces the recorded music data,
Along with the music data, a terminal music database that records impression degree data representing the impression of the music data,
Audio output means for reading out and outputting the music data recorded in the terminal music database;
A light emission color determining means for determining a light emission color based on the impression degree data of the music data output by the sound output means;
A light emitting means for emitting light by switching a plurality of emission colors,
A portable terminal device that causes the light emitting means to emit light in a light emission color determined by the light emission color determining means. The impression data recorded in the terminal music database consists of a plurality of evaluation items,
The luminescent color determining means extracts the evaluation item that is most characteristic in the impression degree data, and causes the luminescent means to emit light with a luminescent color pre-assigned to the extracted evaluation item. Item 2. The mobile terminal device according to Item 1.   The luminescent color determining means extracts the most characteristic evaluation item by determining which of the plurality of evaluation items of the impression degree data is closest to a minimum value or a maximum value. Item 3. A portable terminal device according to Item 2. Comprising terminal display means for displaying the bibliographic data of the music data that has been voice output by the voice output means,
The light emitting means is disposed at an end of the terminal display means, and emits light from a lateral direction with respect to the display surface of the terminal display means to cause the display surface of the terminal display means to color. The mobile terminal device according to any one of 1 to 3. In a portable terminal device that records music data and reproduces the recorded music data, a music expression method for expressing an image of the music data being reproduced,
The impression data representing the impression of the music data is recorded together with the music data in the terminal music database,
Read and output the music data recorded in the terminal music database,
A light emission color is determined based on the impression degree data of the music data that is output as audio,
A music expression method characterized by causing a light emitting means for emitting light by switching a plurality of light emission colors to emit light in the determined light emission color. The impression data consists of a plurality of evaluation items,
6. The music expression method according to claim 5, wherein the most characteristic evaluation item is extracted from the impression degree data, and the light emitting means is caused to emit light with a light emission color pre-assigned to the extracted evaluation item. .   The music expression method according to claim 6, wherein the most characteristic evaluation item is extracted by determining which of the plurality of evaluation items of the impression degree data is closest to a minimum value or a maximum value. .   The music expression according to any one of claims 5 to 7, wherein a display surface display surface of a terminal display means for displaying the bibliographic data of the music data output as audio is colored with the determined emission color. Method.

Images