JP2636685B2 - Music event index creation device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for extracting a time correspondence between a performance recorded on a medium such as a CD or a DAT and a corresponding musical score.

[0002]

2. Description of the Related Art In recent years, as recording devices have increased in density and capacity, the use of multimedia in computers has increased, and in addition to text information, opportunities for handling sounds, moving images, and the like by computers have increased. When these media are used in an integrated manner, synchronization is often made between time-progress media, for example, music and moving images.

[0003] Among them, especially music has a musical score as an expression since ancient times, and it is widely used even in modern times as a method of describing music information. Another representative expression method is the recording of an audio signal of a performance, but there is a difference in time and time between a musical score and a performance.
Synchronization between a musical score and audio media requires a correspondence between a musical event described in the musical score and a position in the performance, but currently, in many cases, the corresponding time is measured manually. . For this reason, there is a serious problem in creating music package software that effectively uses multimedia, especially software that combines music and media such as music.

As a method for determining the correspondence between a musical event on a musical score and a performance, Takami et al. (Extraction of performance information in piano performance, IEICE Transactions D-II Vo)
l. J72-D-II No. 6,917 to 926
Page 1). In Literature 1, the object is limited to the piano, and the musical note expression on the score and the fluctuation in the performance are extracted. As a secondary result, the correspondence between the note on the musical score and the position during the performance is obtained. Have been. However, in the method used in Reference 1, the musical instrument is limited to the piano, and extra calculation time is required to find the rise and fall of the sound.

The limitation of the target musical instrument to a single musical instrument is a condition for making it easy to identify individual musical intervals when a chord is played. However, since there are many performances by a plurality of types of musical instruments such as orchestras, the limitation of musical instruments lacks practicality. In addition, as for multi-pitch recognition that does not depend on musical instruments, only low-accuracy ones conventionally obtained on the premise that performance contents are not given in advance have been used, and the amount of calculation has been large.

[0006]

SUMMARY OF THE INVENTION It is an object of the present invention to provide an apparatus for expanding the range of a target musical instrument to other than a piano and for extracting correspondence between a musical score and a performance with a small amount of calculation.

[0007]

According to the present invention, there is provided a music event index creating apparatus which records music on a recording medium on which music is recorded.
And music that can be expressed as notes in audio data
During the performance of the performance as the corresponding positions between the notes above
Detect the correspondence between the time and the index corresponding to the time on the score
An apparatus comprising: audio data input means for inputting the audio data of the music, and the segment window for selecting and outputting a time-part of the data obtained from the audio data input means, outputs of the segment window A spectrum creation unit for creating a frequency spectrum from data, a score data input unit for a user to input a score, a score window unit for selectively outputting a part of the data from the data provided to the score data input unit in time, A spectrum synthesizer for synthesizing a frequency spectrum corresponding to the partial score output by the score window; and a frequency spectrum output by the spectrum generator and a frequency spectrum output by the spectrum synthesizer. Only the degree, or if the degree is high, the center time of the segment window and the A matching unit for outputting first time information of the musical score window for outputting the Fumado unit, wherein the matching unit
The segment window according to the degree of match output by
And move at least one of the musical score windows
A matching control unit and the matching unit outputs
The time from the segment window and the time from the score window
A record that is recorded as local corresponding time information of performances and scores.
And a channel recording unit .

[0008]

The basic principle of the present invention is to compare the frequency spectrum obtained from the audio recording with the frequency spectrum obtained from the musical score to avoid the pitch identification problem which has been a problem in the past, and to match them. Further, by not performing pitch identification, all calculations conventionally required for pitch identification can be omitted, and as a result, the calculation time is shortened.

[0009]

FIG. 1 is a block diagram showing an embodiment of a music index creating apparatus according to the present invention.

The user provides the A / D data input means 11 with audio data obtained by digitizing the performance of the music, and the musical score data generating means 15 with the corresponding musical score information.
As audio data to be given, for example, music C
There are recorded data such as D and data recorded in DAT. The musical score data to be given may be an electronic representation of a musical score including symbols such as repetition, MIDI data, or the like.

The segment window section 12 extracts and outputs one section of the audio data provided to the A / D data input means 11.

The spectrum creating section 13 creates a power spectrum for each musical scale from the data output from the segment window section 12. To create a power spectrum,
There are known methods such as Fourier transform and autocorrelation coefficient.

A musical score data input unit 16 converts musical expression symbols such as repetitions into musical expressions, scales, strengths, timings, types of musical instruments, etc., which are expanded in the time direction from the musical score information given to the musical score data generating means 15. .

The score window 17 takes out one section in time from the expression converted by the score data input section 16 and outputs it.

In the spectrum synthesizing section 110, the musical score window section 17
Using the musical score data output from, and overtone spectral patterns prepared in advance for each musical instrument and range, a predicted spectral pattern for the corresponding section is synthesized and output. Further, it is also possible to use a spectrum pattern corresponding to a change in the harmonic composition with time transition as the harmonic spectrum pattern.

The matching section 18 compares the spectrums output from the spectrum creating section 13 and the spectrum synthesizing section 110, and if the difference between the two is smaller than a predetermined value, the matching is sufficiently performed. Otherwise, the matching is insufficient. Is determined to be in a good state. When the matching is sufficient, a time corresponding to the center of the segment window provided in the segment window section 12 is output.

The matching result recording unit 19 records the time output by the matching unit 18 in a memory or an external storage device in accordance with the corresponding note in the musical score data input unit.

The matching control unit 14 includes a matching unit 18
If the matching is insufficient, the segment window is moved in the increasing direction of the time axis and the matching is tried again. If the matching is sufficient, the score window and the segment window are moved.

The matching section 18 is realized, for example, by the configurations shown in FIGS. 2 (a), 2 (b) and 2 (c).

In (a), the spectrum creating section 13
The distance when the spectrum obtained from the spectrum synthesizing unit 110 is regarded as a vector, for example, the Euclidean distance, is calculated by the distance comparing unit 21, and matching is performed based on the magnitude of the numerical value when compared with a prepared threshold value.

In (b), the spectrum creating unit 13
A performance spectrum buffer 22 and a musical score spectrum buffer 23 are prepared as a ring buffer for buffering the output of each of the spectrum synthesizing units 110 for a certain period of time, and the DP matching is performed by the DP matching unit 24 using both of them. I take the. The DP matching is described in detail in, for example, the document “Basics of Speech Information Processing”, pages 187 to 190 (published by Ohmsha). DP matching requires a distance function between a standard pattern and a vector to be compared. For example, a value obtained by subtracting the absolute value of the inner product of the vectors obtained by normalizing the two from the unit vector size is used for matching. Used as an evaluation function for

In (c), a performance spectrum buffer 25 similar to the performance spectrum buffer 22, and
A musical score spectrum buffer 27 similar to the musical score spectrum buffer 23 exists. The note density calculation unit 26 controls the segment window unit 12 according to the density of the musical score per unit time.
Determine the moving length of the segment window of the spectrum determined by
The frequency of spectrum calculation is controlled through the matching control unit 14. Finally, DP matching is performed in the DP matching unit 28 using both the buffer units connected to the spectrum creating unit 13 and the spectrum synthesizing unit 110 as in (b). Is used for matching.

Next, the flow of processing during operation will be described with reference to FIG.
The case where the digital input corresponding to the waveform shown in FIG. 4 is matched with the musical score input shown in FIG. Here, in the embodiment using each of the above (a), (b), and (c), the matching control section that controls the operation timing has an interactive control relationship with the matching section. Therefore, even though the operation of each part is the same, the overall timing may be different.
The overall operation timing for each of the cases (b) and (c) will be described.

First, the case where the matching unit 18 has the configuration shown in FIG. 2A will be described.

1. At the start of the operation, the user provides the A / D data input means 11 with data obtained by quantizing the audio signal of the performance in chronological order. Further, the user gives the musical score data generating means 15 musical score information to be matched with the data. The given musical score information converts the musical score data supplied to the musical score data input unit 16 into a format suitable for processing in the present apparatus. The format suitable for processing is:
Among the information of the given music score, at least the information of the pitch,
It is a format that contains absolute and relative time and time information to be pronounced. Other information, such as the type of musical instrument, is discarded and converted as appropriate. One of the specific formats is, for example, a standard MIDI file format. FIG. 5 shows an example in which FIG. 4 is converted into a standard MIDI file format. However, FIG. 5 shows the standard MID
I file format type 0 or type 1 M
Only the Trk chunk is shown. Here, when specifying the type of instrument, FF0 is added to the head of the MTrk chunk.
A musical instrument name can be described as a meta event starting from 4. For details of the standard MIDI file format, see Ritto Music Co., Ltd., “Keyboard Magazine”, Vol.
I File Handbook ".

2. After the A / D data input unit 11 and the score data generating unit 15 have the data, the matching unit 18 requests the matching control unit 14 to output the first segment of the A / D data and the first segment of the score data. Issues a sound output request.

3. The matching control unit 14 receives the received requests from the segment window unit 12 and the score window unit 17 respectively.
, And waits for a matching result output by the matching unit 18.

4. The segment window unit 12 outputs the first segment of the data obtained from the A / D data input unit 11 to the spectrum creation unit 13 based on the request.
The size of the segment at this time is specified in the system in advance, or the user directly or indirectly specifies the size of the device. Spectrum creator 1
In 3, the power spectrum of the segment received from the segment window unit 12 is calculated, and the matching unit 1
8 is output.

5. On the other hand, the musical score window unit 17 outputs at least pitch data of the first sound to the spectrum synthesizing unit 110 based on the received request. At this time, information other than the original musical score information or the pitch identified by the musical score data input unit is output to the spectrum synthesizing unit 110 as appropriate. In addition, information that is generally treated as a chord, such as performances by a plurality of parts, is transmitted to the spectrum synthesizing unit 110 for all sounds that are sounding at the same time. The spectrum synthesizing unit 110 synthesizes an expected power spectrum using the pitch information obtained from the musical score window unit 17 and the additional information, and outputs the synthesized power spectrum to the matching unit 18. At this time, if the additional information passed from the score window section 17 specifies the type or tone of the musical instrument, an expected spectrum pattern is created using the overtone spectrum pattern accordingly. At this time, the spectrum synthesizing unit 110 records the spectrum pattern used as the internal state and the time of the pronunciation on the musical score. This timing is used later when synthesizing a new spectrum. Note that, in the present invention,
Includes absolute or relative time and time information, as
Use the format internally as score data. Specific examples and
The standard MIDI file format
These times and times are commonly used as "hours (or hours)
Interval / minute / second).
Specific “physical”
By setting the "time", you can use the "time"
(Or hours) / minutes / seconds ”
Using only units called ticks
Sometimes represented. In the present invention, the position on the score is the same.
Time information is used to determine
Time, daily time / time, or tick.
The spectrum synthesizer 110 converts the received pitch data
The time represented by such a tick is used as the time.
Before the sound of the
It is recorded together with the aforementioned spectral pattern. This recorded
Time is not matched if the result described later matches.
To know the pitch data processed so far
Is used as the time for

6. The matching unit 18 receives the power spectrum and the expected power spectrum from the spectrum creating unit 13 and the spectrum synthesizing unit 110, respectively, and matches them. If this does not match,
The matching unit 18 returns a matching result to the matching control unit 14 and has new data from the spectrum creating unit 110. The operation when the result matches will be described later in Section 8 below.

7. If the matching result does not match, the matching control unit 14
Request to move the segment window. In the segment window unit 12, a new segment is output to the spectrum creation unit 13 by moving the time segment in the increasing direction of the time axis by a predetermined movement amount. The spectrum creating unit 13 calculates a power spectrum in the same manner as in the immediately preceding case, finds a new power spectrum, and outputs the new power spectrum to the matching unit 18. In the matching 18, matching between the newly obtained power spectrum and the expected spectrum is performed. If the result is different, the matching result is returned to the matching control unit 14 and new data is received from the spectrum creating unit 110, and 7 is repeated. Eventually, the movement of the segment window and the spectrum creation are repeated until the matching result indicates a match. If there is no performance data before a matching segment is obtained from the performance data, the operation is terminated.

[0032] 8. If the result matches, the matching recording unit 19 sets the time at the beginning of the segment measured from the beginning of the performance data and the time of the pitch data handled by the score window unit 17 as the time measured from the beginning of the original score information. Output to After the output is completed, the matching unit 18 performs matching.
The result of the match is returned to the control unit 14, and the top of the score window is
Move to notes, new power spectrum and expected power
-Request a spectrum and the operation repeats from 3
You. By moving the score window in this way, the score window
Always indicates the note currently being matched.
And the above-mentioned “pitch data handled by the score window section 17”.
The “time of data” is the time information at the top of the score window.

Here, when the data ends and it becomes impossible to move the score window or the segment window, the operation is ended. By repeating the above operation until the end of the data, pairs of the time in the performance data and the position on the musical score are successively obtained in the matching recording unit 19.

Next, the case where the matching unit 18 has the configuration shown in FIG. 2B will be described.

1. At the start of the operation, the user provides the A / D data input means 11 with quantized data of the audio signal of the performance in the order of time, as in the case of (a). Further, the user gives the musical score data generating means 15 musical score information to be matched with the data. The given musical score information converts the musical score data supplied to the musical score data input unit 16 into a format suitable for processing by the present apparatus and similar to that of the case (a).

2. After the A / D data input unit 11 and the score data generating unit 15 have the data, the matching unit 18 requests the matching control unit 14 to output a few segments from the beginning of the A / D data, Issues a sound output request for a few minutes from the beginning of the data. It is assumed that the number of segments and the number of sounds are set in the system in advance, or that the user inputs together with music data at the start of the operation.

3. The matching control unit 14 receives the received requests from the segment window unit 12 and the score window unit 17 respectively.
, And waits for a matching result output by the matching unit 18.

4. The segment window unit 12 sequentially outputs the segments obtained from the A / D data input unit 11 to the spectrum creating unit 13 in the required number. At this time, the size of the segment is set in the system in advance or the user directly or indirectly sets the size in the device. Spectrum creator 13
Then, the power spectrum is calculated for each segment received from the segment window unit 12 and output to the matching unit 18 in order.

5. On the other hand, the musical score window unit 17 outputs at least the data of the musical interval to the spectrum synthesizing unit 110 based on the received request. At this time, information other than the original musical score information or the pitch identified by the musical score data input unit is output to the spectrum synthesizing unit 110 as appropriate. In addition, information that is generally treated as a chord, such as performances by a plurality of parts, is transmitted to the spectrum synthesizing unit 110 for all sounds that are sounding at the same time. The spectrum synthesizing unit 110 synthesizes an expected power spectrum by using the pitch information and the additional information for each sound obtained from the score window unit 17, and sequentially outputs the synthesized power spectrum to the matching unit 18. At this time, if the additional information passed from the music score window 17 specifies the type or tone of the musical instrument, an expected spectrum pattern is created according to the case of (a). Also, at this time, the spectrum synthesis unit 110
The spectrum pattern used as the internal state and the time of its pronunciation on the musical score are recorded. After this timing,
It is used when synthesizing a spectrum at a new different time timing, and is mainly used to realize power attenuation over time.

6. The matching unit 18 receives the power spectrum sequence and the expected power spectrum sequence from the spectrum creating unit 13 and the spectrum synthesizing unit 110, respectively, and performs DP matching between the two. At this time, the matching operation is not started until the predetermined number of segments and the number of sounds are obtained. As a result of the DP matching, when the length of each spectrum sequence is insufficient such that two or more tails of the power spectrum sequence are concentrated at one point of the expected power spectrum, the matching unit 18 A matching result is returned, at least one of the power spectrum and the expected power spectrum is requested to be added, and the arrival of the requested data is waited. The operation in other cases will be described later in Section 8.

7. If the matching control unit 14 finds that the length of each spectrum string is insufficient, the matching control unit 14 moves the segment window unit 12 and the score window unit 17 based on the request from the matching unit. And a request to move the score window. In the segment window unit 12, a new segment is output to the spectrum creation unit 13 by moving the time segment in the increasing direction of the time axis by a predetermined movement amount. The spectrum creating unit 13 calculates a power spectrum in the same manner as in the immediately preceding case, finds a new power spectrum, and outputs the new power spectrum to the matching unit 18. This series of segment window movement and power spectrum calculation is repeated for the requested number of segments for the requested number. The matching unit 18 adds the newly obtained power spectrum and the newly obtained expected power spectrum to the respective spectrum strings received so far, and performs DP matching again. As a result, if the length of the spectrum sequence is still insufficient, the matching result is returned to the matching control unit 14, and 7 is repeated with new additional data. Eventually, the movement of the segment window and the spectrum creation are repeated until the matching result indicates a match. If there is no performance data before a matching segment is obtained from the performance data, the operation is terminated.

[0042] 8. If the results match, the matched pair is handled by the score window 17 and the time at the beginning of the segment measured from the beginning of the performance data.
Time of the pitch data, ie, the time information at the top of the score window
Is output to the matching recording unit 19 as the time measured from the head of the original score information. After output is completed, matching unit 1
8 removes the output pair from the power spectrum sequence and the expected power spectrum sequence, respectively, and thereafter returns the result of the match to the matching control unit 14 until the power spectrum sequence becomes longer than a predetermined length. Request power spectrum and expected power spectrum and repeat from 3.

Here, when the data ends and it becomes impossible to move the score window or the segment window, the operation is ended. By repeating the above operation until the end of the data, pairs of the time in the performance data and the position on the musical score are successively obtained in the matching recording unit 19.

Next, a case where the matching unit 18 has the configuration shown in FIG. 2C will be described.

1. At the start of the operation, as in the cases (a) and (b), the user provides the A / D data input means 11 with data obtained by quantizing the audio signal of the performance in chronological order. Further, the user gives the musical score data generating means 15 musical score information to be matched with the data. The given musical score information converts the musical score data supplied to the musical score data input unit 16 into a format suitable for processing by the present apparatus and similar to that of the case (a).

2. After the A / D data input unit 11 and the score data generating unit 15 have the data, the matching unit 18 requests the matching control unit 14 to output a few segments from the beginning of the A / D data, Issues a sound output request for a few minutes from the beginning of the data. It is assumed that the number of segments and the number of sounds are set in the system in advance or are input by the user at the start of operation.

3. The matching control unit 14 receives the received requests from the segment window unit 12 and the score window unit 17 respectively.
, And waits for a matching result output by the matching unit 18.

4. The segment window unit 12 sequentially outputs the segments of the data obtained from the A / D data input unit 11 to the spectrum creation unit 13 based on the request. The size of the segment at this time is specified in the system in advance, or the user instructs the apparatus directly or indirectly. Spectrum creator 13
Then, the power spectrum is calculated for each segment received from the segment window unit 12 and output to the matching unit 18 in order.

5. On the other hand, the musical score window unit 17 outputs at least the data of the musical interval to the spectrum synthesizing unit 110 based on the received request. At this time, information other than the original musical score information or the pitch identified by the musical score data input unit is output to the spectrum synthesizing unit 110 as appropriate. In addition, information that is generally treated as a chord, such as performances by a plurality of parts, is transmitted to the spectrum synthesizing unit 110 for all sounds that are sounding at the same time. The spectrum synthesizing unit 110 synthesizes an expected power spectrum by using the pitch information and the additional information for each sound obtained from the score window unit 17, and sequentially outputs the synthesized power spectrum to the matching unit 18. At this time, if the additional information passed from the score window section 17 specifies the type or tone of the musical instrument, an expected spectrum pattern is created according to the cases (a) and (b). At this time, the spectrum synthesis unit 1
10 is a spectrum pattern used as an internal state,
Record the time of the pronunciation on the musical score. This timing is thereafter used when synthesizing a spectrum at a new different time timing, and is mainly used to realize power attenuation over time.

6. The matching unit 18 receives the power spectrum sequence and the expected power spectrum sequence from the spectrum creating unit 13 and the spectrum synthesizing unit 110, respectively, and performs DP matching between the two. At this time, the matching operation is not started until the predetermined number of segments and the number of sounds are obtained. As a result of the DP matching, if one point in the power spectrum sequence corresponds to multiple points in the expected power spectrum sequence, shorten the frame period before and after the point in the corresponding power spectrum sequence and perform DP matching again. The note density calculator 26 calculates the number of points in the corresponding expected power spectrum sequence, and calculates a new frame period based on the number. For example, power spectrum C
(T) and expected power spectra P (t), P (t +
1),..., P (t + n−1), t, and n, if natural numbers correspond to each other, use a frame period obtained by dividing n between P (t−1) and P (t + 1). afterwards,
The matching unit 18 returns a matching result to the matching control unit 14, requests data with a shortened frame period, and waits for its arrival. In particular, similarly to the case of (b),
If many-to-one correspondence is found at the end of either power spectrum sequence, at least one or more data additions are requested.

The operation in other cases will be described later in Section 8.

7. If the result of the matching is a many-to-one correspondence and the length of each spectrum row is insufficient, the matching control unit 14 sends the segment window unit 12 and the score window unit 17 based on the request from the matching unit. , A request for moving the segment window and a request for moving the score window, respectively. In the segment window unit 12, a segment whose frame period is shortened by moving in the direction of increasing the time axis at the position set from the matching unit 18 through the matching control unit 14 by the movement amount designated in the same manner, Output to Further, the same processing is performed for additional segments. The spectrum creating unit 13 calculates a power spectrum in the same manner as in the immediately preceding case, finds a new power spectrum, and outputs the new power spectrum to the matching unit 18. The series of segment window movement and power spectrum calculation are repeated for the number of segments requested. On the other hand, for the score window, the movement of the score window and the calculation of the expected power spectrum are repeated by the requested number. Matching unit 18
Then, the newly obtained power spectrum and the newly obtained expected power spectrum are replaced or added with the corresponding portions of the respective spectrum strings received so far, and DP matching is performed again. As a result, if the length of the spectrum sequence is still insufficient, or if there is a many-to-one correspondence, the matching result is returned to the matching control unit 14, and 7 is stored with new additional data. repeat. Eventually, the movement of the segment window and the spectrum creation are repeated until the matching result indicates a match. If there is no performance data before a matching segment is obtained from the performance data, the operation is terminated.

[0053] 8. If the results match, the matched pair is handled by the score window 17 and the time at the beginning of the segment measured from the beginning of the performance data.
Time of the pitch data that is
The information is output to the matching recording unit 19 as the time measured from the beginning of the original score information. After the output is completed, the matching unit 18 removes the output pair from the power spectrum sequence and the expected power spectrum sequence, respectively, and thereafter returns a result of the match to the matching control unit 14 so that each power spectrum sequence has a predetermined length. Until longer
Request additional power spectrum and expected power spectrum and repeat from 3.

Here, when the data ends and it becomes impossible to move the score window or the segment window, the operation ends. By repeating the above operation until the end of the data, pairs of the time in the performance data and the position on the musical score are successively obtained in the matching recording unit 19.

[0055]

According to the invention described above, it is possible to automatically create a time correspondence between a musical score and an actual performance record. In addition, by directly comparing spectra, it is possible to automatically create composite sounds, and furthermore, by recalculating only those parts whose temporal correspondence is insufficiently separated,
Calculation can be performed faster than before.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment of a music index creation device of the present invention.

FIG. 2 is a block diagram illustrating an embodiment of a matching unit according to an embodiment of the present invention.

FIG. 3 is a digital input waveform diagram used in an embodiment of the present invention.

FIG. 4 is a musical score used in the embodiment of the present invention.

FIG. 5 is a standard MIDI file format used in the embodiment of the present invention.

[Explanation of symbols]

 Reference Signs List 11 A / D data input unit 12 Segment window unit 13 Spectrum creation unit 14 Matching control unit 15 Music score data generation unit 16 Music score data input unit 17 Music score window unit 18 Matching unit 19 Matching recording unit 110 Spectrum synthesis unit 21 Distance comparison unit 22 Performance Spectrum buffer 23 Music score spectrum buffer 24 DP matching unit 25 Performance spectrum buffer 26 Note density calculation unit 27 Music score spectrum buffer 28 DP matching unit

Continuation of front page (56) References JP-A-58-152292 (JP, A) JP-A-58-152290 (JP, A) Transactions of the Institute of Electronics, Information and Communication Engineers, D-IIII VOL. J72-D-2 NO. 6P. 917-926 (June 1989) "Extraction of Performance Information in Piano Performance" Takami, Katayose, Iguchi

Claims (1)

(57) [Claims] 1. An audio recording medium on which music is recorded.
Sounds that can be expressed as notes in audio data and notes on music scores
The elapsed time of the performance and the score as the corresponding positions between
This device detects the correspondence with the index corresponding to the time above.
I, the audio data input means for inputting the audio data of the music, and the segment window for selecting and outputting a time-part of the data obtained from the audio data input means, a data output of the segment window A spectrum creation unit for creating a frequency spectrum, a score data input unit for a user to input a score, and a score window unit for selectively outputting a part of the data from the data provided to the score data input unit in time. A spectrum synthesizer for synthesizing a frequency spectrum corresponding to the partial score output by the score window; and comparing the frequency spectrum output by the spectrum generator with the frequency spectrum output by the spectrum synthesizer, the degree of matching is low. If only the degree of match is high, or if the degree is high, the center time of the segment window and the A matching unit for outputting first time information of the musical score window for outputting the Fumado section, before according to the degree of match the matching portion outputs
At least one of the segment window and the score window
A matching control unit that moves the window of the segment window from the segment window unit that is output by the matching unit.
The time and the time from the score window are played and the local
A music event index creation device, comprising: a matching recording unit for recording as corresponding time information .