JPH01219624A - Automatic score taking method and apparatus - Google Patents

Abstract

PURPOSE:To perform accurate segmentation at the start point of sound, by performing segmentation processing for dividing an acoustic signal into a section regarded as the same intervale at the rising change point of power data extracted from the acoustic signal. CONSTITUTION:The digital acoustic signal such as singing passing through an acoustic signal input apparatus 8 and an A/D converter 7 by a CPU 1 corresponding to the order of a keyboard 4 is stored in the auxiliary memory device 6 of a working memory and the stored content is subjected to process control processing by the CPU 1 corresponding to the program of a main memory device 3 to extract power data and the rising change point of the extracted power data is detected to perform segmentation dividing the acoustic signal into the same intervale section. Therefore, accurate segmentation is performed at the start point of sound and a highly accurate score is automatically formed by score taking processing on and after based on said segmentation.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to an automatic musical notation method and apparatus for creating musical score data from acoustic signals such as singing voices, humming voices, musical instrument sounds, etc. This relates to segmentation processing that divides a signal into sections that can be considered to have the same pitch.

[Prior Art] In an automatic score transcription system that converts acoustic signals such as singing voices, humming voices, and musical instrument sounds into musical score data, basic information as a musical score such as length, interval, key, etc. is extracted from the sound signal. It includes detecting time signature and tempo.

By the way, the acoustic signal is only a signal that continuously includes a repeating waveform of the basic waveform, and the above-mentioned information cannot be obtained immediately.

Therefore, in the conventional automatic music transcription method, first, repetition information (hereinafter referred to as pitch information) and power information of the basic waveform representing the pitch of the acoustic signal are extracted for each analysis period, and then at least the extracted power The process of dividing an acoustic signal into sections (segments) that can be considered to have the same pitch based on information is called segmentation).Then, from the pitch information of the segments, the pitch of the acoustic signal of each segment is identified as an interval along the absolute pitch axis. However, each piece of information was obtained in the following order: the key of the acoustic signal was determined based on the distribution of pitch information, and the time signature and tempo of the acoustic signal were further determined based on the segments.

Therefore, since pitches, beats, tenbos, etc. are determined based on segments (note lengths), segmentation processing is particularly important when creating musical score data.

[Problem M to be solved by the invention] As described above, segmentation is an important element in creating musical score data, and if the accuracy of segmentation is low, the accuracy of the final musical score data obtained will also be significantly low. , when final segmentation is performed from both the segmentation result based on pitch information and the segmentation result based on power information, or when the final segmentation is performed from power information, the accuracy of the segmentation process itself from power information is also improved. It is hoped that

The present invention has been made in consideration of the above points, and aims to provide an automatic score transcription method and device that can successfully perform segmentation based on power information and improve the accuracy of musical score data. It is something to do.

[Means for solving the problem] In order to solve the problem, in the first invention,
In an automatic music score method for converting an acoustic signal into musical score data, the method includes at least a process of extracting power information from an input acoustic signal and a segmentation process of dividing the acoustic signal into sections that can be considered to have the same pitch based on the power information, and the segmentation process However, this can be done by extracting the rising change point of the power information and classifying the acoustic signal based on the extracted rising change point.

Further, in the second aspect of the present invention, the power extraction means for extracting the input acoustic signal power information and the segmentation means for dividing the acoustic signal into sections that can be considered to have the same pitch from the extracted power information are provided. In an automatic notation device that converts an acoustic signal into musical score data, the segmentation means includes a change point extracting section that extracts a rising change point of power information, and a dividing section that divides the acoustic signal at the extracted rising change point. The configuration is now equipped with the following.

[Operation] In the first aspect of the present invention, when the power information is divided into intervals that can be considered to be the same pitch, the power of the acoustic signal is maximized at the starting point of a new sound, regardless of its source. , and then gradually attenuating, we extracted the rising change point of the power information and classified the acoustic signal at the extracted change point.

Further, in the second aspect of the present invention, when the power information is divided into intervals that can be considered to be the same pitch by the segmentation means, the change point of the rising edge of the power information is extracted by the change point extracting section by paying attention to the way the power information changes as well. Then, the acoustic signal is divided by the dividing section at the extraction change point.

[Example] Hereinafter, an example of the present invention will be described in detail with reference to the drawings.

Anal swallowing difficulty method First, an automatic score recording method to which the present invention is applied will be explained.

In FIG. 3, a central processing unit (CPU) 1 controls the entire device, and performs the music transcription process shown in FIG. 4 stored in a main storage device 3 connected via a bus 2. It executes the program. Connected to the bus 2 are a keyboard 4 as an input device, a display device 5 as an output device, an auxiliary storage device 6 used as a working memory, and an analog/digital converter 7, in addition to the CPUI and main storage device 3. There is.

Connected to the analog/digital converter 7 is an acoustic signal input device 8 consisting of, for example, a microphone. This acoustic signal input device 8 captures acoustic signals such as singing or humming vocalized by a user, or musical sounds generated from musical instruments, and converts them into electrical signals, and converts the electrical signals into electrical signals. 7.

When the processing is instructed by the keyboard input device 4, the CPtJl starts the scoring processing, executes the program stored in the main storage device 3, and converts the audio signal into a digital signal by the analog/digital converter 7. The signals are temporarily stored in the auxiliary storage device 6, and then the above-mentioned program is executed to convert these acoustic signals into musical score data, which is output to the display device 5 as necessary.

Next, the score transcription process performed by the CPU after capturing the audio signal will be described in detail according to the flowchart shown at the functional level in FIG. 4.

First, the CPU 1 performs autocorrelation analysis on the acoustic signal to extract pitch information of the acoustic signal for each analysis period, performs square sum processing to extract power information for each analysis period, and then performs noise removal and smoothing processing. Execute post-processing such as (step SPI
, SP2>. After that, the CPU calculates the amount of deviation of the pitch axis of the acoustic signal from the absolute pitch axis based on the distribution status of the pitch information, and executes tuning processing to shift the obtained pitch information according to the amount of deviation. (Step 5P3). That is, the pitch information is corrected so that the difference between the pitch axis of the singer or musical instrument that generated the acoustic signal and the absolute pitch axis becomes small.

Next, the CPU obtains continuous periods of pitch information in which the obtained pitch information is considered to indicate the same pitch, performs segmentation to cut the acoustic signal into segments for each note, and also calculates the obtained power. Segmentation is performed based on changes in information (step SP4.5P5). Based on both of the obtained segment information, the CPUI calculates a reference length corresponding to the time length of a quarter note, eighth note, etc., and executes segmentation again based on this reference length (step 5P6).
.

Based on the pitch information of the segments segmented in this way, the CPUI identifies the pitch of the segment to the pitch on the absolute pitch axis that can be determined to be closest to the pitch information, and further identifies the pitch of the identified continuous segment. Segmentation is performed again based on whether or not they are the same (step SP7.5P8).

After that, the CPU calculates the product sum of the appearance frequency of the pitch obtained by summing up the pitch information after the tuning process and the predetermined weighting coefficient determined according to the key, and calculates the product sum based on the maximum information of this product sum. For example, the key of the music of the input acoustic signal is determined, such as C major or A minor, and the pitch information is reviewed for a predetermined interval of the scale in the determined key to confirm and correct the interval (step SP9 .5P
10). Then, the CPUI performs a segmentation review based on whether consecutive segments from the finally determined pitch are the same and whether there is a change in power between consecutive segments;
Perform final segmentation (step 5P1
1).

Once the pitch and segment are determined in this way, C
PUI extracts measures from the viewpoints of whether a song starts from the first beat, the last note of a phrase does not span the next measure, or there is a break between each measure, and determines the time signature from this measure information and segmentation information. Then, the tempo is determined from this determined time signature information and measure length (step 5).
P12.5P13).

The CPU then organizes the determined pitch, length, key, time signature, and tempo information and finally creates musical score data (Step 5P14>).

Power Based Segmentation Next, the segmentation process (see step SP5) based on power information in such an automatic score transcription method will be described in detail using the flowchart of FIG.

Note that, as the power information of the acoustic signal, the acoustic signal is squared for each sampling point within the analysis period, and the sum of these squared values is used as the power information for that analysis period.

After first clearing the analysis point parameter t, the CPU confirms that the data to be processed has not finished, and then calculates the power information P(t
) and the power information P (t+k) at the analysis point t+, which is advanced by 1 point, and the power information change function shown in the following formula (% ()) d(t) (hereinafter referred to as a rising edge extraction function) (steps 5P20 to 22). Note that k is selected to be an appropriate time difference suitable for capturing changes in power information.

After that, the CPU determines whether the rising edge extraction function d (t) at the analysis point t is equal to or greater than the threshold 0, and if it is smaller than the threshold θ and obtains a negative result, it increments the parameter t and returns to step 5P21 described above. (Step 5P
23-24).

By repeating this process, the CPUI eventually finds the analysis point immediately after the rise extraction function d(t) changes to the threshold value 0 or more, and obtains a positive result in step 5P23. At this time, after marking the start of a segment on the analysis point, the CPU confirms that the data for the analysis point to be processed has not ended, and calculates the power information P(t) for that analysis point t. The rising edge extraction function d(t) of the power information is calculated again based on the power information P (t+k) at the analysis point t+, which is advanced by one point (steps 5P25 to 27).

After that, the CPU determines whether the rise extraction function d (t) at the analysis point t is smaller than the threshold θ, and if it is greater than or equal to the threshold 0 and obtains a negative result, it increments the parameter t and performs the step 5P26 described above. Return to step 5 (P28-2).

On the other hand, it becomes smaller than the threshold value θ and step 5P2
If a positive result is obtained in step 8, step 5P2 described above is performed.
1 and then proceeds to the process of extracting the change point immediately after the rise extraction function d(1) changes to the threshold value 0 or more.

By repeating such processing, a segment start mark is placed at each rising and changing point of the power information, and eventually the processing is completed for all power information, and step 5P21
Alternatively, an affirmative result is obtained in 5P26, and the program is terminated.

The reason for performing segmentation by extracting the rise of power information in this way is that, for example, when a singer changes the pitch of a note, the power at the starting point of the new note is maximized, This is because it is designed to gradually attenuate thereafter. This is also because musical instrument sounds have the characteristic of attacking at the beginning of the sound and then decaying.

FIG. 2 shows an example of the time change of the power information P (t) and the time change of the rising edge extraction function d(t). In this example, by executing the process of FIG. It is divided into S1, S2...SN.

Therefore, according to the above-described embodiment, since the rising edge of the power information is extracted and segmentation is performed at that rising edge, highly accurate segmentation that can be performed at the starting point of the sound can be performed. .

In addition, in the above-mentioned embodiment, the sum of squares of the acoustic signal is used as the power information, but other parameters may be used. For example, the square root of the sum of squares may be used.

Further, in the above-described embodiment, the function shown in equation (1) is used as the function for extracting the rising edge, but other functions may be applied. For example, a differential function with a fixed denominator may be applied.

Furthermore, in the above-described embodiment, all the processes shown in FIG. You can also run it with For example, as shown in FIG. 5, in which parts corresponding to those in FIG. The signal processing processor 13 performs autocorrelation analysis on the acoustic signal converted to the digital signal to extract pitch information, and performs sum-of-squares processing to obtain power. The information may be extracted and provided to a software processing system using the CPUI. Hardware configuration like this (10
The signal processing processor 13 used in ~13) is a processor that is capable of real-time processing of audio band signals and is equipped with an interface signal with the CPU of the post (for example, μP manufactured by NEC Corporation).
D7?20) may be applied.

[Effects of the Invention] As described above, according to the present invention, the rising change point of the power information is extracted and the acoustic signal is segmented at the change point, so segmentation can be performed accurately at the starting point of the sound. This allows subsequent processing such as pitch identification processing using this segmentation result to be executed satisfactorily.

[Brief explanation of the drawing]

FIG. 1 is a flowchart showing segmentation processing based on power information according to an embodiment of the present invention, FIG. 2 is a characteristic curve diagram showing temporal changes in power information and a rising edge extraction function together with segmentation results, and FIG. 3 is a flowchart showing a segmentation process based on power information according to an embodiment of the present invention. A block diagram showing the configuration of an automatic score transcription method applying
The figure is a flowchart showing the automatic music transcription processing procedure.
The figure is a block diagram showing another configuration of the automatic score transcription system. 1... CPU, 3... Main storage device, 6... Auxiliary storage device, 7... Analog/digital converter, 8...
Acoustic signal input device.

Claims (2)

[Claims] (1) Automatically converting the acoustic signal into musical score data, including at least a process of extracting power information from the input acoustic signal and a segmentation process of dividing the acoustic signal into sections that can be considered to have the same pitch based on the power information. An automatic score transcription method, wherein the segmentation process is a process of extracting a rising edge change point of the power information and dividing the acoustic signal at the extracted rising edge changing point. (2) A power extractor for extracting power information from an input acoustic signal, and a segmentation means for dividing the acoustic signal into sections that can be considered to have the same pitch based on the extracted power information. In an automatic music transcription device that converts a signal into musical score data, the segmentation means includes a change point extractor that extracts a rising change point of the power information, and a segmentation unit that divides the acoustic signal by the extracted rise change point. An automatic score transcription device characterized by: