CN101271457A - A melody-based music retrieval method and device - Google Patents

Abstract

The invention discloses a digital music retrieval method and its device, which can search for music containing a specified melody by using the music melody as a keyword. The invention provides users with two methods of inputting the melody: playing and humming. For the humming input method, a series of signal processing methods are used to analyze the humming audio signal and extract the melody information from it. For the music library, the inverted algorithm is used to compile the index to improve the efficiency of the search. The device of the present invention is divided into a server end and a client end. The function of the server end is to maintain the music database and its index, and respond to the inquiry request of the client end; The invention uses music melody to search for music to make up for the shortcomings of the traditional text-based search method, enabling users to search for desired music without knowing the text information; users can use common equipment such as computers, mobile phones, etc. to search for music.

Description

A melody-based music retrieval method and device

technical field

The invention belongs to the application field of computer technology, and in particular relates to a retrieval method for digital music using melody as a keyword, as well as computer hardware and communication equipment that enable the method to run smoothly.

Background technique

With the exponential growth of the amount of Internet information, how to quickly and accurately find the information we need from the massive information database has become a major bottleneck for people to use the Internet. Content-based multimedia retrieval is an emerging research field, which provides people with a new way of searching: using multimedia itself to search multimedia information. Multimedia information has various forms such as audio, video, image, and animation, among which audio information occupies a considerable proportion. Among audio, music is the most common form. The current music search is mainly based on text keywords, such as music name, author, singer, album, genre, lyrics, etc. However, music itself is fundamentally different from text keywords. Users use keywords to search, but the prerequisite is that users must understand the target music and be familiar with the relevant text information. If the user is only interested in the music melody itself, but knows nothing about text information such as song titles and lyrics, the existing music search methods will be useless.

Contents of the invention

Existing music keyword retrieval technology, if do not know the text keyword of target music, this text keyword search method just can't do anything, in order to solve the problem of prior art, the purpose of the present invention is to provide a kind of digital music based on melody Retrieval method and device.

In order to achieve the stated purpose, the first aspect of the present invention provides a melody-based music retrieval method, the steps are as follows:

Step S1: Designate a melody in the music to be searched as the melody keyword for searching;

Step S2: Input the specified melody keyword into the query client device, and obtain a digitized melody signal after processing;

Step S3: indexing the music in the music library, the index reflects the melody characteristics of the music, and forms an indexed music database;

Step S4: The search engine compares the digitized melody signal with the melody in the generated music database, and selects a group of music containing the specified keyword music melody from the music database;

Step S5: sort the selected music in descending order according to the degree of similarity to the melody keyword.

The music input methods include: playing input and humming input.

The index is indexing for the melody features of the melody segment.

For the humming input method, the following steps are taken to obtain the digitized melody signal:

Step S21: using an audio collection device to collect the user's humming input;

Step S22: Perform pre-filtering processing on the audio signal input by the user, including DC elimination, gain normalization, and low-pass filtering processing, to obtain audio frame sequence signals;

Step S23: Perform time domain or frequency domain analysis on the audio frame sequence signal to extract the fundamental frequency sequence;

Step S24: Perform further processing on the fundamental frequency sequence, including linearization and difference calculation, to obtain a digitized melody signal.

In order to achieve the stated purpose, the second aspect of the present invention provides a melody-based music retrieval device, including:

At least one server provides online music melody retrieval service;

Send an online music melody retrieval request with at least one client terminal device, and receive the result of querying the music melody from the server.

The client includes:

The input module is used to input the music melody information to be searched and send it to the server; the display module of the search result, the client obtains the search result from the server through the network or other transmission methods, and presents it to the user.

The input module includes:

The audio collection unit is used to collect the humming audio signal of the user; the note collection unit is used to collect the note melody signal played by the user; the audio signal processing unit converts the audio signal collected by the audio collection unit into a music melody signal.

The server includes:

The music data source interface unit is used to provide an interface for accessing various data sources to obtain original music data; the data acquisition and analysis unit is used to collect original music data and analyze the music data to extract music melody information; index The compilation unit is used to index the original music data obtained by the data acquisition and analysis unit according to its melody characteristics; the search unit is used to receive the query request from the input module of the client, and search the index generated by the index compilation unit for information related to the client. The terminal input module provides music with the same or similar melody as the melody keyword, sorts the search result list in reverse order of similarity, and feeds back to the search result display module of the client.

The music data source interface unit provides the interface of one or more of the following data acquisition methods:

Web: Take the method of web crawling, automatically roam on the Internet, grab music files and information related to the music files; file: grab and analyze music files stored in the local or network file system; database: Extract and analyze music files recorded in the database.

The client is one or more of the following devices:

Personal computers; smart mobile devices include: mobile phones, personal digital assistants, vehicle-mounted intelligent terminals, etc.; telephones; audio and video entertainment devices with media on-demand functions: including karaoke singing devices.

When the client selects a personal computer device, the personal computer client downloads and installs specific Web browser plug-in software from the server, and when the user accesses the music retrieval Web site provided by the server, it is used to provide the user with audio collection input and note collection melody The user interface, and collect the user's query input, and send it to the server through the Internet.

When the client selects a smart mobile device, the client installs specific software, which provides the user with a user interface for audio collection and note collection, and collects the user's query input and sends it to the server through the wireless network.

When the client selects the telephone equipment, the server provides a specific telephone audio station, the client dials the audio station number, uses the telephone number keypad, or uses the telephone receiver as the note collection and audio collection input devices respectively, and the server and the client pass through the public exchange Telephone network for information exchange.

When the client selects an audio-video entertainment device with a media-on-demand function, the client is equipped with a digital piano keyboard device, or virtual piano keyboard software is installed to collect the user's piano keyboard note input, and the karaoke microphone is used to collect the user's humming input, The server is a dedicated local server, and the scope of searching is the local music database of karaoke.

The server sorts the music list selected by the search result in descending order according to the similarity between the search result and the query input melody, and sends it back to the client for display.

The present invention provides a new search mode for users, that is, to search music by music melody. It makes up for the shortcomings of the traditional text-based search method, enabling users to search for desired music without knowing the text information; the invention also implements this search method on a specific hardware platform, so that users can use common equipment such as computers, mobile phone, etc., for music search.

Description of drawings

Fig. 1 structural representation of the present invention

Detailed ways

The present invention and its advantages will be described in detail below with reference to the accompanying drawings. It should be noted that the described embodiments are only intended to facilitate the understanding of the present invention, and have no limiting effect on it.

The present invention mainly researches the content based Music Retrieval (Content based Music Retrieval), provides a kind of mode that uses music itself to search for music. Specifically, a short piece of music melody is used as a search keyword, and the search engine returns a group of music that contains the specified key melody. The melody is used as a keyword, which is different from the text keyword. The user cannot input directly from the keyboard, but needs to provide a special method for inputting the melody. The method most in line with people's habits is humming input. The user only needs to use an audio acquisition input device, such as a microphone, to hum a melody that needs to be searched. In addition, users can also play input through the virtual piano keyboard.

Embodiments of the present invention provide a complete computer technology application system platform, its function is to provide music search service based on melody, the platform simultaneously realizes music original data acquisition, music original data analysis, music database indexing, online query , audio signal processing, information feedback and other functions. The system platform has the conditions for humming input and piano keyboard playing input music melody on terminal equipment such as ordinary personal computers, smart mobile devices, telephones, and karaoke equipment, and has the ability to communicate with the above terminal equipment. The conditions under which the user displays or renders search results.

The present invention is formed by the organic combination of a plurality of functional modules, and each functional module completes a specific function. The complete structure of the system is shown in Figure 1. The melody-based music retrieval device of the present invention includes at least one computer as server 2 to provide online music retrieval service, and at least one client terminal device 1 sends an online music retrieval request, and receives the query result of server 2, and server 2 from multiple A data source acquires and stores a music melody database containing a large number of music melody features, and indexes the database. When receiving the query request from client 1, server 2 compares the query melody segment input by the user with the melody in the database, and filters out the music that is not related to the query melody segment, and divides the remaining candidate music according to the Query the similarity ranking of melody fragments, and return the sorted music list to the client. Client 1 provides two input interfaces for the user, receives the user's melody input and converts it into a digitized melody signal that can be used for query.

In the structural diagram shown in Figure 1, the components in the dotted box on the left half are modules in the terminal device of the client 1, including: the input module 11 collects user input and sends it to the server 2, and the search result display module 12 sends the server 2 The returned query results are presented to the user.

The input module 11 includes: an audio collection unit 111 and a note collection unit 113, which are respectively used to collect humming input and playing input of the user; an audio signal processing unit 112, which converts the audio signal collected by the audio collection unit 111 into music Melody information.

The audio collection unit 111 collects the user's humming input. It consists of audio acquisition equipment and a piece of recording program software. The audio collection device is usually a microphone on a personal computer and a karaoke terminal, and is usually a receiving microphone on a communication terminal such as a mobile phone. It is driven by the recording software, digitally collects the analog signal of the audio waveform according to the sampling frequency specified by the recording software, and stores the collected digital pulse sequence in the memory of the client 1. Since the fundamental frequency (first harmonic) of the human voice is usually within 2000 Hz, according to the Nyquist sampling theorem, in order to ensure that the collected digital signal does not have frequency aliasing, the sampling frequency should be greater than twice the highest effective frequency. Since the present invention needs to analyze the harmonics of human voice, the sampling frequency is 8000Hz or 11025Hz. The audio collection unit 111 defaults to 10 seconds for each collection, which can be set according to the situation.

The audio signal processing unit 112 converts the audio signal collected by the audio collection unit 111 into melody information of music. The audio signal processing unit 112 performs the following processing on the audio signal:

Step 1), the audio signal collected by the audio collection unit 111 usually contains a DC component, and the DC component causes a shift in the potential of the equilibrium position of the signal, causing errors in the analysis of the low-frequency spectrum of the signal. Therefore it is necessary to eliminate the DC component of the signal. Due to the sometimes constant characteristic of the DC signal, the DC component can be eliminated by subtracting the potential value of the global equilibrium point of the sampling signal from the potential values of all sampling points. In order to eliminate the error caused by the difference in signal strength, the audio signal processing unit 112 also standardized the signal strength by setting it as 1 for the energy maximum value of a sampled signal, and taking this point as the reference point for all other points. The standard is scaled up or down proportionally, ensuring that the maximum energy value for any one sample is equal. In addition, processing the sampling signal through a low-pass filter can suppress high-frequency noise and improve the signal-to-noise ratio.

Step 2), the signal processed in step 1) is framed, and there is a certain overlap between adjacent frames. In speech signal processing, the length of each frame signal is usually within 200 milliseconds, so that each frame signal can be viewed approximately Make a smooth signal. Windowing and filtering are performed on each frame of data. The Hanning window filter formula is as follows:

${\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0.5</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; cos</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&pi;n</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>$

Step 3), Fourier Transform (Fourier Transform) is a method of transforming a time-domain signal into a frequency-domain signal. In the frequency domain, the energy distribution of the signal on different frequency components can be clearly and intuitively reproduced. In this step, the fast Fourier transform (FFT) algorithm is used to transform each frame signal processed in step 2) into the complex frequency domain to obtain a complex vector of each frequency component. Each complex vector includes two components of the real axis and the imaginary axis, and the sum of their squares is taken to obtain the energy value, which represents the strength of the frame signal on each frequency component. The fast Fourier transform requires that the number of input sampling points be 2 ^N , if the number of sampling points in each frame in step 2) is less than 2 ^N , fill the insufficient points with 0.

Step 4), in the frequency domain distribution of each frame processed in step 3), if the peak energy can be found in the human voice frequency band, and significantly exceeds the energy of the background noise, then the frequency corresponding to the first peak that satisfies the condition is human The fundamental frequency of the sound. Compare the fundamental frequency values of adjacent frames, if the change is not large, it is considered to be the same note, and if the change is large, it is considered to be the conversion of the note. Additionally, silence frames can also serve as note boundaries.

Step 5), between two adjacent notes, find the logarithmic difference of their frequencies, and obtain the difference feature sequence of the melody notes. Taking the logarithm of the frequency values linearizes the frequency values that grow exponentially with the scale so that the difference between the scales is proportional to the logarithmic difference of their frequencies. Using the logarithmic frequency difference of the note as the feature of the melody can eliminate the difference caused by different keynotes when different users hum.

After the above five steps, the audio of human humming is converted into melody feature information, which can be sent to the server as a key feature for search. In the above fundamental frequency extraction step, a time-domain method, such as an autocorrelation method, can also be used.

The note collection unit 113 is an interface for playing and inputting melody by means of piano keyboard input. The musical note collection unit 113 displays a piano keyboard on the terminal device of the client 1, and the user can use a mouse or other touch devices such as a touch screen, a stylus, etc. to click on the corresponding keys to input the melody. The musical note collection unit 113 numbers each key of the piano in order of pitch as an ID of each key. The difference between the IDs of two adjacent keys clicked by the user is the note difference with the same meaning as the output of the audio signal processing unit 112, which is sent to the server 2 as a melody feature. The note information collected by the piano keyboard does not require signal processing operations. Therefore, the melody inputted by the piano keyboard has the advantages of being error-free and fast.

Since ordinary telephone equipment does not have data processing capabilities, in the telephone terminal equipment, the audio signal processing unit 112 runs on the server 2 side, and the client terminal 1 telephone equipment is only responsible for collecting user input. In the input mode of humming, the user uses the telephone to receive the microphone as the audio collection unit 111, and the audio signal is transmitted to the server end through the public switched telephone network (PSTN); Enter the melody in the form of musical numbered notation. After the server 2 receives the phone key signal, the server 2 and the client 1 exchange information through the public switched telephone network (PSTN), convert it into corresponding musical notes, and feed back to the user for correction. .

The display module 12 of the search result, the client 1 obtains the music melody information result required by the search from the server 2 through the network or other transmission methods. The search result is presented in the form of a list, and each item in the list is a music name (title), and information such as author and singer. The music in the list is sorted in descending order of similarity.

In the structural diagram of Fig. 1, what is in the virtual box on the right is the server 2, including: music data source interface unit 21, data acquisition and analysis unit 22, indexing unit 23, search unit 24, they complete data collection and analysis data in the background , indexing, and online search operations.

Data acquisition and analysis unit 22, it is responsible for collecting original music data file, and music data file is analyzed, therefrom extracts music melody information; The music file format directly supported by the present invention is MIDI format, therefore, data acquisition and analysis unit 22 mainly analyzes MIDI music files. The MIDI file format stores elements of music in the form of digital instructions, such as pitch, duration, timbre, rhythm, etc. By analyzing the music digital instruction sequence in the MIDI file, the parameters of the music can be extracted conveniently and accurately. MIDI music files can be viewed as a hierarchical structure. Common MIDI files come in two formats: single-track format (Type 0), and multi-track format (Type 1). In the single-track format, each file contains a track, and each track has 16 channels, and each channel can store an instrument. During playback, 16 channels play simultaneously. The single-track format can play up to 16 instruments at the same time, which can meet the needs of general digital music. In a multi-track format music file, each file contains multiple audio tracks (tracks), and each audio track also includes 16 channels, but only one channel of each audio track is active, and the other channels are all empty. Multiple audio tracks are also played simultaneously. The multi-track format can play more than 16 instruments at the same time, so some expressive digital music often uses this format. The data acquisition and analysis unit 22 unifies the two file formats to establish a hierarchical structure: MIDI-track-channel-note four levels, and the upper-level elements are composed of lower-level elements. Each non-empty channel contains a sequence of notes. The data acquisition and analysis unit 22 converts each music file into an object with a hierarchical structure, and also saves the fingerprint information, title, author and other related information of the music.

The job of any search engine is to return a list of information that matches the user's query within an acceptable amount of time. Here, there are three concepts to pay attention to:

1) Acceptable time. This refers to response time. For software that provides services to a large number of users on the Internet, this time cannot be too long, usually on the order of "seconds". This is a basic index to measure the availability of search engines, and it is also a difference from traditional information retrieval systems. Furthermore, such a response time requirement must not only be able to satisfy a single user's query, but also satisfy all users under the system design load. In other words, the system should guarantee a second-level response time at the rated throughput rate.

2) Match. Taking a webpage as an example, it means that the webpage contains the query keyword input by the user in some form, or the content very similar to the query keyword appears. In the melody-based music search engine system, matching means that the main melody of the music contains the melody keyword input by the user. The input of the user melody deviates from the target melody. Therefore, the matching must not only be able to match accurately, but also need to have a certain degree of fault tolerance.

3) List. The search result returned to the user by the search engine is usually a list containing multiple results, and each element in the list is similar or related to the keyword entered by the user to a certain extent. However, the vast majority of users only care about the elements ranked on the first page in the result list, so it is necessary to sort the elements in the search result list by similar relevance. This sorting is called Rank. At present, different search engines adopt different ranking algorithms. For example, Google uses the PageRank algorithm, which ranks the importance of the pages in the results, while Baidu uses the method of bidding ranking.

In the search engine system, the quality of the indexing algorithm has a crucial impact on the above three performance indicators. In current melody-based music search engines, most of them adopt linear matching algorithms. This algorithm is to regard the user's input melody and the melody in the music file as two strings respectively, and compare the similarity of the strings. In the field of content-based music search, methods such as Suffix Tree, Suffix Array, and Linear Alignment are commonly used. However, linear search has a common defect. During the search process, each element in the database needs to be scanned to determine whether it matches. This is acceptable when the amount of data in the original database is not large, but as the amount of data in the database increases, in the most ideal case, the search time will also increase linearly, that is, the time complexity of the search At least O(n), for example, in the Suffix Array algorithm, its time complexity is O(nlogn). Now the data volume of large search engines is usually on the order of 10 ⁸ to 10 ⁹ , if such a huge database is linearly scanned, the calculation time is unacceptable to users. Therefore, large search engines generally use the inverted index algorithm.

Among the many search algorithms, Inverted Index has been widely used rapidly due to its flexibility, high efficiency, and versatility. It is a word-based indexing algorithm, which can directly filter out irrelevant content in the database according to the keywords entered by users, and can sort the relevance of related content, and has good fault tolerance performance, and can approximate Content is identified.

In the text of most languages, there are natural separators between words, such as spaces, punctuation marks, etc. In languages that do not have natural word segmentation, such as Chinese, there are relatively mature word segmentation technologies. Inverted index is to classify the same word that appears in different articles into one category according to the frequency of each word in the article. The word is used as the primary key of the index, and the article containing the word is used as the element list. In this way, when several specific words appear in a query, the system will directly search for article elements under these specific words, and articles irrelevant to the query will be automatically filtered out. This automatic filtering does not require CPU resources, so it is very efficient. This efficient and automatic filtering mechanism of irrelevant information is the advantage of the unique data structure of the inverted index.

In the music search engine system, the object of search is music melody, not text. Therefore, some modification of the text-based inverted index model is needed to adapt it to the indexing of music melodies.

A musical melody is composed of a continuous sequence of notes. In music, although there are also bars that can divide a piece of music into small sections, in the MIDI music format, there is no obvious sign of bar separation. In addition, rests are very similar to spaces in text, but in different styles of music, the appearance of rests is very random, and there is no regularity with obvious characteristics. Therefore, the natural separators of music such as bars and rests are not suitable for dividing melody.

Since the music melody itself has not found a good word segmentation mechanism at present, the present invention adopts a melody segment segmentation method. Divide a continuous melody into small sections, each section contains 3 to 4 notes, and there is a certain overlap between sections. The invention uses the melody segment as the word segmentation of the music melody, and uses the inverted algorithm to compile the index. When there is a new music track that needs to be added to the index, it is only necessary to divide the song into melody segments, and add the song to the element set of each melody segment.

The indexing unit 23 is used for indexing the pieces of music melody information as word segments of the music melody according to the above method, and building an index for the music data provided by the data acquisition and analysis unit 22 .

The search unit 24 is used to receive the query request of the client input module 11, and search for the same or similar melody as the music melody information inquired by the audio collection unit 111 or the note collection unit 113 in the client 1 in the index generated by the index compilation unit 23 The online music search operation is used to sort the search result list in reverse order of similarity, and feed back to the search result display module 12 of the client.

As mentioned above, sorting search results by similarity is an important function of search engines. The search unit 24 calculates the similarity according to the number of identical notes in the client query string and the melody string in the music library. The more identical notes, the more similar they are.

The search unit 24 adopts different interaction modes according to different client devices 1 .

When client 1 is a personal computer device, the personal computer client downloads and installs specific Web browser plug-in software from server, and this plug-in software integrates the recording program in the audio collection module 111 and the virtual piano keyboard program of the musical note collection module 113. When the user accesses the music retrieval Web site provided by the server, it is used to provide the user with a user interface for audio collection input and note collection melody, and collects the user's query input and sends it to the server through the Internet.

When client 1 is a smart mobile device, client 1 installs specific software, which is developed based on the mobile device operating system platform used by the user (such as Windows Mobile platform, Linux platform, Nokia S60 platform, Java platform, etc.), for the user Provide a user interface for audio collection input and note collection melody, and collect user query input and send it to the server through the wireless network.

When client 1 selects a telephone device, server 2 provides a specific telephone audio station, client 1 dials the audio station number, uses the telephone numeric keypad, or uses a telephone receiver as an audio collection input device, server 2 and client 1 through the public switching telephone Network (PSTN) for information exchange.

Select the audio-video entertainment device with media on demand function for client 1, client 1 is equipped with hardware digital piano keyboard equipment, or installs virtual piano keyboard software to collect user's piano keyboard note input, utilizes karaoke microphone to collect user's humming input, Server 2 is a dedicated local server, and the scope of searching is the local music library of karaoke.

For computers and mobile smart devices, the search results are presented to users in the form of a list, and users can download, play and other operations without infringing on the intellectual property rights of music works. For the client 1 of the phone, the server 2 will read the search result list in the form of voice prompts, and the user can select it with the phone buttons. For the jukebox device client 1, after the user selects it, he can perform operations such as reservation and on-demand.

The music data source interface unit 21 is used to provide multiple different data source access interfaces, so that the server can obtain original music data from different data sources, and expand the music database according to specific purposes and requirements, for example:

1. Take the method of web crawling to automatically roam on the Internet to grab music files and information related to the music files; or

2. Take the crawling and analysis of files stored in the local or network file system; or

3. Extract and analyze the music records in the database.

The present invention is not limited to the above three data sources, but provides an application program interface (API) capable of secondary development, which can further expand the data source.

The above description is an embodiment for implementing the present invention, and those skilled in the art should understand that any modification or partial replacement that does not depart from the scope of the present invention belongs to the scope defined by the claims of the present invention.

Claims (15)

1. the music retrieval method based on melody is characterized in that,

Step S1: specify one section melody waiting to look in the music as the melody key word of searching for;

Step S2:, obtain the digitizing melody signal through handling with specified melody key word input inquiry client device;

Step S3: the music in the music libraries is set up index, and this index embodies the melody characteristics of music, forms the musical database of indexation;

Step S4: by search engine the melody in the musical database of digitizing melody signal and generation is compared, select one group of one group of music that comprises the nominal key music rhythm from musical database;

Step S5: with the music selected according to the similarity degree sort descending of melody key word.

2. music retrieval method according to claim 1 is characterized in that, described music input mode comprises: play input and humming input.

3. music retrieval method according to claim 1 is characterized in that, described index is the index of working out at the melody characteristics of melody fragment.

4. music retrieval method according to claim 2 is characterized in that, for the humming input mode, takes following steps to obtain digitized melody signal:

Step S21: use audio collecting device to gather user's humming input;

Step S22: the sound signal of user's input is carried out pre-filtering handle, comprise direct current elimination, gain normalization, low-pass filtering treatment, obtain the audio frame sequence signal;

Step S23: the audio frame sequence signal is carried out time domain or frequency-domain analysis, extract the fundamental frequency sequence;

Step S24: the fundamental frequency sequence is further handled, comprised linearization, ask poor, obtain digitized melody signal.

5. the music retrieval device based on melody is characterized in that, comprising:

At least one station server (2) provides online music rhythm retrieval service;

Send online music rhythm retrieval request with at least one client (1) terminal device, and the result of reception server query music melody.

6. music retrieval device according to claim 5 is characterized in that, described client (1) comprising:

Load module (11) is used to import the music rhythm information that need search, and sends it to server (2);

The display module of Search Results (12), client (1) obtains Search Results by network or other transmission modes from server (2), and presents to the user.

7. music retrieval device according to claim 6 is characterized in that, described load module 11 comprises:

Audio collection unit (111) is used to gather user's humming sound signal;

Note collecting unit (113) is used to gather the note melody signal that the user plays;

Audio signal processing unit (112), the sound signal that audio collection unit (111) are gathered is converted into the music rhythm signal.

8. music retrieval device according to claim 5 is characterized in that, described server (2) comprising:

Music data source interface unit (21) is used to provide the various data sources of visit to obtain the interface of original music data;

Data are obtained and analytic unit (22), be used to collect original music data, and music data is analyzed, and therefrom extract music rhythm information;

Authorized index unit (23) is used for that data are obtained the original music data of obtaining with analytic unit 22 and sets up index according to its melody characteristics;

Search unit (24), be used to receive the query requests of client load module (11), and the music of the identical or close melody of the melody key word that provides with client load module (11) is provided in search in the index that authorized index unit (23) generate, search result list is pressed the ordering of similarity degree inverted order, and feed back to the search result display module (12) of client (1).

9. music retrieval device according to claim 8 is characterized in that, described music data source interface unit (21) provides the interface of following one or more data obtain manners:

Web: the mode of taking the Web network to grasp, music file and the information relevant with this music file are grasped in roaming on the internet automatically;

File: the music file of storing in this locality or the network file system(NFS) is grasped and analyzes;

Database: the music file that writes down in the database is extracted and analyzes.

10. music retrieval device according to claim 5 is characterized in that, described client (1) is one or more in the following equipment:

PC, intelligent mobile device, phone, audio frequency and video amusement equipment with media-on-demand function.

11. according to claim 5 or 10 described music retrieval devices, it is characterized in that, when described client (1) is selected personal computer equipment, PC client (1) downloads and installs specific Web browser plug-in software from server (2), during music retrieval Web website that user access server (2) provides, the user interface that is used to the user to provide audio collection input and note to gather melody, and collection user's inquiry input are sent to server (2) by the internet.

12. according to claim 5 or 10 described music retrieval devices, it is characterized in that, when described client (1) is selected intelligent mobile device, client (1) is installed specific software, the user interface that this software provides audio collection and note to gather for the user, and gather user's inquiry input, be sent to server by wireless network.

13. according to claim 5 or 10 described music retrieval devices, it is characterized in that, when described client (1) is selected telephone plant, server (2) provides specific phone information service center, client (1) is dialed this information service center number, utilize the phone numbers keyboard, or use telephone receiver respectively as note collection and audio collection input equipment, server (2) carries out information interaction with client (1) by PSTN.

14. according to claim 5 or 10 described music retrieval devices, it is characterized in that, when described client (1) is selected to have the audio frequency and video amusement equipment of media-on-demand function, client (1) is equipped with digital piano keyboard equipment, or the fingerboard note that virtual piano keyboard software collection user is installed is imported, utilize microphone of carok collection user's humming input, server (2) is a dedicated local server, and the scope of search is the local musical database of Karaoke.

15. according to claim 5 or 10 described music retrieval devices, it is characterized in that, the music list that described server (2) is chosen for Search Results according to the similarity sort descending of Search Results with inquiry input melody, and sends it back client (1) and shows.

Images