CN115565507A - A musical instrument recognition and automatic notation system based on AI technology - Google Patents

Abstract

The invention belongs to the field of musical instrument identification, and discloses a musical instrument identification and automatic notation system based on AI technology, which comprises a client, a server and an algorithm end; the client is used for displaying and editing the user recording and the electronic music score; the server side is used for collecting user audio data and deploying algorithms; the algorithm end is deployed on the server end and comprises an AI musical instrument identification algorithm and an AI rotating spectrum algorithm; the AI musical instrument recognition algorithm is used for recognizing the playing musical instrument in the audio to be analyzed and converting the audio into MIDI and a corresponding Musicxml file through an AI spectral conversion algorithm; and the server is also used for sending the MIDI and Musicxml files to the client for the user to carry out secondary editing. The scheme is based on AI technology, integrates instrument identification, automatic music score recording and music score editing, meets the requirements of users using different instruments, and provides convenience for the users to record music inspiration at any time and make and edit music scores.

Description

A musical instrument recognition and automatic notation system based on AI technology

technical field

The present invention relates to the field of musical instrument recognition, and more specifically, relates to a musical instrument recognition and automatic notation system based on AI technology.

Background technique

Computer music is a new subject that combines computer technology and music art. Music artificial intelligence belongs to computer music technology. With the development of artificial intelligence technology, music artificial intelligence technology is also becoming more and more mature. Instrument recognition and automatic notation are the application fields of music artificial intelligence. In addition, with the improvement of people's living standards, the music lovers and professional groups engaged in music career are growing day by day. Notation is one of the basic needs of music groups, and it can help them record inspiration or score.

At present, the notation tools on the market mainly include notation music software, automatic notation software, etc. There are mainly two ways for the notation software to realize the notation: 1. Manually input the music symbols required for notation in sequence, such as MuseScore, Sibelius, etc.; 2. Connect the midi input device, input the note information, and then manually Adjust scores, such as Cubase and some smart piano devices, etc.;

Automatic notation software is currently mainly based on piano or human vocal humming, using computer technology and acoustic technology to analyze the acoustic signal and convert it into a stave, such as ScoreCloud.

In the existing notation software and systems, the steps of traditional manual notation software are too cumbersome and the workload is heavy. Although the method of notation through midi equipment simplifies the notation process, this way through additional hardware, Ease of use is also affected.

However, the current automatic notation software and systems collect audio through the microphone of the mobile device, use computer technology to analyze the audio signal and convert it into a musical score. Although this method greatly simplifies the cumbersome process of the first two methods and is very convenient, its disadvantages cannot be ignored: on the one hand, most of them currently use traditional audio signal processing algorithms, which are not combined with music theory. Therefore, the recognition accuracy is low and the effect is not good; on the other hand, most of the current software and systems are based on the recognition of piano and human voice signals, and can only be converted into staves, which largely limits the use of other musical instruments or People who are not staves.

Contents of the invention

The purpose of the present invention is to solve the shortcomings in the prior art, and propose a musical instrument recognition and automatic notation system based on AI technology.

In order to solve the above problems, the present invention adopts the following technical solutions.

A musical instrument recognition and automatic notation system based on AI technology, including client, server and algorithm;

The client is used for user recording and display and editing of electronic scores;

The server end is used to collect user audio data and deploy algorithms;

The algorithm end is deployed on the server end, and the algorithm end includes an AI musical instrument recognition algorithm and an AI transcoding algorithm;

The AI musical instrument recognition algorithm is used to identify the performance musical instrument in the audio to be analyzed, and converts the audio to MIDI and the corresponding Musicxml file by the AI transspectral algorithm;

The server is also used to send MIDI and Musicxml files to the client for secondary editing by the user.

As a further solution of the present invention: the client adopts the form of developing apps on iOS and Android.

As a further solution of the present invention: the musical instruments that can be recognized by the AI musical instrument recognition algorithm include piano, violin, guitar, drum kit, and human voice humming.

As a further solution of the present invention: the algorithm end further includes a noise reduction algorithm, which is used to eliminate other noises in the audio that do not have musical features and some background noises.

As a further solution of the present invention: the server is also equipped with a model library, which stores pre-trained musical instrument audio models, and the model library can be expanded by user audio data collected by the server.

As a further solution of the present invention: the algorithm end further includes a musical note quantization correction algorithm, and the musical note quantization correction algorithm corrects the duration of the musical note in the MIDI file.

Compared with the prior art, the present invention has the advantages of:

1. Based on AI technology, this solution integrates instrument recognition, automatic notation and score editing. It not only meets the needs of users who use different musical instruments, but also provides convenience for users to record music inspiration and make and edit scores at any time.

2. The operation steps of this program are simple, just install the app and use it, it is compatible with iOS and Android systems, and does not require additional hardware devices.

3. This solution is compatible with the recognition of various musical instruments (including human voice humming), compatible with various score conversions, using AI technology, and based on a huge data set, the recognition accuracy is high.

Description of drawings

Fig. 1 is a schematic diagram of the system of the present invention;

Fig. 2 is a system flow chart of the present invention;

Fig. 3 is a diagram of specific implementation steps of the present invention.

Explanation of symbols in the figure:

1. Client; 2. Server; 3. Algorithm; 4. Model library.

detailed description

The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention; obviously, the described embodiments are only part of the embodiments of the present invention, not all embodiments, based on The embodiments of the present invention and all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Please refer to Figure 1-2, a musical instrument recognition and automatic notation system based on AI technology, including client 1, server 2 and algorithm 3, client 1 is used for user recording and electronic score display and editing, server Terminal 2 is used to collect user audio data and deploy algorithms. Algorithm terminal 3 is deployed on server terminal 2. Algorithm terminal 3 includes AI musical instrument recognition algorithm and AI transcoding algorithm. AI musical instrument recognition algorithm is used to identify musical instruments in the audio to be analyzed , and convert the audio into MIDI and corresponding Musicxml files through the AI spectrum conversion algorithm. The spectrum algorithm can also be combined into an AI model, which can simultaneously process musical instrument recognition and automatic spectrum conversion. This AI model can be transformed into a lighter model suitable for mobile devices through technical means. In this way, the algorithm terminal 3 Deployed on client 1.

Furthermore, the client 1 adopts the form of developing an app on iOS and Android, and the app can also be expanded and used in systems such as windows and macOS.

Further, the musical instruments that the AI musical instrument recognition algorithm can recognize include piano, violin, guitar, drum kit, and vocal humming.

Further, the algorithm terminal 3 also includes a noise reduction algorithm, which is used to eliminate other noises in the audio that do not have musical characteristics and some background noises. The noise reduction algorithm is not limited to the AI algorithm, and other noise reduction algorithms can also be used. An alternative to using acoustic front-end noise reduction processing.

Further, the server end 2 is also equipped with a model library 4, which stores pre-trained musical instrument audio models, and the model library 4 can be expanded by user audio data collected by the server end 2.

Further, the algorithm terminal 3 also includes a note quantization correction algorithm, which corrects the duration of the note in the MIDI file.

The specific implementation steps of this program are as follows, as shown in Figure 3:

Step 1: Apply for microphone permission to the device where client 1 is located;

Step 2: After obtaining the microphone permission, record the user's performance and collect audio data;

Step 3: The server side 2 receives the audio data sent by the client 1, and the algorithm side 3 performs audio preprocessing and noise reduction processing through the noise reduction algorithm to eliminate other noises and some background noises in the audio that do not have musical characteristics, so as to To improve the accuracy of subsequent musical instrument recognition and audio conversion;

Step 4: Input the noise-reduced audio data into the AI musical instrument recognition algorithm for musical instrument recognition. The recognized musical instruments include but are not limited to piano, violin, guitar, drum kit, vocal humming, etc., and there are various types of recognition models. This is related to the pre-trained musical instrument audio model stored in the model library 4, and the model library 4 can continuously expand the number of samples and sample types of musical instruments, and use technical methods such as improved model structure and parameters to improve the model. Iteratively optimizing the performance and identifying the type of musical instrument, and the model library 4 can also be expanded through the user audio data collected by the server end 2;

Step 5: According to the musical instrument recognition result in step 4, the algorithm terminal 3 selects the AI conversion algorithm of the corresponding musical instrument to convert the audio data into a MIDI file. In addition, the AI conversion algorithm can also be implemented by expanding the data set and improving the model structure. iterative optimization;

Step 6: Use the note quantization correction algorithm to correct the time value of the note in the MIDI file, and obtain the corrected MIDI file. The note quantization correction algorithm mainly converts each note in the MIDI file obtained in step 5 into a standard time value. notes, making the final electronic score more standardized;

Step 7: Convert MIDI files to Musicxml electronic score files;

Step 8: Client 1 receives and displays the MIDI and Musicxml files sent by server 2, and provides users with secondary editing. Among them, MIDI contains the basic music information of the music, and Musicxml is a general file of electronic score, mainly used For the storage and display of score information.

The above are only preferred specific embodiments of the present invention; however, the protection scope of the present invention is not limited thereto. Anyone familiar with the technical field within the technical scope disclosed in the present invention, according to the technical solution of the present invention and its improved concept to make equivalent replacements or changes shall fall within the scope of protection of the present invention.

Claims (6)

1. An AI technology based musical instrument identification and automatic score recording system, characterized in that: the system comprises a client (1), a server (2) and an algorithm end (3);

the client (1) is used for displaying and editing the sound recording of a user and the electronic music score;

the server (2) is used for collecting user audio data and deploying algorithms;

the algorithm end (3) is deployed on the server end (2), and the algorithm end (3) comprises an AI musical instrument identification algorithm and an AI rotating spectrum algorithm;

the AI musical instrument recognition algorithm is used for recognizing the playing musical instrument in the audio to be analyzed and converting the audio into MIDI and a corresponding Musicxml file through an AI spectral conversion algorithm;

and the server (2) is also used for sending the MIDI and Musicxml files to the client (1) for the user to edit for the second time.

2. An AI-technology-based instrument identification and automatic score recording system as claimed in claim 1, wherein: the client (1) is in the form of developing apps at the iOS end and the Android end.

3. An AI-technology-based instrument identification and automatic score recording system as claimed in claim 1, wherein: instruments that can be recognized by the AI musical instrument recognition algorithm include pianos, violins, guitars, drum sets, human humming.

4. An AI-technology-based instrument identification and automatic score recording system as claimed in claim 1, wherein: the algorithm end (3) also comprises a noise reduction algorithm which is used for eliminating other noise without music characteristics and some background noise in the audio.

5. An AI-technology-based instrument identification and automatic scoring system according to claim 1, characterized in that: the server side (2) is also provided with a model base (4), pre-trained instrument audio models are stored in the model base (4), and the model base (4) can be expanded through user audio data collected by the server side (2).

6. An AI-technology-based instrument identification and automatic scoring system according to claim 1, characterized in that: the algorithm end (3) also comprises a note quantification correction algorithm, and the note quantification correction algorithm corrects note durations in the MIDI file.

Images