KR20240059075A - Machine learning-based online vocal analysis service system - Google Patents

Abstract

The present invention removes instruments and noise other than voice, extracts features for each frame by segmenting voice signals, compares the extracted features with the original song, outputs the comparison result, and performs analysis, singing, recording, and community activities according to user type. , provides classes, statistics, training, contests, experiences, sharing services, and sampling data including voice time series data, STFT signals, frame signals, pitch, intensity, vocalization, breathing, pronunciation, emotional features, vector signals, and analysis results. Verify data errors based on

Description

Machine learning-based online vocal analysis service system {MACHINE LEARNING-BASED ONLINE VOCAL ANALYSIS SERVICE SYSTEM}

The present invention relates to a machine learning-based online vocal analysis service system, and more specifically, to perform generalized statistical analysis through diversified singing analysis, provide professional training to users, and provide verification and promotion of training courses to course providers. As a means, it is about a machine learning-based online vocal analysis service system that efficiently improves students' skills and increases the operation reliability of the system.

Prior art related to the present invention includes guided voice reproduction, vocal therapy, timbre analysis, vocal evaluation, and speech analysis. Patent Document 1 A terminal device, guide voice reproduction method, and storage medium change the properties of the synthesized guide voice according to performance data. In addition, Patent Document 2's vocal practice system allows the person being corrected to visually recognize the degree of correction and whether correction has been made. In addition, in Patent Document 3, a user tone analysis device and tone analysis method, a user receives information about the degree of originality of his or her song tone from the timbres of existing singers. In addition, Patent Document 4's vocal evaluation system allows students preparing for vocal entrance exams and vocal aspirants to determine their vocal abilities based on more objective and absolute standards, and can compensate for deficiencies through standardized evaluation information from various angles. In addition, Patent Document 5, a method for analyzing a user's vocalization and a device for performing the same, determines a recommended agency that matches the user's vocalization and generates recommended agency data indicating the determined recommended agency.

However, the prior art has the problem of not being able to remove instruments and noises other than voices, segment voice signals to extract features for each frame, compare the extracted features with the original song, and output the comparison results.

Registered Patent Publication No. 10-0530916 Terminal device, guide voice reproduction method, and storage medium Registered Patent Publication No. 10-1471741 Vocal Practical System Registered Patent Publication No. 10-1813704 User tone analysis device and tone analysis method Registered Patent Publication No. 10-1917216 Vocal Evaluation System Publication of Patent No. 10-2022-0115157 Method for analyzing user's vocalization and device for performing the same

The present invention provides a machine learning-based online vocal analysis service system that removes instruments and noises other than voices, extracts features for each frame by segmenting voice signals, compares the extracted features with the original song, and outputs the comparison results. The purpose.

Another purpose of the present invention is to provide a machine learning-based online vocal analysis service system that provides analysis, singing, recording, community, classes, statistics, education, contests, experience, and sharing services depending on the user type.

In addition, the present invention is a machine learning-based online vocal system that verifies data errors based on sampling data including voice time series data, STFT signal, frame signal, pitch, intensity, vocalization, breathing, pronunciation, emotional characteristics, vector signal, and analysis results. Another purpose is to provide an analysis service system.

The machine learning-based online vocal analysis service system according to a preferred embodiment of the present invention is an analysis that removes noise from the absorbed sound, extracts features by segmenting them, compares the extracted features with a reference value, and outputs the comparison results. wealth (31); User service (32) that provides analysis, singing, recording, community, class, statistics, education, contest, experience, and sharing services according to user type 1 (40) and user type 2 (60); and receives input operations of the keyboard 21, mouse 22, and touch screen 23, and performs a machine learning-based online vocal analysis service using the microphone 24 and speaker 25 according to the input operations, It is characterized in that it includes an analysis unit 31 and a control unit 5 that controls the operation of the user service 32.

In addition, the analysis unit 31 includes a voice signal input step (S201), a musical instrument and noise removal step (S202) excluding voices, a subdividing voice signal into certain time units (S203), and signal analysis and characteristics for each frame. It is characterized by performing an extraction step (S204), a step of deriving detailed analysis contents through comparison with the original song (S205), and a step of completing the analysis and outputting the results (S206).

In addition, the step (S202) is characterized by separating the signal corresponding to the voice by combining a method of analyzing the voice signal by converting it to Short Time Fourier Transform (STFT) and a method of directly analyzing the voice signal.

In addition, in the step (S204), the voice signal from which the surrounding noise has been removed is segmented into frames to analyze pitch, intensity, breathing, vocalization, pronunciation, and emotion, and the pronunciation is characterized by converting Korean pronunciation into international pronunciation symbols. Do it as

In addition, in step S205, the original song is also analyzed frame by frame through step 204, matched to the beat of each frame, and two time series data and two time series data are generated using dynamic time wrapping (DTW). It is characterized by defining a distance function for each data, selecting the path with the minimum distance, comparing each frame for each element, and calculating the error for each element.

In addition, the user type 1 (40) includes an analysis report (41) that includes a detailed analysis of the user's recording by time and a total of seven singing elements; If you want to repeat practice, repeat audible part (42) to repeatedly practice only the section; A real-time analysis unit 43 that performs various exercises for each singing element and analyzes them in real time; a recording/playback unit 44 that provides a practice environment, records, and reproduces music; Skill improvement department (45) that visualizes and confirms singing skills; A comparison unit 46 that compares and analyzes the degree of similarity to the original song for each song element over time; Community Department (47), which questions and answers experts about singing-related questions; a class matching unit (48) that matches offline classes between experts and non-experts; Statistical Analysis Department (49), which conducts statistical analysis based on data on changes in singing skills of students taking the course; It is characterized by including an education recommendation unit 50 that recommends education suitable for the user.

In addition, the user type 2 (60) is a competition unit (61) that competes within the scope of acquaintances, regions, and affiliations; Contests (62) where users of the app compete against each other; Competitive contests (63) between users within a region or affiliation; Experience Department (64), which provides experience points and achievements; a chord analysis unit 65 that analyzes duets and chords; It is characterized by including a sharing unit 66 that shares the recording contents or analysis results.

In addition, the results of the repetition audible unit 42, the real-time analysis unit 43, and the chord analysis unit 65 are compared and analyzed between the recording and the original song using the DB 9, and an analysis report is output. It is characterized by further comprising (41).

The present invention removes instruments and noises other than voices, extracts features for each frame by segmenting voice signals, compares the extracted features with the original song, and outputs the comparison results to perform generalized statistical analysis through diversified song analysis. It can have the effect of:

In addition, the present invention provides professional education to users by providing analysis, singing, recording, community, classes, statistics, education, contests, experience, and sharing services depending on the user type, and provides verification and promotion of the educational course to the course provider. As a means, it can have the effect of efficiently improving students' skills.

In addition, the present invention improves the operational reliability of the system by verifying data errors based on sampling data including voice time series data, STFT signal, frame signal, pitch, intensity, vocalization, breathing, pronunciation, emotional characteristics, vector signal, and analysis results. Height can have an effect.

In addition, the analysis report 41 provides an analysis report by comparing and analyzing the recording and the original song using the results of the repetition audible unit 42, the real-time analysis unit 43, and the chord analysis unit 65, and the DB 9. It is characterized by output.

Figure 1 is a block diagram showing the configuration of the machine learning-based online vocal analysis service device of the present invention.
Figure 2 is a flowchart showing the analysis operation of the machine learning-based online vocal analysis service method in Figure 1.
Figure 3 is an example diagram showing the noise removal process of the analysis operation in Figure 2.
Figure 4 is an example diagram showing the feature extraction process of the analysis operation in Figure 2.
Figure 5 is an example diagram showing the comparison process of the analysis operation in Figure 2.
Figure 6 is a block diagram showing the user service of the machine learning-based online vocal analysis service device of Figure 1.
Figure 7 is an example of outputting an analysis report by comparing and analyzing the recording and the original song using DB 9, the results of the repetitive audible part, real-time analysis part, and chord analysis part of the machine learning-based online vocal analysis service device in Figure 1. .
Figure 8 is an exemplary diagram illustrating a configuration for verifying data errors for illustrating the present invention.
Figure 9 is an example diagram illustrating the hardware resources, operating system, operation of the core control unit, and system authentication configuration that grants authority to execute the control unit operation to explain the present invention.

Hereinafter, a machine learning-based online vocal analysis service system according to an embodiment of the present invention will be described in detail with reference to the drawings. Below, descriptions of previously known matters are omitted or simplified to clarify the gist of the present invention. The components included in the description of the present invention operate individually or in combination.

Figure 1 is a block diagram showing the configuration of the machine learning-based online vocal analysis service device of the present invention. Referring to Figure 1, the terminal 6 includes a keyboard 21, a mouse 22, a touch screen 23, and a microphone ( 24), speaker 25, control unit 5, analysis unit 31, and user service 32.

The terminal 6 performs a machine learning-based online vocal analysis service.

The keyboard 21 is one of the computer input devices that has a similar shape to the keyboard of a typewriter and is used to input numbers and letters into the system. When a key is pressed, the corresponding signal is input into the system.

The mouse 22 is an input device used to move a cursor or icon on the screen, and inputs commands including cursor movement, clicking, and scrolling into the system.

The touch screen 23 is an input device that receives the location when touched by the hand. Instead of using the keyboard 21, the touch screen 23 detects the location of the text displayed on the screen or touches a specific location and inputs it into the system. .

The microphone 24 absorbs sound, converts it into an electrical signal, and transmits it to a mixer or recorder, and the control unit 5 processes the electrical signal.

The speaker 25 converts the electrical signal into vibration of the diaphragm, generates a small density wave in the air, copies the sound wave, and generates an electrical signal in the control unit 5.

The control unit 5 receives operations from the keyboard 21, mouse 22, and touch screen 23, and provides a machine learning-based online vocal analysis service using the microphone 24 and speaker 25 according to the input operations. and controls the operations of the analysis unit 31 and the user service 32.

The analysis unit 31 removes noise from the absorbed sound, extracts features by segmenting them, compares the extracted features with a reference value, and outputs the comparison result.

The user service 32 provides analysis, singing, recording, community, classes, statistics, education, contests, experience, and sharing services depending on the user type.

Figure 2 is a flowchart showing the analysis operation of the machine learning-based online vocal analysis service method of Figure 1. Referring to Figure 2, the terminal 6 performs a voice signal input step (S201) and a musical instrument and noise removal step (S202) excluding voice. ), segmenting the voice signal into certain time units (S203), analyzing the signal for each frame and extracting features (S204), deriving detailed analysis contents through comparison with the original song (S205), completing the analysis and outputting the results ( Perform S206).

Figure 3 is an example diagram showing the instrument and noise removal process of the analysis operation in Figure 2.

Step (S202) combines a method of converting and analyzing the voice signal 91 in FIG. 3 into a STFT (Short Time Fourier Transform) (frequency signal) and a method of directly analyzing the voice signal 92 in FIG. 3 to obtain a signal corresponding to the voice. Separate. Step (S202) consists of an encoder (91, 95) and a decoder (92, 96) structure among the artificial intelligence structures. The encoders (91, 95) are divided into voice signals and frequency signals, and through each encoder (91, 95), the voice signals and frequency signals are converted into respective latent vectors. Different latent vectors through the two encoders are cross-analyzed and converted into latent vectors to extract only signals corresponding to voices (93). Afterwards, the potential vector corresponding to the voice signal and the frequency signal for cross-analyzed voice extraction are returned to the voice signal and frequency signal from which signals other than the voice have been removed through the separated decoder (92, 96). It becomes (97). The two decoded signals are cross-analyzed again and separated into the final voice signal, a voice signal (98).

Figure 4 is an example diagram showing the feature extraction process of the analysis operation in Figure 2.

In step S204, the voice signal from which the surrounding noise has been removed is segmented and analyzed on a frame-by-frame basis. Referring to FIG. 4, a latent vector 81 is extracted from the segmented input signal 85 using a CNN (Convolution Neural Network) structure 84. At this time, similar input signals must be converted into similar latent vectors (81). To correct this, each latent vector 81 is converted into a content vector 82 through a transformer 86. In addition, the masked language model (MLM) technique is applied to the latent vector 81 and the similarity between input signals is learned through a quantized representation 83. Afterwards, the input signal 85 is converted into a latent vector 81 through the learned machine learning model 86. Generally, the conversion between an input signal and a latent vector uses all voice signals as learning data, but in the present invention, only the vocal data corresponding to the song among the voice signals is used as learning data to extract a latent vector 81 from the vocal information, Vocal-related features are extracted with higher accuracy.

The latent vectors extracted through step S204 of FIG. 4 are divided into pitch, intensity, breathing, vocalization, pronunciation, and emotion through five additional machine learning models, and each element is analyzed. Compared to other singing elements, pronunciation is a unique linguistic characteristic of Korean, so an analysis method appropriate for it is applied. The present invention converts Korean pronunciation into international pronunciation symbols, and the applied pronunciation rules and exception processing order are exception handling - aspirated consonants (overlapping consonants) - consonant consonants - consonant simplification - nasal consonants - li re-syllabification - consonant consonants - palatal consonants - aspirated consonants. (Hole consonant) - Yeoneum - final consonant neutralization - Li is re-syllableized. Emotions are classified into six categories: ‘joy’, ‘anxiety’, ‘hurt’, ‘sadness’, ‘anger’, and ‘embarrassment’ based on data from the AI integrated platform “AI-hub” operated by the Korea Intelligence Information Society Promotion Agency. do.

Figure 5 is an example diagram showing the comparison process of the analysis operation in Figure 2.

In step S205, the audio signal is analyzed frame by frame and compared with the original song. In step S205, the original song is also analyzed for each frame through step 204, and is matched to the beat of each frame. The present invention uses dynamic time wrapping (DTW) of FIG. 5. Dynamic time wrapping defines two time series data and a distance function for each data, and selects the path with the minimum distance. Afterwards, in step S205, each element is compared frame by frame, and the method of calculating the error for each element is to apply the pitch and intensity model Mean Squared Error (MSE), which is a regression model, and the classification model, which is the Mean Squared Error (MSE). Apply the breathing, vocalization, pronunciation, and emotion model Cross Entropy, Error (CEE). Step (S205) calculates the error between the original song and the voice signal for each frame of beat, pitch, intensity, breathing, vocalization, pronunciation, and emotion through DTW, MSE, and CEE, and outputs the analysis result according to the moving average.

Figure 6 is a block diagram showing the user service of the machine learning-based online vocal analysis service device of Figure 1. Referring to Figure 6, the user service 32 includes user type 1 (40) and user type 2 (60). .

The user service 32 provides analysis, singing, recording, community, classes, statistics, education, contests, experience, and sharing services according to user type 1 (40) and user type 2 (60).

User type 1 (40) includes an analysis report (41) that includes a detailed analysis of the user's recordings by time and a total of 7 singing elements; If you want to repeat practice, repeat audible part (42) to repeatedly practice only the relevant section; A real-time analysis unit (43) that performs various exercises for each singing element and analyzes them in real time; a recording/playback unit 44 that provides a practice environment, records, and reproduces music; Skill improvement department (45) that visualizes and confirms singing skills; A comparison unit 46 that compares and analyzes the degree of similarity to the original song for each song element over time; Community Department (47), which questions and answers experts about singing-related questions; a class matching unit (48) that matches offline classes between experts and non-experts; Statistical Analysis Department (49), which conducts statistical analysis based on data on changes in singing skills of students taking the course; It includes an education recommendation unit 50 that recommends education suitable for the user.

User type 2 (60) is a competition department (61) that competes within the scope of acquaintances, region, and affiliation; Contests (62) where users of the app compete against each other; Competitive contests (63) between users within a region or affiliation; Experience Department (64), which provides experience points and achievements; a chord analysis unit 65 that analyzes duets and chords; It includes a sharing unit 66 that shares recorded content or analysis results.

Figure 7 is an example of outputting an analysis report by comparing and analyzing the recording and the original song using DB 9, the results of the repetitive audible part, real-time analysis part, and chord analysis part of the machine learning-based online vocal analysis service device in Figure 1. .

Referring to FIG. 7, the analysis report 41 uses the results of the repetition audible unit 42, the real-time analysis unit 43, and the chord analysis unit 65. The analysis report 41 receives data from the DB 9, achieves time synchronization with the original song, performs analysis asynchronously in multiple analysis units 31, and performs a process of synthesizing the data again. The analysis report 41 can dramatically reduce the time required to provide the analysis report service by using the asynchronous analysis unit 31.

Figure 8 is an example diagram illustrating a configuration for verifying data errors to explain the present invention.

Referring to FIG. 8, the control unit 5 stores sampling data, calculates a probability distribution by accumulating the number of occurrences for each size of the sampling data over a certain period of time, calculates a probability distribution for another period of time, and calculates two probabilities. By calculating the distribution difference, area difference, and difference distance accumulation (S101), sampling circuit abnormalities, data errors, and data changes can be predicted and responded to (S102). The control unit 5 notifies the user of the prediction result so that the user can respond, or the control unit 5 can respond to hardware failure, data error, or data change.

The sampling data includes voice time series data, STFT signal, frame signal, pitch, intensity, vocalization, breathing, pronunciation, emotional characteristics, vector signal, and analysis results, and the control unit 5 detects hardware failure and data error based on the sampling data. , responds to data changes.

The control unit 5 monitors the trend of each probability distribution for a certain period of time, predicts abnormalities in the probability distribution, responds to abnormal accidents, and reports normal operation to the outside if the data change in the probability distribution is constant. Additionally, the control unit 5 feeds back the data change rate to adjust a certain time interval. For example, if the data change rate is large, the certain time interval is increased, and if the data change rate is small, the certain time interval is shortened.

FIG. 9 is an exemplary diagram illustrating hardware resources and an operating system for explaining the present invention, operations of a control unit as a core, and a system authentication configuration that grants authority to execute control unit operations. Referring to FIG. 9, the present invention is a processor (1). ), memory (2), input/output device (3), operating system (4), and control unit (5).

The processor (1) is a CPU (Central Processing Unit), GPU (Graphic Processing Unit), FPGA (Field Programmable Gate Array), and NPU (Neural Processing Unit), and the operating system (4) and control unit (5) mounted on the memory (2) ) executes the execution code.

Memory (2) includes permanent mass storage devices such as random access memory (RAM), read only memory (ROM), disk drives, solid state drives (SSD), and flash memory. can do.

The input/output device 3 is an input device, such as a camera, keyboard, microphone, mouse, etc. including an audio sensor and/or image sensor, and an output device such as a display, speaker, haptic feedback device, etc. May include devices.

The operating system 4 may include Windows, Linux, IOS, virtual machines, web browsers, and interpreters, and supports tasks, threads, timer execution, scheduling, resource management, graphics, font processing, communication, etc.

The control unit 5 determines the state based on sensor, key, touch, and mouse input of the input/output device 3 with the support of the operating system 4 and performs operations according to the determined state. The control unit 5 performs job scheduling by timers and threads using parallel execution routines.

The control unit 5 determines the state using the sensor value of the input/output device 3 and performs an algorithm according to the determined state.

Referring to Figure 9, the system authentication configuration includes a terminal 6 including a control unit 5, and an authentication server 7.

The terminal 6 duplicates the data channel, receives the key value and biometric information of the terminal 6, and requests user authentication through the first data channel to the authentication server 7, and the terminal 6 receives the generated kit value. It is displayed on the display and transmitted to the authentication server (7).

The terminal 6 inputs the kit value displayed on the display of the terminal 6 and transmits it along with the user information to the authentication server 7 through the second data channel. The terminal 6 requests the authentication server 7 to authenticate the system mounted on the terminal 6 using the kit value and user information. The kit value of the terminal 6 can be generated from computer-specific information, such as the CPU manufacturing number and the Ethernet chip number. The terminal 6 can obtain user information through face recognition using a camera, voice recognition using a microphone, and handwriting recognition using a display, and use it for authentication.

The authentication server 7 receives the kit value from the terminal 6, receives the kit value and user information from the terminal 6 through a duplicated data channel, compares the kit value and the user information of the terminal 6, and By matching, authentication for use of the system of the terminal 6 is processed. The authentication server 7 transmits the authentication result to the terminal 6 to authorize the user's use of the system. Due to the dual data channels of the terminal 6, kit value loss can be minimized.

The authentication server 7 performs history analysis of user information and compares and determines consistency and changes in user information over time. In history analysis, if user information shows consistency, the user's use is permitted; if it shows changes, the user's use is not permitted. By allowing users to use the system when user information shows consistency, security is strengthened to prevent users with altered user information from accessing the system.

The authentication server 7 assigns weights to consistency, change, frequency, frequency trend, and high frequency, and blocks access by untrusted users using a combination of weights. For example, if the frequency threshold is exceeded, access by untrusted users can be blocked in proportion to the accumulated number of excesses, and users who attempt access over a long period of time can be authenticated. At this time, additional authentication is requested for untrusted users.

The terminal 6, which is a means of authenticating the use of the system, does not connect directly to the system, but forms a bypass route through the authentication server 7, so that the network that makes up the Internet network is composed of an internal network and an external network, and the IP address setting process There is an advantage that the authentication process using the terminal 6 is performed smoothly in this cumbersome time. At this time, the system is mounted on the terminal 6, the terminal 6 becomes an authentication terminal means, and the authentication server 7 becomes an authentication server means.

The cloud (12) is a modularization of the container (7) with the support of the operating system (4) that manages the processor (1), memory (2), input/output device (3), and communication unit (6), and the web (8) and DB ( 9), provides the services of the protocol 10 and library 11, and the control unit 5 executes a cloud application using the services of the container 7. A standard software package, called a container (7), bundles an application's code with associated configuration files, libraries (11), and dependencies needed to run the app.

The cloud 12 integrates control of multiple terminals 6, stores sensor values received from the terminal 6, monitors them over time, processes operation errors of the terminal 6, and sends error messages to other terminals. Notifies the terminal 6, and performs switching control on the terminal 6 that is the control target.

Neural network learning selects features from time series data collected from input devices such as temperature, altitude, fingerprints, various sensors, images, infrared cameras, and lidar, selects a model through algorithm selection, and repeats through the learning and performance verification process. Model selection is repeated through trial and error. After performance verification is completed, an artificial intelligence model is selected.

The control unit 5 performs a deep learning algorithm using a neural network to determine sensor values, uses training data to learn the neural network, and verifies the neural network performance with test data.

The present invention is not limited to the specific preferred embodiments described above, and various modifications can be made by anyone skilled in the art without departing from the gist of the invention as claimed in the claims. Of course, such changes are within the scope of the claims.

1: processor
2: memory
3: Input/output device
4: Operating system
5: Control unit
6: Terminal
7: Authentication server
8: web
9: DB
11: Library
12: Cloud
14: Container
16: Department of Communications
21: keyboard
22: mouse
23: Touch screen
24: microphone
25: speaker
31: analysis unit
311: noise removal unit
312: Subdivision unit
313: feature extraction unit
314: Comparison part
32: User service
40: User type 1
41: Analysis report
42: Repeat singing part
43: Real-time analysis unit
44: Recording/playback unit
45: Skill improvement department
46: Comparison part
47: Community Department
48: Class matching department
49: Statistical analysis department
50: Education recommendation department
60: User type 2
61: Competition Department
62: Contest
63: Competitive Contest
64: Experience points
65: Chord analysis section
66: Shared section

Claims (8)

An analysis unit 31 that removes noise from the absorbed sound, extracts features by segmenting them, compares the extracted features with a reference value, and outputs the comparison result;
User service (32) that provides analysis, singing, recording, community, class, statistics, education, contest, experience, and sharing services according to user type 1 (40) and user type 2 (60); and
It receives input operations from the keyboard 21, mouse 22, and touch screen 23, performs a machine learning-based online vocal analysis service using the microphone 24 and speaker 25 according to the input operations, and performs the analysis. A machine learning-based online vocal analysis service system comprising a unit 31 and a control unit 5 that controls the operation of the user service 32. According to paragraph 1,
The analysis unit 31,
Voice signal input step (S201), instrument and noise removal step except voice (S202), subdividing the voice signal into certain time units (S203), signal analysis and feature extraction step for each frame (S204), comparison with the original song A machine learning-based online vocal analysis service system, characterized by performing a detailed analysis content derivation step (S205), analysis completion and result output step (S206). According to paragraph 2,
In the step (S202),
A machine learning-based online vocal analysis service system that separates signals corresponding to voices by combining a method of analyzing voice signals by converting them to STFT (Short Time Fourier Transform) and a method of directly analyzing voice signals. According to paragraph 2,
The step (S204) is,
Machine learning-based online vocal analysis that analyzes pitch, intensity, breathing, vocalization, pronunciation, and emotion by segmenting the voice signal with surrounding noise removed by frame, and converts Korean pronunciation into international pronunciation symbols for pronunciation. service system. According to paragraph 2,
In the step (S205),
The original song is also analyzed frame by frame through step 204, corresponds to the beat of each frame, and uses dynamic time wrapping (DTW: Dynamic Time Wraping) to define two time series data and a distance function for each data, A machine learning-based online vocal analysis service system that selects the path with the minimum distance, compares each element frame by frame, and calculates the error for each element. According to paragraph 1,
The user type 1 (40) is,
Analysis report (41) containing detailed analysis of the user's recordings by time and by a total of 7 singing elements; If you want to repeat practice, repeat audible part (42) to repeatedly practice only the relevant section; A real-time analysis unit (43) that performs various exercises for each singing element and analyzes them in real time; a recording/playback unit 44 that provides a practice environment, records, and reproduces music; Skill improvement department (45) that visualizes and confirms singing skills; A comparison unit 46 that compares and analyzes the degree of similarity to the original song for each song element over time; Community Department (47), which questions and answers experts about singing-related questions; a class matching unit (48) that matches offline classes between experts and non-experts; Statistical Analysis Department (49), which conducts statistical analysis based on data on changes in singing skills of students taking the course; A machine learning-based online vocal analysis service system comprising an education recommendation unit 50 that recommends education suitable for the user. According to paragraph 1,
The user type 2 (60) is,
Competition department (61), where people compete within their circle of acquaintances, region, and affiliation; Contests (62) where users of the app compete against each other; Competitive contests (63) between users within a region or affiliation; Experience Department (64), which provides experience points and achievements; a chord analysis unit 65 that analyzes duets and chords; A machine learning-based online vocal analysis service system, characterized in that it includes a sharing unit 66 for sharing recording contents or analysis results. According to any one of paragraphs 6 or 7,
Analysis report 41 that compares and analyzes the recording and the original song using the DB 9 and outputs an analysis report based on the results of the repetition audible unit 42, the real-time analysis unit 43, and the chord analysis unit 65. ), a machine learning-based online vocal analysis service system, characterized in that it further includes.