CN113056784B - Voice information processing method and device, storage medium and electronic equipment - Google Patents

Abstract

This embodiment discloses a method for processing voice information, which includes collecting voice information, extracting target voice feature information and inputting it into a preset model to obtain target voiceprint parameters, obtaining the voice to be recognized, extracting the first voiceprint parameters of the voice to be recognized, matching the first voiceprint parameters with the target voiceprint parameters, obtaining identification information according to the matching result, and marking the identification information into the playback video. The processing accuracy of voice information is improved.

Description

Voice information processing method, device, storage medium and electronic device

Technical Field

The present invention relates to the field of speech processing, and in particular to a speech information processing method, device, storage medium and electronic equipment.

Background Art

With the development of information technology, the data used by users is no longer limited to text and pictures. Video has become the main medium for information transmission.

At present, in order to help users better understand the content of the video, adding subtitles to the video using speech synthesis technology has become a common choice. At the same time, adding subtitles to the video can also speed up the sharing of videos in different languages. However, the existing subtitles only contain the text content in the speech, which makes it difficult to determine the identity of the speaker in some videos based on the text content alone, thus affecting the user's understanding of the video content.

Summary of the invention

The embodiments of the present application provide a method, device, storage medium and electronic device for processing voice information, which can improve the accuracy of voice information processing.

In a first aspect, an embodiment of the present application provides a method for processing voice information, including:

Collecting voice information of a target user and extracting target voice feature information of the voice information;

Input the target speech feature information into the preset model to obtain the target voiceprint parameters;

Acquire the voice information to be recognized in the played video, and extract the first voiceprint parameter of the voice information to be recognized;

The first voiceprint parameter is matched with the target voiceprint parameter, identification information of the matched target voiceprint parameter is obtained according to the matching result, and the identification information is marked in the played video.

In a second aspect, an embodiment of the present application provides a voice information processing device, comprising:

A collection unit, used to collect voice information of a target user and extract target voice feature information of the voice information;

An input unit, used to input target speech feature information into a preset model to obtain target voiceprint parameters;

An acquisition unit, used for acquiring voice information to be recognized in the played video, and extracting a first voiceprint parameter of the voice information to be recognized;

A matching unit is used to match the first voiceprint parameter with a target voiceprint parameter, obtain identification information of the matched target voiceprint parameter according to the matching result, and mark the identification information into the played video.

In a third aspect, the storage medium provided in the embodiments of the present application stores a computer program thereon, and when the computer program runs on a computer, the computer executes a method for processing voice information as provided in any embodiment of the present application.

In a fourth aspect, an electronic device provided by an embodiment of the present application includes a processor and a memory, wherein the memory has a computer program, and the processor executes the steps by calling the computer program:

Collecting voice information of a target user and extracting target voice feature information of the voice information;

Input the target speech feature information into the preset model to obtain the target voiceprint parameters;

Acquire the voice information to be recognized in the played video, and extract the first voiceprint parameter of the voice information to be recognized;

The first voiceprint parameter is matched with the target voiceprint parameter, identification information of the matched target voiceprint parameter is obtained according to the matching result, and the identification information is marked in the played video.

BRIEF DESCRIPTION OF THE DRAWINGS

The technical solution and other beneficial effects of the present application will be made apparent by describing in detail the specific implementation methods of the present application in conjunction with the accompanying drawings.

FIG1 is a flow chart of a method for processing voice information provided in an embodiment of the present application.

FIG. 2 is another flowchart of the method for processing voice information provided in an embodiment of the present application.

FIG. 3 is a module diagram of a device for processing voice information provided in an embodiment of the present application.

FIG. 4 is a schematic diagram of another module of the voice information processing device provided in an embodiment of the present application.

FIG5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

FIG. 6 is another schematic diagram of the structure of the electronic device provided in an embodiment of the present application.

DETAILED DESCRIPTION

Please refer to the drawings, where the same component symbols represent the same components, and the principles of the present application are illustrated by implementing them in an appropriate computing environment. The following description is based on the illustrated specific embodiments of the present application, which should not be considered as limiting other specific embodiments of the present application that are not described in detail herein.

The term "module" used herein can be regarded as a software object executed on the computing system. The different components, modules, engines and services described herein can be regarded as implementation objects on the computing system. The devices and methods described herein are preferably implemented in software, but can also be implemented in hardware, all within the scope of protection of this application.

The embodiment of the present application provides a method for processing voice information, and the execution subject of the method for processing voice information can be a voice information processing device provided in the embodiment of the present application, or an electronic device integrating the voice information processing device, wherein the voice information processing device can be implemented in hardware or software. Among them, the electronic device can be a smart phone, a tablet computer, a PDA (Personal Digital Assistant), etc.

The following is a detailed analysis and explanation.

The embodiment of the present invention provides a method for processing video and voice, comprising:

Collecting the target user's voice information and extracting the target voice feature information of the voice information;

Input the target speech feature information into the preset model to obtain the target voiceprint parameters;

Acquire the voice information to be recognized in the played video, and extract the first voiceprint parameter of the voice information to be recognized;

The first voiceprint parameter is matched with the target voiceprint parameter, identification information of the matched target voiceprint parameter is obtained according to the matching result, and the identification information is marked in the played video.

In one embodiment, before the step of inputting the target voice feature information into the preset model, it can also include: training the background data through a preset algorithm to generate a preset model containing the common voice feature information corresponding to each target user, and the background data includes the voice information of each target user.

In one embodiment, the step of inputting the target voice feature information into a preset model to obtain target voiceprint parameters may include: inputting the target voice feature information into a preset model to obtain target difference feature information corresponding to the common voice feature information; determining a second voiceprint parameter based on the target difference feature information; and performing channel compensation on the second voiceprint parameter to obtain a corresponding target voiceprint parameter.

In one implementation, the step of performing channel compensation on the second voiceprint parameter may include: performing channel compensation on the second voiceprint parameter using a linear discriminant analysis method.

In one embodiment, the step of matching the first voiceprint parameter with the target voiceprint parameter and obtaining identification information of the matching target voiceprint parameter based on the matching result may include: matching the first voiceprint parameter with the target voiceprint parameter to generate a corresponding matching value; when the matching value is greater than a preset threshold, obtaining identification information of the matching target voiceprint parameter.

In one embodiment, the step of obtaining identification information of the matching target voiceprint parameters may include: sorting the matching values, obtaining the maximum matching value among the matching values greater than a preset threshold, and obtaining the matching target voiceprint parameters based on the maximum matching value; and obtaining the corresponding identification information based on the target voiceprint parameters.

In one embodiment, the step of marking the identification information in the playing video includes: inputting the voice information to be recognized into a voice recognition model to generate corresponding text information; combining the identification information with the text information to generate subtitle information corresponding to the voice information to be recognized; and marking the subtitle information in the playing video.

The present application embodiment provides a method for processing voice information, as shown in FIG1, which is a flow chart of the method for processing voice information provided by the present application embodiment. The method for processing voice information may include the following steps:

In step S101, voice information of a target user is collected, and target voice feature information of the voice information is extracted.

The target user may refer to the main speaker in the video. It is understandable that in interviews, movies, variety shows and other types of videos, the speakers are mostly concentrated in a limited number of roles. For example, in interview videos, the target users are the host and the interview guests, in movie or TV series videos, the target users are the actors with more important roles, or in the music video (MV) of an idol group, the target users are all the members of the idol group.

Among them, the voice information of the target user refers to the voice information after annotation, and the voice information of the target user contains the identification information of the target user. Further, the identification information may refer to the identity information of the target user, such as name, gender, age, title and other personal information. At the same time, the target voice feature information refers to the target voice voiceprint feature information. It can be understood that because the vocal organs used by people when speaking, such as tongue, teeth, larynx, lungs, and nasal cavity, vary greatly in size and shape, each person's voiceprint is different. Therefore, the voiceprint feature information is a unique feature of each person, just like everyone has his or her own unique fingerprint. Further, the target voice feature information includes the Mel-Frequency Cepstral Coefficient (MFCC) of the target voice information.

In some embodiments, in order to ensure the stability of the target speech feature information of the speech information, the speech information can be de-silenced and de-noised to generate processed speech information; the target speech feature information of the processed speech information is extracted and the target speech feature information is processed using feature mean variance normalization and feature bending.

In step S102, the target speech feature information is input into a preset model to obtain target voiceprint parameters.

Among them, the preset model can refer to the Universal Background Model (UBM model for short), and the target voiceprint parameters containing the identification information of the target user are obtained by inputting the target speech feature information, i.e., the target voiceprint features, into the UBM model, wherein different target voiceprint parameters correspond to the identification information of different target users, i.e., the target user of each voice message can be determined by the target voiceprint parameters of each voice message. At the same time, if the target voiceprint parameters output by different voice segments are the same, it can be determined that the speakers of different voice segments are the same user. In addition, the process of inputting the target speech feature information into the preset model to obtain the target voiceprint parameters is the process of establishing a voiceprint model according to the target voiceprint parameters. It can be understood that different target voiceprint parameters correspond to the voiceprint models of different target users.

In some embodiments, before the step of inputting the target voice feature information into the preset model, it can also include: training the background data through a preset algorithm to generate a preset model containing the common voice feature information corresponding to each target user, and the background data includes the voice information of each target user.

At this time, the step of inputting the target speech feature information into the preset model to obtain the target voiceprint parameters may include:

(1) inputting the target speech feature information into a preset model to obtain target difference feature information corresponding to the common speech feature information;

(2) determining a second voiceprint parameter based on the target difference feature information;

(3) Channel compensation is performed on the second voiceprint parameter to obtain a corresponding target voiceprint parameter.

Among them, the preset algorithm can be an EM algorithm, and the background data is trained by the EM algorithm, that is, the target voice feature information in the background data is trained to generate a universal background model, and the corresponding common voice feature information of each target user is obtained through the UBM model. At this time, the common voice feature information is the corresponding common voiceprint features obtained from all target users.

Furthermore, the target voice feature information is input into the UBM model, and the target difference feature information corresponding to the common voice feature information can be calculated based on the target voice feature information and the common voice feature information, and the second voiceprint parameter corresponding to each voice information is determined based on the target difference feature information, wherein the second voiceprint parameter includes the identification information of the target user. It is understandable that due to the uniqueness of the voiceprint, the target voice feature information of different target users is different, so the difference of each target voice feature information is amplified by obtaining the target difference feature information corresponding to the common voice feature information, so that compared with the target voice feature information, the target user information corresponding to each voice information can be more accurately determined based on the target difference feature information.

In addition, since the voice information in the background data and the voice information to be recognized are collected in different transmission channels, there are large channel differences, which leads to a decrease in recognition performance and affects the recognition rate. Therefore, channel compensation is performed on the second voiceprint parameters so that the second voiceprint parameters can minimize the intra-class differences and maximize the inter-class differences, so as to obtain low-dimensional target voiceprint parameters that are easy to distinguish.

In step S103, the voice information to be recognized in the played video is obtained, and the first voiceprint parameter of the voice information to be recognized is extracted.

Among them, the method of obtaining the voice information to be recognized in the playing video may include obtaining the voice information to be recognized of the video being played or the live video in real time, or obtaining the voice information to be recognized of the locally stored video. In addition, the method of extracting the first voiceprint parameter of the voice information to be recognized is the same as the process of extracting the target voiceprint parameter, that is, extracting the first voice feature information in the voice information to be recognized, and inputting the first voice feature information into the preset model, and calculating the first difference feature information corresponding to the common voice feature information and the first voice feature information corresponding to each target user in the preset model, and determining the first voiceprint parameter corresponding to the voice information to be recognized based on the first difference feature information.

In step S104, the first voiceprint parameter is matched with the target voiceprint parameter, identification information of the matched target voiceprint parameter is obtained according to the matching result, and the identification information is marked in the played video.

Among them, the first voiceprint parameter is matched with the target voiceprint parameter, and a corresponding matching result is obtained. According to the matching result, the target voiceprint parameter that matches the first voiceprint parameter can be determined. Since each target voiceprint parameter corresponds to the information of each target user, that is, the target voiceprint parameter contains the identification information of the corresponding target user, the information of the target user corresponding to the first voiceprint parameter can be confirmed according to the target voiceprint parameter.

In some implementations, the step of matching the first voiceprint parameter with the target voiceprint parameter and obtaining identification information of the matched target voiceprint parameter according to the matching result may include:

(1) matching the first voiceprint parameter with the target voiceprint parameter to generate a corresponding matching value;

(2) When the matching value is greater than a preset threshold, identification information of the matched target voiceprint parameters is obtained.

Among them, when the matching value is greater than the preset threshold, that is, the first voiceprint parameter and the matching target voiceprint parameter are extremely similar, it can be determined that the speaker of the first voiceprint parameter and the target user corresponding to the matching target voiceprint parameter are the same user, so the identification information of the matching target voiceprint parameter can be obtained as the identification information of the speech to be recognized corresponding to the first voiceprint parameter.

In addition, in some implementations, the step of marking the identification information into the played video may include:

(1.1) Inputting the speech information to be recognized into a speech recognition model to generate corresponding text information;

(2.1) combining the identification information with the text information to generate subtitle information corresponding to the voice information to be recognized;

(3.1) The subtitle information is marked in the playing video.

Among them, when the voice information to be recognized is input into the preset model to obtain identification information, the voice to be recognized is simultaneously input into the voice recognition model to obtain text information, and the time information corresponding to the text information and the identification information are recorded respectively, and the identification information and the text information are combined according to the time information to generate subtitle information of the voice information to be recognized, and the subtitle information is marked into the playback video according to the time information.

In some embodiments, the subtitle information can be marked in a preset position of the playing video in a preset combination, for example, the identification information and the subtitle information are combined and marked side by side at the bottom of the playing video screen. Alternatively, the identification information in the subtitle information marks the first area of the playing video in a special form, and the identification information marks the text information in a second area of the playing video in a different form. For example, the identification information is added to the upper end of the playing video screen with a font size smaller than the text information, and the text information is added to the lower end of the playing video screen.

As can be seen from the above, the present embodiment provides a method for processing voice information, which collects the voice information of the target user, extracts the target voice feature information of the voice information; inputs the target voice feature information into a preset model to obtain the target voiceprint parameter; obtains the voice information to be identified in the playback video, and extracts the first voiceprint parameter of the voice information to be identified; and matches the first voiceprint parameter with the target voiceprint parameter, obtains the identification information of the matching target voiceprint parameter according to the matching result, and identifies the identification information to the playback video. In this way, the identification information of the target user, such as identity information, can be identified to the playback video, which helps the user to better understand the content of the video when watching the video to ensure the user experience. At the same time, the identification information is automatically added to the playback video through voiceprint recognition technology, which greatly reduces manual operations and saves labor costs.

The method described in the above embodiment is further described in detail below with examples.

Please refer to FIG. 2 , which is another flowchart of the method for processing voice information provided in an embodiment of the present application.

Specifically, the method comprises the following steps:

In step S201, voice information of a target user is collected, and target voice feature information of the voice information is extracted.

The target user's voice information refers to the annotated voice information, so the target user's voice information contains the target user's identification information. Furthermore, the identification information may refer to the target user's identity information, such as name, gender, age, title, and other personal information. In addition, the target voice feature information refers to the voiceprint feature information of the target voice. Since the voiceprint feature information is a unique feature of each person, the user information corresponding to the voice information can be distinguished according to the voiceprint feature.

In some embodiments, in order to ensure the stability of the target speech feature information of the speech information, the speech information can be de-silenced and de-noised to generate processed speech information; the target speech feature information of the processed speech information is extracted and the target speech feature information is processed using feature mean variance normalization and feature bending.

In step S202, the background data is trained by a preset algorithm to generate a preset model containing common voice feature information corresponding to each target user, and the background data includes voice information of each target user.

Among them, the preset algorithm can be an EM algorithm, and the background data is trained by the EM algorithm, that is, the target voice feature information in the background data is trained to generate a universal background model, and the corresponding common voice feature information of each target user is obtained through the UBM model. At this time, the common voice feature information is the corresponding common voiceprint features obtained from all target users.

In step S203, the target speech feature information is input into a preset model to obtain target difference feature information corresponding to the common speech feature information.

The target feature information of each speech segment is input into a preset model. At this time, the target difference feature information can be obtained according to the target speech feature information corresponding to each speech segment and the common speech feature information of all target users obtained in step S202.

In step S204, a second voiceprint parameter is determined according to the target difference feature information;

The target difference feature information is transformed through the total factor space (Total Variability Space (TVS)-based model) to obtain the second voiceprint parameter. The total factor matrix of the total factor space can be estimated by the EM algorithm.

In step S205, a linear discriminant analysis method is used to perform channel compensation on the second voiceprint parameters to obtain corresponding target voiceprint parameters.

Among them, in order to reduce the problem of reduced recognition accuracy caused by channel differences, the linear discriminant analysis (LDA) method can be used for channel compensation. It should be noted that LDA uses label information to find the optimal projection direction so that the projected sample set has the smallest intra-class difference and the largest inter-class difference. When applied to voiceprint recognition, the vector of the voiceprint parameters of the same speaker represents a class. Minimizing the intra-class difference is to reduce the changes caused by the channel, and maximizing the inter-class difference is to increase the difference information between speakers. Therefore, the linear discriminant analysis method can be used to obtain low-dimensional target voiceprint parameters that are easy to distinguish.

In addition, at this time, the process of obtaining the target voiceprint parameters according to the target speech feature information is the process of establishing the corresponding voiceprint model. At this time, the voiceprint model is the i-vector voiceprint model corresponding to each target user.

In step S206, the voice information to be recognized in the played video is obtained, and the first voiceprint parameter of the voice information to be recognized is extracted.

Among them, the voice feature information of the voice information to be recognized is extracted, and the voice feature information is input into the voiceprint model in step S205, and the corresponding target difference feature information is obtained according to the common voice feature information in the UBM model; the first voice feature information is input into the preset model, and the first difference feature information corresponding to the common voice feature information is calculated according to the common voice feature information corresponding to each target user in the preset model and the first voice feature information, and the first voiceprint parameter corresponding to the voice information to be recognized is determined according to the first difference feature information; and the first voiceprint parameter is channel compensated to obtain the processed first voiceprint parameter. The step of extracting the first voiceprint parameter of the voice information to be recognized is the same as the step of extracting the target voiceprint parameter, which will not be repeated here.

In step S207, the first voiceprint parameter is matched with the target voiceprint parameter to generate a corresponding matching value.

The first voiceprint parameters are respectively matched with the target voiceprint parameters of the target user for similarity to generate corresponding matching values.

In step S208, when the matching value is greater than the preset threshold, the matching values are sorted to obtain the maximum matching value among the matching values greater than the preset threshold, and the matching target voiceprint parameters are obtained according to the maximum matching value.

Among them, when the matching value is greater than a preset threshold value such as 0.8, that is, at this time, the first voiceprint parameter successfully matches the corresponding target voiceprint parameter, and it can be determined that the target user corresponding to the first voiceprint parameter corresponding to the matching value and the target user corresponding to the target voiceprint parameter are the same user with a high probability. If there are multiple matching values greater than the preset threshold, the matching values greater than the preset threshold are sorted and the maximum matching value is obtained. At this time, it is determined that the target user corresponding to the first voiceprint parameter corresponding to the maximum matching value and the target user corresponding to the target voiceprint parameter are the same person with a high probability, and the target voiceprint parameter corresponding to the maximum matching value is obtained.

In addition, in some embodiments, when the matching values are all less than the preset threshold, it means that the first voiceprint parameters do not match the target voiceprint parameters, that is, the speaker corresponding to the voice to be recognized does not match the target user in the model, and the voiceprint model inputs an unmatched matching result.

In step S209, corresponding identification information is obtained according to the target voiceprint parameters.

Among them, since the target voiceprint parameters include the identification information of the target user, the corresponding identification information can be obtained according to the successfully matched target voiceprint parameters.

In step S210, the speech information to be recognized is input into a speech recognition model to generate corresponding text information.

When the voice information to be recognized is input into the voiceprint model to obtain identification information, the voice information to be recognized is simultaneously input into the voice recognition model to obtain text information.

In step S211, the identification information is combined with the text information to generate subtitle information corresponding to the speech information to be recognized.

When the identification information and the text information are obtained, the time information corresponding to the text information and the identification information are recorded respectively, and the identification information and the text information are combined according to the time information to generate the subtitle information of the voice information to be recognized.

In step S212, the subtitle information is marked in the played video.

The subtitle information is marked to a preset area in the playing video according to the time information to ensure that the subtitle information is synchronized with the voice information in the playing video.

As can be seen from the above, the present embodiment provides a method for processing voice information, which collects the voice information of the target user and extracts the target voice feature information of the voice information;

Input the target voice feature information into the preset model to obtain the target voiceprint parameters; obtain the voice information to be identified in the playback video, and extract the first voiceprint parameters of the voice information to be identified; match the first voiceprint parameters with the target voiceprint parameters, obtain the identification information of the matching target voiceprint parameters according to the matching results, and mark the identification information into the playback video. In this way, the identification information of the target user, such as identity information, can be marked into the playback video, helping users to better understand the content of the video when watching the video to ensure user experience. In addition, using voice recognition and voiceprint recognition technology to automatically add subtitle information to the video can greatly reduce manual annotation operations and save labor costs.

In order to better implement the voice information processing method provided in the embodiment of the present application, the embodiment of the present application also provides a device based on the voice information processing method. The meaning of the nouns is the same as that in the voice information processing method, and the specific implementation details can refer to the description in the method embodiment.

The embodiment of the present invention provides a video and voice processing device, including:

A collection unit, used to collect voice information of a target user and extract target voice feature information of the voice information;

An input unit, used to input target speech feature information into a preset model to obtain target voiceprint parameters;

An acquisition unit, used to acquire the voice information to be recognized in the played video, and extract the first voiceprint parameter of the voice information to be recognized;

The matching unit is used to match the first voiceprint parameter with the target voiceprint parameter, obtain identification information of the matched target voiceprint parameter according to the matching result, and mark the identification information into the played video.

In one embodiment, the device may further include: a training unit for training background data using a preset algorithm to generate a preset model containing common voice feature information corresponding to each target user, wherein the background data includes voice information of each target user.

In one embodiment, the input unit may include: an input subunit, used to input the target voice feature information into a preset model to obtain target difference feature information corresponding to the common voice feature information; a determination subunit, used to determine a second voiceprint parameter based on the target difference feature information; and a processing subunit, used to perform channel compensation on the second voiceprint parameter to obtain a corresponding target voiceprint parameter.

In one embodiment, the matching unit may include: a matching subunit, used to match the first voiceprint parameter with the target voiceprint parameter to generate a corresponding matching value; and an acquisition subunit, used to acquire identification information of the matched target voiceprint parameter when the matching value is greater than a preset threshold.

In one embodiment, the matching unit may also include: a generating subunit, used to input the voice information to be recognized into a voice recognition model to generate corresponding text information; a combining subunit, used to combine the identification information with the text information to generate subtitle information corresponding to the voice information to be recognized; and an identifying subunit, used to identify the subtitle information in the played video.

Please refer to Fig. 3, which is a module diagram of a voice information processing device provided in an embodiment of the present application. Specifically, the voice information processing device 300 includes: a collection unit 31, an input unit 32, an acquisition unit 33 and a matching unit 34.

The collecting unit 31 is used to collect the voice information of the target user and extract the target voice feature information of the voice information.

Among them, the voice information of the target user collected by the collection unit 31 refers to the labeled voice information, so the voice information of the target user contains the identification information of the target user. Furthermore, the identification information can refer to the identity information of the target user, such as name, gender, age, title and other personal information.

In addition, the target speech feature information extracted by the acquisition unit 31 refers to the target speech voiceprint feature information. It is understandable that because the vocal organs used by people when speaking, such as tongue, teeth, larynx, lungs, and nasal cavity, vary greatly in size and shape, each person's voiceprint is different. Therefore, the voiceprint feature information is a unique feature of each person, just like everyone has their own unique fingerprint. Further, the voiceprint feature information can be represented by Mel-Frequency Cepstral Coefficient (MFCC).

The input unit 32 is used to input the target speech feature information into a preset model to obtain target voiceprint parameters.

Among them, the input unit 32 inputs the target voice feature information of the target user's voice information into the preset model to obtain the adjusted voice feature information corresponding to the voice information. In addition, since the preset model contains the common voice feature information corresponding to each target user, the input unit 32 can determine the corresponding target voiceprint parameters based on the adjusted voice feature information and the common voice feature information.

The acquisition unit 33 is used to acquire the voice information to be recognized in the played video and extract the first voiceprint parameter of the voice information to be recognized.

The method of acquiring the voice information to be recognized in the playing video in the acquisition unit 33 may include acquiring the voice information to be recognized of the playing video or live video in real time, or acquiring the voice information to be recognized of the locally stored video. In addition, the step of extracting the first voiceprint parameter of the voice information to be recognized by the acquisition unit 33 is the same as the step of acquiring the target voiceprint parameter through the input unit 32.

The matching unit 34 is used to match the first voiceprint parameter with the target voiceprint parameter, obtain identification information of the matched target voiceprint parameter according to the matching result, and mark the identification information into the played video.

Among them, the matching unit 34 matches the first voiceprint parameter with the target voiceprint parameter and obtains a corresponding matching result. According to the matching result, the target voiceprint parameter that matches the first voiceprint parameter can be determined. Among them, since each target voiceprint parameter corresponds to the information of each target user, that is, the target voiceprint parameter contains the identification information of the corresponding target user, the information of the target user corresponding to the first voiceprint parameter can be confirmed according to the target voiceprint parameter.

Reference may also be made to FIG4 , which is another schematic diagram of a module of a voice information processing device provided in an embodiment of the present application. The voice information processing device 300 may also include: a training unit 35, which is used to train background data through a preset algorithm to generate a preset model containing common voice feature information corresponding to each target user, wherein the background data includes voice information of each target user.

Among them, the input unit 32 may include: an input subunit 321, used to input the target voice feature information into a preset model to obtain target difference feature information corresponding to the common voice feature information; a determination subunit 322, used to determine a second voiceprint parameter based on the target difference feature information; a processing subunit 323, used to perform channel compensation on the second voiceprint parameter to obtain a corresponding target voiceprint parameter.

The matching unit 34 may include: a matching subunit 341, used to match the first voiceprint parameter with the target voiceprint parameter to generate a corresponding matching value; an acquisition subunit 342, used to acquire identification information of the matched target voiceprint parameter when the matching value is greater than a preset threshold. A generation subunit 343, used to input the voice information to be recognized into a voice recognition model to generate corresponding text information; a combination subunit 344, used to combine the identification information with the text information to generate subtitle information corresponding to the voice information to be recognized; and an identification subunit 345, used to identify the subtitle information to the played video.

The embodiment of the present application further provides an electronic device. Referring to FIG5 , the electronic device 500 includes a processor 501 and a memory 502 . The processor 501 is electrically connected to the memory 502 .

The processor 500 is the control center of the electronic device 500. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device 500 and processes data by running or loading computer programs stored in the memory 502 and calling data stored in the memory 502, thereby monitoring the electronic device 500 as a whole.

The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the computer programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

In the embodiment of the present application, the processor 501 in the electronic device 500 will load instructions corresponding to the processes of one or more computer programs into the memory 502 according to the following steps, and the processor 501 will run the computer program stored in the memory 502, thereby realizing various functions, as follows:

Collecting the target user's voice information and extracting the target voice feature information of the voice information;

Input the target speech feature information into the preset model to obtain the target voiceprint parameters;

Acquire the voice information to be recognized in the played video, and extract the first voiceprint parameter of the voice information to be recognized;

The first voiceprint parameter is matched with the target voiceprint parameter, identification information of the matched target voiceprint parameter is obtained according to the matching result, and the identification information is marked in the playback video.

In some implementations, before inputting the target speech feature information into the preset model, the processor 501 may further specifically perform the following steps:

The background data is trained by a preset algorithm to generate a preset model containing common voice feature information corresponding to each target user, and the background data includes voice information of each target user.

In some implementations, when inputting the target speech feature information into a preset model to obtain the target voiceprint parameters, the processor 501 may specifically perform the following steps:

Inputting the target speech feature information into a preset model to obtain target difference feature information corresponding to the common speech feature information;

Determining a second voiceprint parameter according to the target difference characteristic information;

The second voiceprint parameters are channel compensated using a linear discriminant analysis method to obtain corresponding target voiceprint parameters.

In some implementations, when matching the first voiceprint parameter with the target voiceprint parameter and acquiring identification information of the matched target voiceprint parameter according to the matching result, the processor 501 may specifically perform the following steps:

Matching the first voiceprint parameter with the target voiceprint parameter to generate a corresponding matching value;

When the matching value is greater than a preset threshold, identification information of the matching target voiceprint parameters is obtained.

In some implementations, when obtaining identification information of the matching target voiceprint parameters, the processor 501 may specifically perform the following steps:

The matching values are sorted to obtain a maximum matching value among the matching values that are greater than a preset threshold, and a matching target voiceprint parameter is obtained according to the maximum matching value;

The corresponding identification information is obtained according to the target voiceprint parameter.

In some implementations, when the identification information is added to the played video, the processor 501 may specifically perform the following steps:

Inputting the speech information to be recognized into the speech recognition model to generate corresponding text information;

Combining the identification information with the text information to generate subtitle information corresponding to the voice information to be recognized;

Add subtitle information to the playing video.

Please refer to FIG6 , in some embodiments, the electronic device 500 may further include: a display 503, a radio frequency circuit 504, an audio circuit 505, and a power supply 506. The display 503, the radio frequency circuit 504, the audio circuit 505, and the power supply 506 are electrically connected to the processor 501, respectively.

The display 503 may be used to display information input by a user or information provided to a user and various graphical user interfaces, which may be composed of graphics, text, icons, videos, and any combination thereof. The display 503 may include a display panel, and in some embodiments, the display panel may be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

The radio frequency circuit 504 can be used to send and receive radio frequency signals, so as to establish wireless communication with a network device or other electronic devices through wireless communication, and to send and receive signals between the network device or other electronic devices.

The audio circuit 505 can be used to provide an audio interface between a user and an electronic device through a speaker and a microphone.

The power supply 506 can be used to supply power to various components of the electronic device 500. In some embodiments, the power supply 506 can be logically connected to the processor 501 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions.

Although not shown in FIG. 6 , the electronic device 500 may further include a camera, a Bluetooth module, etc., which will not be described in detail here.

An embodiment of the present application also provides a storage medium, which stores a computer program. When the computer program runs on a computer, the computer executes the voice information processing method in any of the above embodiments, such as: collecting voice information of a target user and extracting target voice feature information of the voice information; inputting the target voice feature information into a preset model to obtain a target voiceprint parameter; obtaining voice information to be recognized in a played video, and extracting a first voiceprint parameter of the voice information to be recognized; matching the first voiceprint parameter with the target voiceprint parameter, obtaining identification information of the matched target voiceprint parameter according to the matching result, and marking the identification information in the played video.

In the embodiment of the present application, the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

It should be noted that, for the voice information processing method of the embodiment of the present application, ordinary testers in the field can understand that all or part of the process of implementing the voice information processing method of the embodiment of the present application can be completed by controlling the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, such as stored in the memory of an electronic device, and executed by at least one processor in the electronic device, and the execution process may include the process of the embodiment of the voice information processing method. Among them, the storage medium can be a disk, an optical disk, a read-only memory, a random access memory, etc.

For the voice information processing device of the embodiment of the present application, each functional module can be integrated into a processing chip, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk.

The above is a detailed introduction to a voice information processing method, device, storage medium and electronic device provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims (14)

1. A processing method of voice information includes:

collecting voice information of a target user, and extracting target voice characteristic information of the voice information, wherein the target user is a main speaker in a playing video;

Training background data through a preset algorithm to generate a preset model containing corresponding common voice feature information of each target user, wherein the background data comprises the voice information of each target user, and the common voice feature information is corresponding common voice print features acquired according to all target users;

inputting target voice characteristic information into a preset model to obtain target voiceprint parameters, wherein the target voice characteristic information is input into the preset model to obtain target difference characteristic information corresponding to the common voice characteristic information, and determining second voiceprint parameters according to the target difference characteristic information; performing channel compensation on the second voiceprint parameters to obtain corresponding target voiceprint parameters;

Acquiring voice information to be recognized in a playing video, extracting first voiceprint parameters of the voice information to be recognized, wherein first voice characteristic information of the voice information to be recognized is extracted, the first voice characteristic information is input into the preset model, first difference characteristic information is calculated according to the common voice characteristic information and the first voice characteristic information of all target users, the first voiceprint parameters of the voice information to be recognized are determined according to the first difference characteristic information, and channel compensation is carried out on the first voiceprint parameters to obtain processed first voiceprint parameters;

And matching the first voiceprint parameters with the target voiceprint parameters, acquiring identification information of the matched target voiceprint parameters according to a matching result, and identifying the identification information into the playing video.

2. The method of claim 1, wherein the step of channel compensating the second voice-print parameter comprises:

And carrying out channel compensation on the second voice channel parameters by using a linear discriminant analysis method.

3. The method of claim 1, wherein the step of matching the first voiceprint parameter with a target voiceprint parameter and obtaining identification information of the matched target voiceprint parameter according to a matching result includes:

Matching the first voiceprint parameters with target voiceprint parameters to generate corresponding matching values;

and when the matching value is larger than a preset threshold value, acquiring the identification information of the matched target voiceprint parameters.

4. A method according to claim 3, wherein the step of obtaining identification information of the matching target voiceprint parameters comprises:

Sorting the matching values to obtain a maximum matching value in the matching values larger than a preset threshold value, and obtaining matched target voiceprint parameters according to the maximum matching value;

and acquiring the corresponding identification information according to the target voiceprint parameters.

5. The method of claim 1, wherein the step of identifying the identification information into the play video comprises:

inputting the voice information to be recognized into a voice recognition model to generate corresponding text information;

Combining the identification information with the text information to generate subtitle information corresponding to the voice information to be recognized;

and identifying the subtitle information into the playing video.

6. A processing apparatus for voice information, comprising:

the system comprises an acquisition unit, a video processing unit and a video processing unit, wherein the acquisition unit is used for acquiring voice information of a target user and extracting target voice characteristic information of the voice information, and the target user is a main speaker in a playing video;

The training unit is used for training the background data through a preset algorithm to generate a preset model containing the corresponding common voice characteristic information of each target user, wherein the background data comprises the voice information of each target user, and the common voice characteristic information is the corresponding common voice characteristic acquired according to all the target users;

the input unit is used for inputting target voice characteristic information into a preset model to obtain target voiceprint parameters, wherein the target voice characteristic information is input into the preset model to obtain target difference characteristic information corresponding to the common voice characteristic information, and the input unit comprises a determination subunit and a processing subunit, and the determination subunit is used for determining second voiceprint parameters according to the target difference characteristic information; the processing subunit is configured to perform channel compensation on the second voiceprint parameter to obtain a corresponding target voiceprint parameter;

The device comprises an acquisition unit, a channel compensation unit and a channel compensation unit, wherein the acquisition unit is used for acquiring voice information to be recognized in a play video, extracting first voiceprint parameters of the voice information to be recognized, extracting first voice characteristic information of the voice information to be recognized, inputting the first voice characteristic information into the preset model, calculating first difference characteristic information according to the common voice characteristic information and the first voice characteristic information of all target users, determining the first voiceprint parameters of the voice information to be recognized according to the first difference characteristic information, and performing channel compensation on the first voiceprint parameters to obtain processed first voiceprint parameters;

The matching unit is used for matching the first voiceprint parameters with the target voiceprint parameters, acquiring the identification information of the matched target voiceprint parameters according to a matching result, and identifying the identification information into the playing video.

7. The apparatus of claim 6, wherein the matching unit comprises:

The matching subunit is used for matching the first voiceprint parameters with the target voiceprint parameters to generate corresponding matching values;

and the acquisition subunit is used for acquiring the identification information of the matched target voiceprint parameters when the matching value is larger than a preset threshold value.

8. The apparatus of claim 6, wherein the matching unit further comprises:

the generation subunit is used for inputting the voice information to be recognized into a voice recognition model so as to generate corresponding text information;

a combining subunit, configured to combine the identification information with the text information, so as to generate subtitle information corresponding to the voice information to be identified;

And the identification subunit is used for identifying the subtitle information into the playing video.

9. A storage medium having a computer program stored thereon, wherein the computer program, when run on a computer, causes the computer to perform the method of processing speech information according to any of claims 1 to 5.

10. An electronic device comprising a processor and a memory, the memory having a computer program, wherein the processor is configured to perform the steps of:

collecting voice information of a target user, and extracting target voice characteristic information of the voice information, wherein the target user is a main speaker in a playing video;

Training background data through a preset algorithm to generate a preset model containing corresponding common voice feature information of each target user, wherein the background data comprises the voice information of each target user, and the common voice feature information is corresponding common voice print features acquired according to all target users;

inputting target voice characteristic information into a preset model to obtain target voiceprint parameters, wherein the target voice characteristic information is input into the preset model to obtain target difference characteristic information corresponding to the common voice characteristic information, and determining second voiceprint parameters according to the target difference characteristic information; performing channel compensation on the second voiceprint parameters to obtain corresponding target voiceprint parameters;

Acquiring voice information to be recognized in a playing video, extracting first voiceprint parameters of the voice information to be recognized, wherein first voice characteristic information of the voice information to be recognized is extracted, the first voice characteristic information is input into the preset model, first difference characteristic information is calculated according to the common voice characteristic information and the first voice characteristic information of all target users, the first voiceprint parameters of the voice information to be recognized are determined according to the first difference characteristic information, and channel compensation is carried out on the first voiceprint parameters to obtain processed first voiceprint parameters;

And matching the first voiceprint parameters with the target voiceprint parameters, acquiring identification information of the matched target voiceprint parameters according to a matching result, and identifying the identification information into the playing video.

11. The electronic device of claim 10, wherein the processor is configured to perform the steps by invoking the computer program:

And carrying out channel compensation on the second voice channel parameters by using a linear discriminant analysis method.

12. The electronic device of claim 10, wherein the processor is configured to perform the steps by invoking the computer program:

Matching the first voiceprint parameters with target voiceprint parameters to generate corresponding matching values;

and when the matching value is larger than a preset threshold value, acquiring the identification information of the matched target voiceprint parameters.

13. The electronic device of claim 12, the processor being operative to perform the steps of:

Sorting the matching values to obtain a maximum matching value in the matching values larger than a preset threshold value, and obtaining matched target voiceprint parameters according to the maximum matching value;

and acquiring the corresponding identification information according to the target voiceprint parameters.

14. The electronic device of claim 10, wherein the processor is configured to perform the steps by invoking the computer program:

inputting the voice information to be recognized into a voice recognition model to generate corresponding text information;

Combining the identification information with the text information to generate subtitle information corresponding to the voice information to be recognized;

and identifying the subtitle information into the playing video.