CN115831127B - Method, device and storage medium for constructing voiceprint reconstruction model based on speech conversion - Google Patents

Abstract

The embodiment of the specification provides a voiceprint reconstruction model construction method, a voiceprint reconstruction model construction device and a storage medium based on voice conversion, wherein the method comprises the following steps: constructing a first training set and a second training set based on voice conversion, wherein the first training set is used for indicating the original sound of a camouflage person, and the second training set is used for indicating the sound obtained by the camouflage person through voice conversion; constructing a first voice vector according to the first training set; constructing a second speech vector according to the second training set; and determining the feature vector of the original sound of the camouflage person through a vector decomposition method according to the first voice vector and the second voice vector. The technical scheme that this application provided is used for solving prior art and can not discern the problem of disguise person's acoustic.

Description

Method, device and storage medium for constructing voiceprint reconstruction model based on speech conversion

technical field

This document relates to the fields of artificial intelligence and computer security, and in particular to a method, device and storage medium for constructing a voiceprint reconstruction model based on voice conversion.

Background technique

Some users use voice conversion technology to pretend to be the corresponding identity by imitating the voice of others, which will bring hidden dangers to personal safety.

Existing speech transition detection techniques are usually based on artificial intelligence to identify whether a piece of audio is actually sampled or artificially generated.

However, the above scheme cannot identify the original voice of the pretender, so it is difficult to identify the identity information of the pretender. For personal safety, the above methods can only cure the symptoms, not the root cause, and cannot thoroughly eliminate the potential safety hazards that exist.

Contents of the invention

In view of the above analysis, this application aims to propose a method, device and storage medium for constructing a voiceprint reconstruction model based on speech conversion, which can identify the original voice of a faker who imitates the voice, so as to identify the identity information of the faker.

In the first aspect, one or more embodiments of this specification provide a method for constructing a voiceprint reconstruction model based on speech conversion, including:

Constructing a first training set and a second training set based on speech conversion, the first training set is used to indicate the original sound of the pretender, and the second training set is used to indicate the voice converted by the pretender;

Constructing a first speech vector according to the first training set;

Constructing a second speech vector according to the second training set;

According to the first speech vector and the second speech vector, a feature vector of the pretender's original voice is determined through a vector decomposition method.

Further, said constructing the first training set and the second training set based on speech conversion includes:

Acquire multiple original sound data and multiple speech conversion models;

constructing the first training set by using the plurality of original sound data;

Using any of the voice conversion models to convert the voiceprint corresponding to any one of the original voice data into another voiceprint corresponding to the original voice data, to obtain a training sample of the second training set.

Further, according to the first training set, construct the first speech vector, including:

determining a speech vector corresponding to each training sample in the first training set;

Determining an average vector of speech vectors corresponding to each of the training samples as the first speech vector.

Further, according to the second training set, construct a second speech vector, including:

determining a speech vector corresponding to each training sample in the second training set;

Determining an average vector of speech vectors corresponding to each of the training samples as the second speech vector.

Further, according to the first speech vector and the second speech vector, the feature vector of the original voice of the pretender is determined by vector decomposition method, including:

Create a coordinate system with the direction of the second speech vector as the coordinate axis;

Under the coordinate system, decomposing the first speech vector into orthogonal components and parallel components;

A feature vector of the original voice of the pretender is determined according to the parallel component and the first voice vector.

In the second aspect, one or more embodiments of this specification provide a voiceprint reconstruction model construction device based on speech conversion, including: a training set construction module, a vector construction module and a data processing module;

The training set construction module is used to construct a first training set and a second training set based on speech conversion, the first training set is used to indicate the original sound of the pretender, and the second training set is used to indicate that the pretender converts the get the sound;

The vector construction module is used to construct a first speech vector according to the first training set; construct a second speech vector according to the second training set;

The data processing module is used to determine the feature vector of the pretender's original voice through a vector decomposition method according to the first speech vector and the second speech vector.

Further, the training set construction module is used to collect a plurality of original sound data and a plurality of speech conversion models; use the plurality of original sound data to construct the first training set; use any of the speech conversion models to convert any of the speech conversion models to The voiceprint corresponding to the original sound data is converted into another voiceprint corresponding to the original voice data to obtain a training sample of the second training set.

Further, the vector construction module is used to determine the speech vector corresponding to each training sample in the first training set; determine the average vector of the speech vectors corresponding to each of the training samples as the first speech vector.

Further, the data processing module is used to create a coordinate system with the direction of the second speech vector as a coordinate axis; under the coordinate system, decompose the first speech vector into an orthogonal component and a parallel component; according to The parallel component and the first voice vector determine a feature vector of the pretender's original voice.

In a third aspect, one or more embodiments of this specification provide a storage medium, including:

It is used to store computer-executable instructions that, when executed, implement the method described in the first aspect.

Compared with the prior art, the present application can at least achieve the following technical effects:

1. Construct a training set based on the original voice of the pretender and the voice obtained by the pretender through speech conversion, providing a data basis for obtaining the original voice of the pretender.

2. The original voice of the pretender and the disguised voice are effectively separated by the vector decomposition method, so that the voiceprint reconstruction model can recognize the original voice of the pretender.

3. The calculation process of this scheme is simple and the applicability is strong. It can be applied to scenarios where the audio language is not included in the training set and the physical domain that imitates telephone recording.

Description of drawings

In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or prior art. Obviously, in the following description The accompanying drawings are only some embodiments described in this specification, and those skilled in the art can also obtain other drawings according to these drawings without any creative work.

Fig. 1 is a flow chart of a method for constructing a voiceprint reconstruction model based on speech conversion provided by one or more embodiments of this specification.

Detailed ways

In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will describe the technical solutions in one or more embodiments of this specification in conjunction with the drawings in one or more embodiments of this specification The technical solution is clearly and completely described, and obviously, the described embodiments are only a part of the embodiments in this specification, rather than all the embodiments. Based on one or more embodiments in this specification, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the scope of protection of this document.

Voice conversion technology based on machine learning can make the voice of the original speaker sound like the target speaker while keeping the content of the speech unchanged. It has been widely used in film dubbing, speech impediment assistance, and voice imitation. Applications. Therefore, based on the above-mentioned technology, the impostor transforms his own voice into that of other people, and the above-mentioned behavior will compromise personal security. And if it is only to identify whether the voice is imitated, the pretender can continue to imitate other people's voices in another way. Obviously, the existing technology cannot deter these pretenders very well, so it is difficult to achieve the effect of treating both symptoms and root causes.

In order to solve the above technical problems, the embodiment of the present application provides a method for constructing a voiceprint reconstruction model based on speech conversion, as shown in Figure 1, including the following steps:

Step 1. Construct a first training set and a second training set based on speech conversion.

In the embodiment of the present application, the first training set is used to indicate the original voice of the pretender, and the second training set is used to indicate the voice converted by the pretender. Step 1 specifically includes:

(1.1) Collect speaker voice data sets: collect several open source speaker voice data sets to form a training data set with a total number of n speakers , each speaker contains several pieces of audio data with different lengths to ensure the adequacy of the number of people trained and the training data. The set S is the first training set, and s ₁ , s ₂ ..., s _n are the original sound data.

(1.2) Preprocessing speaker voice data sets: adjust the naming format of all speaker voice data sets to a unified standard, and use FFmpeg tools to resample the audio data into wav format at the required sampling rate, which is convenient for subsequent voice conversion models training.

(1.3) Training voice conversion models: investigate a variety of existing mainstream voice conversion methods, use open source pre-training models or self-training to reproduce the expected effect, and obtain a variety of voice conversion models .

(1.4) Generating voice conversion data sets: the speakers in the voice data set are used as the original speaker S _source and the target speaker S _target in turn, and the voice conversion data sets are generated for each of the t trained voice conversion models . The speech conversion data set is the second training set.

Among them, for the problem that the samples of the second training set are difficult to obtain, steps 1.1, 1.3 and 1.4 are to ensure the number of samples of the second training set.

Specifically, any voice conversion model is used to convert a voiceprint corresponding to any original voice data into a voiceprint corresponding to another original voice data, so as to obtain a training sample of the second training set. For example, there are three original voice data A, B, and C in the first training sample, and two speech conversion models a, b are collected, then use model a to convert the voiceprint of A (original speaker S _source ) into B (target speech The voiceprint of the person S _target ), a sample of the second training set can be obtained. By analogy, 3*2*3 training samples can be obtained, and the number of samples in the second training set can be greatly increased through the above method.

Step 2. Construct a first speech vector according to the first training set.

In the embodiment of this application, when constructing the first speech vector, it is necessary to obtain the preliminary representation of the voiceprint first, specifically, (2.1.1) constructing the feature spectrum: input the target speaker evidence audio , the Mel filter bank converts the time domain information of the two into dimensions of The characteristic spectrum of

(2.1.2) Extract temporal features: construct a module and three - module, the former is used to extract time-domain features, and the latter is used to establish the correlation of global channels;

(2.1.3) Construct feature map: combine three - The output of the module is connected into a channel number of Feature maps of the target speaker evidence audio The eigenvectors of ;

Afterwards, according to the preliminary feature, determine the speech vector corresponding to each training sample in the first training set; determine the average vector of the speech vector corresponding to each training sample as the first speech vector.

Step 3. Construct a second speech vector according to the second training set.

In the embodiment of this application, when constructing the second speech vector, it is necessary to obtain the preliminary representation of the voiceprint first, specifically, (3.1.1) Constructing the characteristic spectrum: input speech conversion audio , the Mel filter bank converts the time domain information of the two into dimensions of The characteristic spectrum of

(3.1.2) Extract temporal features: construct a module and three - module, the former is used to extract time-domain features, and the latter is used to establish the correlation of global channels;

(3.1.3) Construct feature map: combine three - The output of the module is connected into a channel number of A feature map of the output speech-to-speech audio The eigenvectors are ;

Afterwards, according to the preliminary features, determine the speech vector corresponding to each training sample in the second training set; determine the average vector of the speech vectors corresponding to each training sample as the second speech vector.

Step 4. According to the first speech vector and the second speech vector, the feature vector of the pretender's original voice is determined by vector decomposition method.

In the embodiment of this application, the process of the vector decomposition method is:

Taking the direction of the second voice vector as the coordinate axis, create a coordinate system; under the coordinate system, decompose the first voice vector into orthogonal components and parallel components; determine the feature vector of the original voice of the pretender according to the parallel components and the first voice vector.

Specifically, the target speaker audio feature vector is calculated The mean vector of :remember for First Frame information, averaged on the time axis ;

(2.2.2) Calculate the normalized direction vector : , with a length of , where the constant to prevent errors;

(2.2.3) Decomposing the eigenvectors of speech-to-speech audio : Parallel component , the orthogonal component ,in is the size of the identity matrix;

In the embodiment of this application, the M⊥ part decomposed from M can weaken the influence of the voiceprint feature (N) of the target speaker and enhance the proportion of the voiceprint feature of the original speaker, thereby driving the restored voiceprint feature to Orientation optimization close to the voiceprint features of the original speaker.

(2.2.4) Through the residual structure connection, use the function TDNN to calculate the voice conversion audio length feature vector ,in and have the same dimensions, both ;

It should be noted that the actual optimization direction cannot be completely vertical. Therefore, if only the M⊥ part is used as the output, a considerable part of the effective information may be lost, resulting in an unsatisfactory recovery result. Therefore, step 2.2.4 is used to strengthen the original voiceprint of the pretender.

For the obtained eigenvectors of the pretender's original voice, optimize them through dimension normalization, data pooling and fixed voiceprint length.

Preferably, the AAM-Softmax loss function is used as the output layer, so that the recovered voiceprint information of the original speaker is sufficiently distinguishable from other suspects.

In summary, the model constructed by this application can identify the original voice of the pretender.

In order to illustrate the feasibility of the above-mentioned embodiment of the present application, the following takes the construction and training of the REVELIO voiceprint reconstruction model (whose task is to extract features from the input voice conversion audio and reconstruct the voiceprint of the original speaker) as an example to elaborate on the voice The converted original speaker's voiceprint reconstruction method includes the following steps:

(1) Data set construction, this step includes the following sub-steps:

(1.1) Collect speaker voice data sets: collection includes LibriSpeech , , The open source speaker speech data set including the total number of people is The speaker set of the training data set , each speaker contains several pieces of audio data with different lengths to ensure the adequacy of the number of people trained and the training data;

(1.2) Preprocessing speaker voice data sets: adjust the naming format of all speaker voice data sets to a unified standard, and use tool resamples the audio data into or at the desired sampling rate format, which is convenient for the training of the subsequent speech conversion model;

(1.3) Training voice conversion model: investigate a variety of existing mainstream voice conversion methods, use open source pre-training models or self-training to reproduce the expected effect, a total of Speech Transformation Model ; Among them, VQVC: voice conversion based on vectorquantization (VQ);

VQVC+: Speech conversion based on vector quantization and U-Net structure;

AGAIN: voice conversion based on activation function guidance (activation guidance) and adaptive IN (adaptive instance normalization);

BNE: Sequence to sequence (Seq2seq) speech conversion based on a bottle-neck feature extractor (BNE) and a mixture of logistic (MoL) attention mechanism.

(1.4) Generate a speech conversion dataset: the speakers in the speech dataset are used as the original speakers in turn and target speaker ,for Each trained voice conversion model generates a voice conversion data set .

(2) Model construction, this step includes the following sub-steps:

(2.1) Construction of feature extraction module: Obtain a preliminary representation of the voiceprint from the speech-to-speech audio, specifically including the following sub-steps:

(2.1.1) Constructing Feature Spectrum: Input Speech Conversion Audio and target speaker evidence audio , the Mel filter bank converts the time domain information of the two into dimensions of and The characteristic spectrum of

(2.1.2) Extract temporal features: construct a module (contains layer, layers and layer) and three - module (contains layer, - - layer, another layers and layer), the former is used to extract temporal features, and the latter is used to establish the correlation of global channels;

(2.1.3) Construct feature map: combine three - The outputs of the modules are connected, and their dimensions are both , the final output of the speech-converted audio and target speaker evidence audio The eigenvector of is and , and their dimensions are and ;

(2.2) Construction of the differential correction module: drive the voiceprint to be adjusted in the direction orthogonal to the target speaker's voiceprint to reduce the influence of the target speaker's information, specifically including the following sub-steps:

(2.2.1) Calculate the target speaker audio feature vector The mean vector of :remember for First Frame information, averaged on the time axis ;

(2.2.2) Calculate the normalized direction vector : , where the constant used to prevent Wrong, for example, take 1e-6;

(2.2.3) Decomposing the eigenvectors of speech-to-speech audio : Parallel component , the orthogonal component ,in is the size of the identity matrix;

(2.2.4) Connected by the residual structure, calculate ,in and have the same dimensions, both ;

(2.3) Dimension normalization module construction: Unify the input of different time lengths into an output of fixed dimension size, specifically including the following sub-steps:

(2.3.1) Data pooling: first put through a containing layer, layers and Layer module, and then the output is pooled through the attention mechanism layer to comprehensively represent all the data related to the number of channels and frame content. The dimension of the representation result is ;

(2.3.2) Fixed voiceprint length: further pass the above results through a layers and layer module, which ultimately makes the originally related to the voice conversion audio length normalized to a fixed length The voiceprint feature vector of ;

(2.4) Voiceprint enhancement module construction: use - The loss function is used as the output layer, so that the restored original speaker's voiceprint information is sufficiently distinguishable from other suspects, and the output dimension is .

(3) Model training, this step includes the following sub-steps:

(3.1) Initialize the model: pre-training - model, and use the parameters in its feature extractor as the initialization parameters of the feature extractor part of this model;

(3.2) Training model: Define the set of all possible original speaker identity labels included in the training process as , convert speech to audio and the corresponding target speaker evidence audio As model input, model output , thus training the model's ability to correctly classify the original speaker identity labels;

(3.3) Iterative convergence: Calculate the loss value according to the classification results output by the model, return the gradient and update the parameters, and iterate repeatedly until the loss value converges to a certain threshold to complete the training.

The embodiment of the present application provides a speech conversion-based voiceprint reconstruction model construction device, which is characterized in that it includes: a training set construction module, a vector construction module and a data processing module;

The training set construction module is used to construct a first training set and a second training set based on speech conversion, the first training set is used to indicate the original sound of the pretender, and the second training set is used to indicate that the pretender converts the get the sound;

The vector construction module is used to construct a first speech vector according to the first training set; construct a second speech vector according to the second training set;

The data processing module is used to determine the feature vector of the pretender's original voice through a vector decomposition method according to the first speech vector and the second speech vector.

In the embodiment of the present application, the training set construction module is used to collect a plurality of original sound data and a plurality of speech conversion models; use the plurality of original sound data to construct the first training set; use any of the speech conversion models Converting the voiceprint corresponding to any one of the original sound data into another voiceprint corresponding to the original voice data to obtain a training sample of the second training set.

In the embodiment of the present application, the vector construction module is used to determine the speech vector corresponding to each training sample in the first training set; determine the average vector of the speech vector corresponding to each of the training samples as the first speech vector .

In the embodiment of the present application, the data processing module is configured to use the direction of the second speech vector as a coordinate axis to create a coordinate system; under the coordinate system, decompose the first speech vector into orthogonal components and a parallel component; according to the parallel component and the first speech vector, determine the feature vector of the original voice of the pretender.

An embodiment of the present application provides a storage medium, including:

It is used to store computer-executable instructions, and the computer-executable instructions implement the methods described in the above-mentioned embodiments when executed.

The foregoing describes specific embodiments of this specification. Other implementations are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Multitasking and parallel processing are also possible or may be advantageous in certain embodiments.

In the 1930s, improvements to a technology could be clearly distinguished as improvements in hardware (for example, improvements in circuit structures such as diodes, transistors, switches, etc.) or improvements in software (improvement in method flow). However, with the development of technology, the improvement of many current method flows can be regarded as the direct improvement of the hardware circuit structure. Designers almost always get the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be realized by hardware physical modules. For example, a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA), is an integrated circuit whose logic functions are determined by programming the device by the user. It is programmed by the designer to "integrate" a digital system on a PLD, instead of asking a chip manufacturer to design and make a dedicated integrated circuit chip. Moreover, nowadays, instead of making integrated circuit chips by hand, this kind of programming is mostly realized by "logic compiler" software, which is similar to the software compiler used when writing programs. The original code must also be written in a specific programming language, which is called a hardware description language (Hardware Description Language, HDL), and HDL is not only one, but there are many, such as ABEL (Advanced Boolean Expression Language) , AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., currently the most commonly used is VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. It should also be clear to those skilled in the art that only a little logical programming of the method flow in the above-mentioned hardware description languages and programming into an integrated circuit can easily obtain a hardware circuit for realizing the logic method flow.

The controller may be implemented in any suitable way, for example, the controller may take the form of a microprocessor or a processor and a computer readable medium storing computer readable program code (such as software or firmware) executable by the (micro)processor , logic gates, switches, Application Specific Integrated Circuit (ASIC), programmable logic controllers, and embedded microcontrollers, examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that, in addition to realizing the controller in a purely computer-readable program code mode, it is entirely possible to make the controller use logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded The same function can be realized in the form of a microcontroller or the like. Therefore, such a controller can be regarded as a hardware component, and the devices included in it for realizing various functions can also be regarded as structures within the hardware component. Or even, means for realizing various functions can be regarded as a structure within both a software module realizing a method and a hardware component.

The systems, devices, modules, or units described in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementing device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or Combinations of any of these devices.

For the convenience of description, when describing the above devices, functions are divided into various units and described separately. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in one or more software and/or hardware.

Those skilled in the art should understand that one or more embodiments of this specification may be provided as a method, system or computer program product. Accordingly, one or more embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this description may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The specification is described with reference to flowcharts and/or block diagrams of methods, devices (systems), and computer program products according to embodiments of the specification. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and combinations of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a Means for realizing the functions specified in one or more steps of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in computer-readable media, in the form of random access memory (RAM) and/or nonvolatile memory such as read-only memory (ROM) or flash RAM. Memory is an example of computer readable media.

Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can be implemented by any method or technology for storage of information. Information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridge, tape magnetic disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include transitory computer readable media, such as modulated data signals and carrier waves.

It should also be noted that the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes Other elements not expressly listed, or elements inherent in the process, method, commodity, or apparatus are also included. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

One or more embodiments of this specification may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including storage devices.

Each embodiment in this specification is described in a progressive manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for relevant parts, refer to part of the description of the method embodiment.

The above description is only an embodiment of this document, and is not intended to limit this document. Various modifications and changes to this document will occur to those skilled in the art. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this document shall be included within the scope of the claims of this document.

Claims (8)

1. A voiceprint reconstruction model construction method based on voice conversion is characterized by comprising the following steps:

constructing a first training set and a second training set based on voice conversion, wherein the first training set is used for indicating the original sound of a camouflage person, and the second training set is used for indicating the sound obtained by the camouflage person through voice conversion;

constructing a first voice vector according to the first training set;

constructing a second speech vector according to the second training set;

according to the first voice vector and the second voice vector, determining a feature vector of the original sound of the camouflage person through a vector decomposition method;

the method for determining the feature vector of the original sound of the camouflage person through a vector decomposition method according to the first voice vector and the second voice vector comprises the following steps:

creating a coordinate system by taking the direction of the second voice vector as a coordinate axis;

decomposing the first speech vector into orthogonal components and parallel components in the coordinate system;

determining a feature vector of the camouflage sound according to the parallel component and the first voice vector;

specifically, the audio feature vector of the target speaker is calculated

Average vector of +.>

: record->

Is->

Is>

Frame information, average +.>

；

Calculating normalized direction vectors

：/>

Length of->

Wherein the constant->

For preventing errors;

decomposing feature vectors of speech converted audio

: parallel component->

Orthogonal component

Wherein->

Is of size +.>

Is a unit matrix of (a).

2. The method of claim 1, wherein the step of determining the position of the substrate comprises,

the constructing a first training set and a second training set based on the speech conversion includes:

collecting a plurality of acoustic data and a plurality of voice conversion models;

constructing the first training set using the plurality of acoustic data;

and converting the voiceprint corresponding to any one of the original voice data into the voiceprint corresponding to the other one of the original voice data by using any one of the voice conversion models to obtain one training sample of the second training set.

3. The method of claim 1, wherein the step of determining the position of the substrate comprises,

constructing a first speech vector according to the first training set, including:

determining a voice vector corresponding to each training sample in the first training set;

and determining an average vector of the voice vectors corresponding to the training samples as the first voice vector.

4. The method of claim 1, wherein the step of determining the position of the substrate comprises,

constructing a second speech vector according to the second training set, including:

determining a voice vector corresponding to each training sample in the second training set;

and determining an average vector of the voice vectors corresponding to the training samples as the second voice vector.

5. The voiceprint reconstruction model construction device based on voice conversion is characterized by comprising: the training set comprises a training set construction module, a vector construction module and a data processing module;

the training set construction module is used for constructing a first training set and a second training set based on voice conversion, wherein the first training set is used for indicating the original sound of a camouflage person, and the second training set is used for indicating the sound obtained by the camouflage person through voice conversion;

the vector construction module is used for constructing a first voice vector according to the first training set; constructing a second speech vector according to the second training set;

the data processing module is used for determining a characteristic vector of the original sound of the camouflage person through a vector decomposition method according to the first voice vector and the second voice vector;

the data processing module is used for creating a coordinate system by taking the direction of the second voice vector as a coordinate axis; decomposing the first speech vector into orthogonal components and parallel components in the coordinate system; determining a feature vector of the camouflage sound according to the parallel component and the first voice vector;

specifically, the audio feature vector of the target speaker is calculated

Average vector of +.>

: record->

Is->

Is>

Frame information, average +.>

；

Calculating normalized direction vectors

：/>

Length of->

Wherein the constant->

For preventing errors;

decomposing feature vectors of speech converted audio

: parallel component->

Orthogonal component

Wherein->

Is of size +.>

Is a unit matrix of (a).

6. The apparatus of claim 5, wherein the device comprises a plurality of sensors,

the training set construction module is used for collecting a plurality of acoustic data and a plurality of voice conversion models; constructing the first training set using the plurality of acoustic data; and converting the voiceprint corresponding to any one of the original voice data into the voiceprint corresponding to the other one of the original voice data by using any one of the voice conversion models to obtain one training sample of the second training set.

7. The apparatus of claim 5, wherein the device comprises a plurality of sensors,

the vector construction module is used for determining a voice vector corresponding to each training sample in the first training set; and determining an average vector of the voice vectors corresponding to the training samples as the first voice vector.

8. A storage medium, comprising:

for storing computer-executable instructions which, when executed, implement the method of any of claims 1-4.

Images