FR3058253A1 - METHOD FOR PROCESSING AUDIO DATA FROM A VOICE EXCHANGE, SYSTEM AND CORRESPONDING COMPUTER PROGRAM. - Google Patents

Abstract

The invention relates to a method and a system for processing audio data, said audio data being derived from a voice exchange between at least two speakers. The method includes a processing step (P20) of the audio data which includes a speaker identification step (P202) which comprises a step of cutting said audio data, according to at least one pause pattern, outputting a set of segmented audio data. According to the invention the method comprises, for at least current audio data of said segmented audio data set, recognition of at least one spoken word and identification of the spoken word speaker according to voice reference identification data. and common voice identification data from the segmented audio data.

Description

© Publication no .: 3,058,253 (use only for reproduction orders)

©) National registration number: 16 60550 ® FRENCH REPUBLIC

NATIONAL INSTITUTE OF INDUSTRIAL PROPERTY

COURBEVOIE © Int Cl ⁸ : G 10 L 17/02 (2017.01), G 10 L 17/04, 15/07

A1 PATENT APPLICATION

©) Date of filing: 28.10.16. (© Applicant (s): VOXPASS Simplified joint-stock company (© Priority: trusted - FR. @ Inventor (s): PRIEM XAVIER, RIDE YVAN, GABAL AHMED and WANG XINGJUN. (43) Date of public availability of the request: 04.05.18 Bulletin 18/18. ©) List of documents cited in the report preliminary research: Refer to end of present booklet (© References to other national documents ©) Holder (s): VOXPASS Simplified joint-stock company. related: ©) Extension request (s): © Agent (s): CABINET PATRICE VIDON.

METHOD FOR PROCESSING AUDIO DATA FROM A VOICE EXCHANGE, CORRESPONDING SYSTEM AND COMPUTER PROGRAM.

FR 3,058,253 - A1 yV) The invention relates to a method and a system for processing audio data, said audio data being obtained from a voice exchange between at least two speakers. The method comprises a processing phase (P20) of the audio data which comprises a speaker identification step (P202) which comprises a step of cutting out said audio data, according to at least one pause model, delivering a set of segmented audio data. According to the invention, the method comprises, for at least current audio data of said set of segmented audio data, the recognition of at least one spoken word and the identification of the speaker who said the word according to reference voice identification data. and common voice identification data from the segmented audio data.

i

Method for processing audio data from a voice exchange, system and corresponding computer program.

1. Domain

The present technique relates to the processing of users' voice data. The present technique relates more particularly to the voice identification of users who wish to have access to a set of services by means of their voice fingerprints. More specifically still, a technique for accessing an online service is presented, comprising segregation and multi-voice segmentation.

2. Prior Art

The growing interaction of man with the digital world will require a reinforced use of voice recognition, in particular by extending to the relationship between humans and objects. The best known modes of interaction are keyboards, mice, screens and touch surfaces ... Voice is only used today as a tool for controlling simple services (IVR interactive voice response, from English for "Interactive Voice Response", selection of choices on voice automats having replaced the switchboard operators). Voice control also penetrates automotive uses, for non-vital functions such as destination selection of the navigation system or control of the sound volume of the car radio. Thus, the massive introduction of digital intelligence into everyday objects, from the refrigerator connected to the car, via “Smart City” type services, creates a crucial need for effective interfacing. and acceptable by any type of public with its environment.

It is not possible to converse in a complex way with an object or an automaton as many science fiction authors have imagined. Progress has been made, but both the recognition rate and the consideration of the complexity of the conversation still need to be improved. The digital world will only be truly accessible to everyone when everyone can interact naturally and intuitively with it. To do this, it is necessary to improve the rate of voice recognition on the one hand and to improve the capacity of a system using it to render a correct transcription of the voice exchange. The purpose of this is not to describe new recognition algorithms but rather to focus on the recognition system and how it works.

The problem of audiovisual conference processing has been investigated for many years, in particular for computerized conference processing comprising local participants and remote participants. We have more particularly described and presented spatialization systems of audio visual conferences, comprising a follow-up of the participants, in particular a follow-up of the participants speaking (speakers) with respect to the spectator participants, each participant being able to play the role of speaker and spectator. In addition, the problem of identification and recognition of speech has also been investigated for many years, essentially along two axes which are on the one hand the recognition and transcription of speech into text and on the other voice identification of speaker. Speech recognition and transcription techniques are essentially divided into two categories: on the one hand, "universal" techniques which are not aimed specifically at a user and which make it possible to recognize simple orders or instructions (these are IVR or SIRI ™ or Google Talk ™ voice assistants). The techniques implemented do not particularly aim to recognize a specific user, but are adapted to the recognition of any type of user, and this without requiring any particular training. Conversely, the “specific” techniques, implemented for example in office recognition software, are based on prior learning of the voice, diction and more generally of the manner of speaking of a user, and this in order to retranscribe as faithfully as possible, the texts dictated by the user.

Each of the techniques described above is associated with a given and independent problem. There is no multiplexed solution that can meet other needs. There is therefore a need to provide a flexible recognition solution, which can be adapted according to a given context, and more particularly according to a conference context.

3. Summary of the invention

The proposed technique does not have these drawbacks of the prior art. More particularly, the proposed technique relates to a method of processing audio data, said audio data being derived from a voice exchange between at least two speakers, said method comprising a phase of processing audio data comprising a step of identifying speaker which comprises a step of splitting said audio data, as a function of at least one pause model, delivering a set of segmented audio data. Said method comprises, for at least current audio data of said set of segmented audio data:

at least one step of voice recognition of said current audio data, implemented by means of at least one voice recognition engine, delivering at least one recognized word or an indicator of absence of recognition; when at least one voice recognition engine delivers at least one recognized word:

a step of searching, within a database, for at least one reference word corresponding to said at least one recognized word, delivering at least one reference voice identification datum;

a step of obtaining, from said current audio data, a current voice identification data;

a step of calculating at least one correspondence between said current voice identification data and said at least one reference voice identification data, delivering at least one correspondence score; and when a correspondence score exceeds a predetermined threshold, a step of allocating said current audio data to one of said at least two interlocutors.

According to a particular characteristic, the step of obtaining, from said current audio data, a current voice identification data comprises:

a step of applying, on said current audio data item, which takes the form of a one-dimensional audio signal x (t), a short-term Fourier transform (TFCT), delivering an X matrix, called an energy matrix , of size mxn, m frequency channels and n time frames and whose frequency scale is between O and 1 kHz;

a step of normalizing the values of said matrix X, delivering a standardized matrix which constitutes the current voice identification data.

According to a particular characteristic, the application of said short-term Fourier transform on said mono dimensional audio signal x (t) is carried out at a time step of 40 ms.

According to a particular characteristic, the values of the elements of the normalized matrix are between 0 and 100.

According to a particular characteristic, said at least one reference voice identification datum also takes the form of a standardized energy matrix, previously recorded in said database.

According to a particular characteristic, the step of calculating at least one correspondence, delivering the correspondence score, comprises a step of determining a Pearson correlation value of the current voice identification datum and of said at least one reference voice identification data.

According to a particular characteristic, the method comprises, prior to said processing phase, a phase of obtaining, for each of said at least two speakers, an individual voice print comprising said reference voice identification data.

According to a particular characteristic, said processing phase further comprises the following steps:

recording, in the form of at least one main audio stream of said verbal exchange;

when no audio stream exists for the current speaker, creation of an audio stream specific to the current speaker;

time stamping, in said at least one main audio stream and in said own audio stream, using data representative of said current speaker.

According to another aspect, the present technique also relates to a system designed for the implementation of the method previously described.

According to a preferred implementation, the different steps of the methods according to the proposed technique are implemented by one or more software or computer programs, comprising software instructions intended to be executed by a data processor of a relay module according to the technique proposed and being designed to control the execution of the different stages of the processes.

Consequently, the proposed technique also relates to a program, capable of being executed by a computer or by a data processor, this program comprising instructions for controlling the execution of the steps of a method as mentioned above.

This program can use any programming language, and be in the form of source code, object code, or intermediate code between source code and object code, such as in a partially compiled form, or in any other desirable form.

The proposed technique also relates to an information medium readable by a data processor, and comprising instructions of a program as mentioned above.

The information medium can be any entity or device capable of storing the program. For example, the support may include a storage means, such as a ROM, for example a CD ROM or a microelectronic circuit ROM, or else a magnetic recording means, for example a floppy disk or a disc. hard.

On the other hand, the information medium can be a transmissible medium such as an electrical or optical signal, which can be routed via an electrical or optical cable, by radio or by other means. The program according to the proposed technique can in particular be downloaded from a network of the Internet type.

Alternatively, the information medium can be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the process in question.

According to one embodiment, the proposed technique is implemented by means of software and / or hardware components. In this perspective, the term module can correspond in this document as well to a software component, as to a hardware component or to a set of hardware and software components.

A software component corresponds to one or more computer programs, one or more subroutines of a program, or more generally to any element of a program or of software capable of implementing a function or a set of functions, as described below for the module concerned. Such a software component is executed by a data processor of a physical entity (terminal, server, gateway, router, etc.) and is capable of accessing the hardware resources of this physical entity (memories, recording media, bus communication, electronic input / output cards, user interfaces, etc.).

In the same way, a hardware component corresponds to any element of a hardware assembly (or hardware) capable of implementing a function or a set of functions, according to what is described below for the module concerned. It can be a programmable hardware component or with an integrated processor for the execution of software, for example an integrated circuit, a smart card, a memory card, an electronic card for the execution of firmware ( firmware), etc.

Each component of the system described above naturally implements its own software modules.

The various embodiments mentioned above as well as the various characteristics which constitute them, can be combined with one another for the implementation of the proposed technique.

4. Figures

Other characteristics and advantages of the invention will appear more clearly on reading the following description of a preferred embodiment, given by way of simple illustrative and nonlimiting example, and of the appended drawings, among which:

Figure 1 shows the architecture of a system object of the present; Figure 2 shows the different phases of processing an audio stream; Figure 3 shows the identification of a speaker among two possible speakers; Figure 4 shows an example of an energy matrix.

5. Description

5.1. General elements.

As explained above, the general principle of the invention consists in implementing several speech processing and recognition engines, within a speech processing system accessible online and capable of meeting general needs. . The system object of the present is implemented for example using a distributed processing architecture (of “cloud” type) in which access to resources (computer processing) is carried out on demand. The system comprises on the one hand a set of voice recognition engines, these engines being configured to perform on the one hand voice identification tasks, making it possible to identify a speaker (the voice identification functionality is described below) ) and on the other hand voice recognition tasks (the voice recognition functionality is described below). The system object of the present also includes a speaker processing engine. This processing engine comprises on the one hand a component for managing registered speakers and a component for processing unregistered speakers. This speaker processing engine includes a set of communication interfaces (“API” type) with the voice recognition engines. The processing component for recorded speakers allows recording, updating of voice data (and in particular recording and updating of voice signatures) of speakers of the system. The unregistered speaker processing component automatically records and processes voice data from unknown speakers. FIG. 1 globally illustrates the architecture of the system which is the subject of the present.

The processing system (SystTt) comprises a speaker processing engine (MTrtUsrs) which comprises at least one connection interface (IC) to at least one media processing server (SrvTTMed). The media processing server (SrvTTMed) is in charge of the processing (recordings, multiplexing, equalization) of the audio signals it receives. The media processing server (SrvTTMed) is directly or indirectly, via a communication network (WNTWK) to one (or more) VoIP server (s) (VOIP) including suitable IP telephony software. The VoIP server is itself connected, via an optional gateway (GTWY) to sound production devices (microphone, conference octopus for conference room, smartphone, computer) which are implemented, by example during conference calls.

The speaker processing engine (MTrtUsrs) also includes access and processing interfaces (APIs) to at least one speech recognition engine (ASR # 1, ASR # 2, etc.), which each include access to a voice data processing database (DB1, DB2, etc.). The speaker processing engine (MTrtUsrs) also includes access to its own speaker database (DBU) which includes speaker profiles (registered or not) who have access to the system to which the speaker processing components registered and not registered (CTUe and CTUne) have access.

The speaker processing engine (MTrtUsrs) also includes an interface for accessing a speaker recording server (SRVEU), this server comprising components for implementing a speaker recording service. Such a service allows speakers (who so wish) to register in the system, as described below.

Within the framework of the present technique, there are two complementary tasks which are implemented by the system and more particularly by the speaker processing engine (MTrtUsrs) coupled with the voice recognition engines (ASR # 1, ASR # 2, etc.). .): firstly, it is the identification of the speaker, the purpose of which is to allow a distinction between the speakers for whom a voice stream is processed. Voiceprint identification assumes that certain physical characteristics of the vocal organs, which influence the quality of speech sound, are not exactly the same from one person to another. These characteristics are the size of the vocal cavities, throat, nose and mouth, and the shape of the joint muscles of the tongue, jaw, lips and soft palate. The voice recognition engine (s) are used to firstly generate (possibly on the fly), at least one voice print of a speaker. This is then used by the system during different tasks, and in particular when separating several speakers in the same conversation.

Secondly, it is voice recognition, the purpose of which is to allow recognition of the words spoken by speakers, in order for example to make a written transcription of their speeches. To perform such recognition, the techniques implemented are conventional, with the exception of a selection algorithm adapted for the needs of the present technique, an algorithm which, as presented below, not only makes it possible to use recognition results from the recognition engine (s), but also other parameters.

The speaker identification database (DBU) notably includes information relating to the different speakers who are registered in the system; more particularly, this database comprises at least one unique voice print, originating from each speaker, which makes it possible to identify the speaker, among all the registered speakers.

This unique voiceprint is generated either at the end of a (voluntary) speaker registration phase with the system, or when the system is used by an unregistered speaker (for example during a conference with several speakers, some or all of whom are not registered) or both. The purpose of this database is in particular to allow differentiation of the speakers in order to improve the voice recognition of each of these speakers. The database also has the purpose of processing, grouping and classifying the different audio streams allocated to the user. Indeed, we do not just transcribe in the form of text, the words spoken by the speakers: these words, dialogs exchanges are also kept in order to be able to be reused later, for example to perform a second voice recognition , which could be of better quality, or for statistical analysis and comparison.

Thus, the unique voiceprint that is registered in the system for a given speaker can be obtained in at least two different ways.

A first way to obtain a unique voiceprint is to register manually with the system. This manual recording includes, for example, the creation of a speaker (user) account in the system, via a web service. The creation of this account is accompanied by the provision by the speaker of a set of voice data. This voice data set can either be obtained directly by reading one or more predetermined sentences at the time of recording. In order to allow simple and time-saving recording, the number of sentences to be read is deliberately reduced (a maximum of ten sentences). Another possibility, during manual recording, is to download a sound file that the speaker has prerecorded by reading a certain number of sentences (as in direct online recording). This type of service recording makes it possible, on the one hand, as indicated above, to generate a unique fingerprint for the speaker, this fingerprint being subsequently used to identify the speaker; recording also makes it possible, on the other hand, to train one or more recognition systems, to recognize the speaker's voice.

In a specific embodiment, the online service has a plurality of voice recognition engines (from different publishers). The voice stream (live or prerecorded) which is supplied to the system during the recording of the speaker is automatically transmitted to these voice recognition engines, which independently perform ίο learning of the speaker's voice, based on sentences read by the speaker. This embodiment, in which several speech recognition engines are used, is particularly suitable insofar as it makes it possible to benefit from several different recognition technologies. The modules used are commercial modules, each providing an open interface (API) for exchanging data with the system.

5.2. Voice recognition and recognition

A key element of the system of the invention resides in the ability of the system to separate the speakers and then to recognize the words and sentences spoken by these speakers. The identification of the speaker is implemented by means of one or more voiceprints of this speaker, which have been pre-recorded (in a prior phase). Voiceprints are obtained by known techniques and do not require further description.

Speech recognition is implemented via one or more speech recognition engines which, according to an input stream, deliver recognition results to which the speaker processing engine (MTrtUsrs) accesses. Thus, for example, for the recognition of a word (or of a sentence) spoken by a speaker, each recognition module is called via its interface (API). Each provides, as output, the recognized word (or phrase), as well as a possible probability of success of the recognition, which is then used by the system to determine and select the most probable result, depending in particular other parameters and obtain a veracity score which, when it exceeds a determined threshold, is considered to be the result of the transcription.

The selection parameters are for example:

the number of occurrences of a word or a term during the conversation, the exchange: if a term has already been spoken by the speaker, the probability of occurrence of this term is increased for the rest of the conversation ;

the number of occurrences of a word or a term during the conversation, the exchange: if a term has already been spoken by another speaker, the probability of occurrence of this term is increased for the rest conversation;

the number of occurrences of a word or term during an archived conversation or exchange: if a term has already been spoken by the speaker, in the past, the probability of occurrence of this term is increased for the rest of the conversation; the number of recognition modules that deliver an identical result (when several modules are used);

In other words, rather than relying solely on the results provided by the speech recognition engine (s) used by the system, the speaker processing engine uses past data, whether it is past data from the current conversation or exchange or past data from previous conversations, exchange or recordings, which have been retained and archived, in the speaker database. Referring to FIG. 2, a method for processing voice data implemented by the present system is presented. Such a method includes:

a phase for obtaining (P10), for each speaker, an individual voice print; the manner of obtaining this imprint is described infra and supra in examples of implementation;

a processing phase (P20) of at least one audio stream resulting from a verbal exchange between the speakers comprising at least one iteration of the following steps:

recording (P201), in the form of at least one main audio stream of said verbal exchange;

identification (P202), using the individual voice print, of a current speaker, among the plurality of speakers;

when no audio stream exists for the current speaker (P203), creation (P204) of an audio stream specific to the current speaker;

time stamping (P205), in said at least one main audio stream and in said own audio stream, using data representative of said current speaker; this data representative of the current speaker can advantageously be a hash of the voiceprint of the current speaker;

This process allows, for a given exchange or conversation, to record, separately, the different streams from different speakers. They are marked in time, in order to be able to segment the exchanges and allow individualization of speaking during the exchanges. An embodiment of the identification of the speaker is presented below.

Using audio streams, the recognition phase is implemented. Depending on the embodiments, it is implemented in real time or deferred. This recognition phase includes at least one iteration of the following steps:

transmission, to at least one voice recognition engine, of at least a portion of its own audio stream (to a current speaker);

obtaining, from said at least one voice recognition engine, at least one recognition result, taking the form of at least one recognized word;

determining a truth score of said at least one recognized word as a function of at least one previous recognition result;

when said veracity score exceeds a determined threshold, adding said recognized word to a recognition data structure.

The determination of the veracity score is implemented according to the different embodiments and in particular the presence (or not) of several recognition engines and the presence (or not) of a probability of recognition (provided by the ( s) recognition engine (s) and the availability of previous transcription results.

5.3. Voice identification of a speaker

The voice identification of a speaker is described below, implemented by the speaker processing engine (MTrtUsrs). The vocal identification of a speaker is implemented through a plurality of energy matrix. These energy matrices are recorded in the speaker identification database (DBU), either by being directly associated with a user who is registered in the system, or by being associated with an unregistered user (temporary user) , which is for example identified during a meeting or during a verbal exchange. As explained above, during the implementation of the system, a sort is made between registered users and non-registered users (unknown to the system). The energy matrices are created on the one hand to record the user and on the other hand to recognize the user when he speaks.

Prior registration of the speaker

The principle of user identification consists in creating, from the speech flow of a speaker, one or more energy matrices associated with this speaker. Suppose, by way of illustration, we have several samples or audio files containing the word la, each time pronounced by the same user (for example a user being recorded). These files or samples are in the time domain. For the analysis (and classification) of these signals, a short-term Fourier transform (TFCT) is used. The advantage of TFCT is that it can represent frequency domain information in the time domain.

The choice of the time step of the TFCT is complex. If the time step is too small, the number of samples is not enough, therefore, the results of the Fourier transform are not reliable. If the time step is too large, certain information is missing in the time domain. For this reason, the time step is chosen as 40 ms.

Using the TFCT, the energy matrix is calculated as shown in Figure 3. In this illustrative representation, we have ten times the word 'la', that is to say that the user has pronounced the word 'la' ten times with a more or less different intonation. For each of these matrices, the abscissa represents time, unity is the second. The ordinate represents the frequency, the unit is Hertz (Hz). The time step is 40 ms. The frequency step is 50 Hz. As most of the energy is distributed in the frequencies between 0 and 1000 Hz, the frequency band is between 0 and 1000 Hz.

After the analysis of the TFCT, for each word, we obtain an energy matrix. We have chosen as an energy matrix a matrix with 24 frequency band lines and 8 columns corresponding to time. To compare these matrices, we normalize these energy matrices. The maximum value is fixed at 100.

In the example in Figure 3, there are ten energy matrices. Each matrix has 24 rows for the frequency step and 8 columns for the time step. These matrices are recorded in the speaker identification database (DBU), in connection with the given user (whether registered or not). Each matrix can be saved with a relatively small number of data: one byte is enough for each value of the matrix (value normalized to 100), i.e. a maximum of 24x8 = 192 bytes per matrix (without optimization), to which it is possible to apply a compression algorithm.

In a particular embodiment of the present technique, the voiceprint of a speaker is constructed from a plurality of energy matrixes, relating to one or more words spoken by the speaker. Other elements than the energy matrices can also be integrated and / or used to generate the voice print of a speaker.

Speaker recognition

It is assumed that there is, in the in the speaker identification database (DBU), a plurality of energy matrices associated with a user, and that this user is being discussed and recorded by the system. To recognize this user, the system bases it on matrices from words spoken by the user. In a first phase, it is assumed that a certain number of words have been recognized by the system and that an audio sample is available for these recognized words. By way of illustration, it is assumed that one of the recognized words is the word "la". The system already has energy matrices from the pronunciation of this word "la" (as explained above).

In a second phase, the system therefore searches for the matrices corresponding to the word "la" which were previously generated. This search is performed for all “potential” users (ie a list of possible users, taking into account the circumstances of the registration): if for example the registration concerns four users registered in the system, Matrix searches are performed on these four users in order to shorten processing times.

On the basis of the energy matrices obtained at the end of the research, the identification of the user is implemented using a Pearson correlation to compare these energy matrices. More precisely, we use the Pearson correlation coefficient to compare the energy matrices. This coefficient is calculated as shown in the following equation:

in which :

X {xl, ... xn} is a matrix present in the database, which we are going to compare;

Y {yl, ... yn} is the matrix to compare, i.e. the current matrix:

N is the number of values in the matrices.

The value "r" is between 0 and 1. If / r / <0.4 the two matrices have a weak correlation. When 0.4 = </ r / <0.7 the two matrices have an average correlation. When

0.7 </ r / <= 1, the two matrices have a much stronger correlation: the latter case is considered in the context of this.

Thus by using the value of the Pearson correlation coefficient, with a predetermined threshold equal to 0.7 (or close to 0.7), one decides on the similarity of two matrices. The identification of the speaker is carried out on the correlation calculations whose result is the highest (among the results above the predetermined threshold).

We present, in relation to FIG. 4, the method implemented for carrying out speaker identification. The method implemented comprises a step of cutting (not shown) the audio data, according to at least one pause model, delivering a set of segmented audio data. The method is implemented on the basis of the segmented data and it comprises, for at least current audio data of said set of segmented audio data:

at least one voice recognition step (10) of said current audio data, implemented by means of at least one voice recognition engine, delivering at least one recognized word or an indicator of lack of recognition; when at least one voice recognition engine delivers at least one recognized word (20):

a search step (201), within a database, of at least one reference word corresponding to said at least one recognized word, delivering at least one reference voice identification datum;

a step of obtaining (202), from said current audio data, a current voice identification data;

a step of calculating (203) at least one correspondence between said current voice identification data and said at least one reference voice identification data, delivering at least one correspondence score; and when a correspondence score exceeds a predetermined threshold (205), a step of allocation (206) of said current audio data to one of said at least two interlocutors.

Thus, this method makes it possible to distinguish between the different speakers who intervene during the voice exchange. As indicated above, this identification method can advantageously be combined with other identification methods (in particular sound spatialization), in particular in order to produce more convincing results and / or to obtain these results more fast.

5.4. Description of a use case

We describe, in this use case, an implementation of the system for recognizing and transcribing the minutes of a conference type telephone conference in which a plurality of speakers participate, some of them being located in the same room, around a conference octopus-type device (for example two speakers) and a third participant remotely via a telephone handset, a computer, etc. A conference is implemented and processed at least indirectly by the system. A conference begins, for the system, with a phase of recording a voiceprint of each of the participants in the conference. This phase includes several stages, among which:

a step of transmitting, to each speaker participating in the conference, a request for the speaking, by this speaker, of his name and / or his first name; for example, the request takes the form of a message such as "you are entering a conference, please say your names and first names":

when the speaker participates in the conference by telephone, the fingerprint recording phase is implemented at this telephone line;

when the speaker takes part in the conference from a computer, the recording phase is implemented through the latter, either locally, on the computer, or on the system, by transmitting the audio stream resulting from the saying;

when the speaker participates from a conference room, the process implemented is as follows:

the system requires the statement of the number of participants around the table by a sentence such as "please indicate how many participants are around the table" once the number of participants has been obtained (either by typing on a keyboard or by stated voice), and for each of the participants, the system requires, for each, the statement of their name and / or first name;

moreover, the system implements a location algorithm making it possible, as a function of the statement made, to obtain a spatial location of the speaker;

for each speaker, there follows a step of calculating a voiceprint on the basis of the preceding utterance; several cases are possible:

either the speaker is unknown to the system: the calculated fingerprint is then used during the conference;

either the speaker is known to the system (because he has previously registered): in this case the statement of his name and / or first name (and recognition thereof) is used to search for the prerecorded fingerprint of the speaker ; this allows one or other of these fingerprints to be used for speaker recognition;

The conference continues, for the system, with automatic registration and recognition of conference participants. Again, several cases arise:

when the speaker participates via telephone, the audio stream from the microphone of this telephone is recorded continuously and can be decoded (possibly also continuously); the speaker's fingerprint is used to assign this feed to the speaker in question; when the speaker participates via a computer (or a tablet, an IP connection from a smartphone), the process is the same as for a telephone connection; when the speaker participates around a conference table, several procedures are implemented concomitantly:

at. recognition of the speaker's fingerprint using the pre-recorded fingerprint: this recognition also takes into account the spatialization of the sound picked up by the microphone (s);

b. a recording of the audio stream from the conference (all speakers combined);

vs. a recording of the audio stream from the conference by speaker:

for example if two speakers are around the table for the conference, two separate audio streams are recorded and the audio portions are distributed over the two streams according to the recognition of the speaker's fingerprint;

From a general point of view, the system of the invention intervenes in two stages: it recognizes the speakers in real time during the holding of the conference: the identification is carried out in real time on the basis of the footprints previously calculated; it involves creating the audio streams for each speaker and marking (temporally) the streams to determine the moments at which the different speakers took the floor; one then implements a temporal synchronization of the audio streams which makes it possible in particular to attribute a set of sounds captured to a given user;

it recognizes the words spoken by each speaker: this recognition phase can either be implemented in real time, for example to carry out a real-time transcription of the conference (for example for people who have lost their hearing) or in deferred time; the advantage of real time is to have, for the participants, an immediate trace of the verbal exchanges; the disadvantage is that it requires significant IT resources, and that the risks of errors are higher; the advantage of deferred time is that it requires fewer IT resources, because the flows can be processed one after the other;

From a general point of view, the system of the invention performs either a direct processing of the conference or an indirect processing of the latter. The distinction between direct and indirect treatment of the conference is as follows:

in direct processing, the system is in charge of conference multiplexing, that is to say setting up any telephone bridges with the different parties (when there is a conference call), in addition to recording and processing audio streams;

in indirect processing, the system is in charge of recording and processing audio streams, including in particular the identification and processing of the speech of the interlocutors: it records any telephone streams and the streams coming from the microphone (s) of the room in which the conference is held.

Claims (6)

Method for processing audio data, said audio data originating from a voice exchange between at least two speakers, said method comprising a phase of processing (P20) of audio data comprising a speaker identification step (P202) which comprises a step of splitting said audio data, as a function of at least one pause model, delivering a set of segmented audio data, said method being characterized in that it comprises, for at least one current audio data of said set of segmented audio data : at least one voice recognition step (10) of said current audio data, implemented by means of at least one voice recognition engine, delivering at least one recognized word or an indicator of lack of recognition; when at least one voice recognition engine delivers at least one recognized word (20): a search step (201), within a database, of at least one reference word corresponding to said at least one recognized word, delivering at least one reference voice identification datum; a step of obtaining (202), from said current audio data, a current voice identification data; a step of calculating (203) at least one correspondence between said current voice identification data and said at least one reference voice identification data, delivering at least one correspondence score; and when a correspondence score exceeds a predetermined threshold (205), a step of allocation (206) of said current audio data to one of said at least two interlocutors. Method according to claim 1, characterized in that the step of obtaining (202), from said current audio data, a current voice identification data comprises: a step of applying, on said current audio data item, which takes the form of a one-dimensional audio signal x (t), a short-term Fourier transform 3. 4. 5. 15 6. 20 7. 8. (TFCT), delivering an X matrix, called energy matrix, of size mxn, m frequency channels and n time frames and whose frequency scale is between 0 and 1 kHz; a step of normalizing the values of said matrix X, delivering a standardized matrix which constitutes the current voice identification data. Method according to claim 2, characterized in that the application of said short-term Fourier transform on said audio signal x (t) mono dimensional is carried out at a time step of 40 ms. Method according to claim 2, characterized in that the values of the elements of the standardized matrix are between 0 and 100. Method according to claim 2, characterized in that said at least one reference voice identification data item is also in the form of a standardized energy matrix, previously recorded in said database. Method according to claim 1, characterized in that the step of calculating (203) at least one match, delivering the match score, comprises a step of determining a Pearson correlation value of the identification data current voice and said at least one reference voice identification data. Method according to claim 1, characterized in that it comprises, prior to said processing phase (20), a phase for obtaining (P10), for each of said at least two speakers, an individual voice print comprising said data reference voice identification. Method according to claim 1, characterized in that said treatment phase further comprises the following steps: recording (P201), in the form of at least one main audio stream of said verbal exchange; when no audio stream exists for the current speaker (P203), creation (P204) of an audio stream specific to the current speaker; time stamping (P205), in said at least one main audio stream and in said own audio stream, using data representative of said current speaker. 9. Audio data processing system, said audio data being produced by a voice exchange between at least two speakers, said system comprising a speaker processing engine (MTrtUsrs), which comprises speaker identification means comprising means for splitting said audio data, as a function of at least one pause model, delivering a set of segmented audio data, Said system being characterized in that it comprises, for at least one current audio data of said set of segmented audio data: means of voice recognition of said current audio data, implemented by means of at least one voice recognition engine, delivering at least one recognized word or an indicator of absence of recognition; 15 - means for searching, within a database, at least one reference word corresponding to said at least one recognized word, delivering at least one reference voice identification datum; means for obtaining from said current audio data, a current voice identification data; Means for calculating at least one correspondence between said current voice identification datum and said at least one reference voice identification datum, delivering at least one correspondence score; and means for allocating (206) said current audio data item to one of said at least two interlocutors, implemented when a score of 25 match exceeds a predetermined threshold. 10. Product computer program downloadable from a communication network and / or stored on a medium readable by computer and / or executable by a microprocessor, characterized in that it comprises program code instructions for the execution of a processing method according to claim 1 to 8, when executed on a processor.