KR102287431B1 - Apparatus for recording meeting and meeting recording system - Google Patents

Abstract

The present invention relates to a meeting recording method. The method includes the steps of: obtaining audio data including a user's voice; converting the user's voice into text using the obtained audio data; extracting a guide text corresponding to a predefined intent from among the text; retrieving guide content corresponding to the guide text; and providing the searched guide content and the extracted text to a user's device. Accordingly, even if the meeting is held for a long time, it is possible to reduce burden of recording meeting contents and writing minutes from the participants.

Description

Conference Recording Device and Conference Recording Recording System

The present invention relates to a conference recording apparatus and a conference recording recording system.

In general, when a meeting is held in a corporate or national institution, the contents of the meeting are recorded and, at the same time, the contents of each speaker are recorded as minutes by the person in charge of recording. These minutes are used to understand the main contents of the meeting or the results of the meeting, and the meeting recorder records all the utterances of the meeting participants in text and summarizes the meeting contents.

In addition, the person in charge of the meeting recorder must collect the meeting materials distributed in printed form and the meeting materials in document and video format and record them in the meeting minutes. Together, you will take notes individually that applies to you.

Since this series of meeting recording process is performed by a human, a situation arises where the speech of the meeting participants is missed while recording the meeting contents, and the productivity of work is reduced until the meeting minutes are written.

The description underlying the invention has been prepared to facilitate understanding of the invention. It should not be construed as an admission that the matters described in the background technology of the invention exist as prior art.

Therefore, there is a need for a technology capable of automatically and quickly recording all utterances in a meeting and organizing the contents of the meeting at a glance.

Accordingly, the inventors of the present invention separate the utterances of a plurality of meeting participants and record them as text, so that the utterances of the meeting participants are not omitted even when the meeting continues for a long time or the utterances of some meeting participants overlap, and a method and performing the same It was intended to develop a server and conference recording device.

As a result, the inventors of the present invention grasp the gist of the meeting, that is, the intention (intent) of the meeting, based on the memos, documents, and some utterances written by the meeting participants, and the contents of the meeting based on the identified intent. A method for providing related information for easy understanding, a server for performing the same, and a conference recording device have been developed.

Furthermore, the inventors of the present invention intend to provide customized information that requires a detailed understanding of the meeting participants during the meeting by job and qualification of the meeting participants during the meeting according to any one intent.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above problems, there is provided a conference recording recording method according to an embodiment of the present invention. The method includes: acquiring audio data including a user's voice; converting the user's voice into text using the acquired audio data; and a guide text corresponding to a predefined intent among the texts. It is configured to include the steps of extracting, searching for guide content corresponding to the text for the guide, and providing the searched guide content and the extracted text to the user's device.

According to a feature of the present invention, before the step of acquiring the audio data, the steps of receiving the meeting minutes prepared by the user from the user's device and extracting text for designating the intent from the received meeting minutes may further include.

According to another feature of the present invention, before the step of acquiring the audio data, checking whether the intent is defined and as a result of checking, if the intent is not defined, based on the initial part of the audio data Converting a user's voice within a predetermined time into text, and extracting a word to be defined as an intent from the converted text through any one of morpheme analysis, entity name analysis, and semantic role analysis method It may include further steps.

According to another feature of the present invention, the converting of the voice into text comprises separating the audio signal for the user and other users other than the user by using pre-stored voice information of the user or an audio signal grouped according to a waveform. It may further include the step of

According to another feature of the present invention, before the step of acquiring the audio data, the method may further include receiving personal information or the voice information of a user for defining the intent.

According to another feature of the present invention, the step of separating the audio signal includes: confirming the number of microphone channels of the device that obtained the audio data, and collecting audio signals for two or more users for each of the checked channels may include more.

According to another feature of the present invention, the separating of the audio signal includes checking the number of microphone channels of the device from which the audio data is obtained, and audio signals of other users other than the audio signal for the user for each checked channel. It may further include the step of deleting.

According to another feature of the present invention, the audio signals of the other users may be collected through audio data obtained from the device of the other user and adjacent to the device of the user.

According to another feature of the present invention, the providing to the user's device may further include displaying and providing the separated audio signals of the user and the other user in a graphic form according to time.

According to another feature of the present invention, the step of searching for the guide content may be a step of searching for guide content corresponding to the text for the guide in a database corresponding to the predefined intent.

According to another feature of the present invention, the guide content may include at least one of a dictionary definition of the guide text, laws related to the guide text, internal regulations, and guide guidelines.

According to another aspect of the present invention, the acquiring of the audio data may include acquiring recorded audio data or audio data being recorded in real time.

According to another feature of the present invention, after the step of providing to the user device, the converted text, the guide text, and any one information among the guide content is combined to generate a meeting minute summarizing the audio data. It may include further steps.

In order to solve the above problems, there is provided a conference recording server according to another embodiment of the present invention. The server includes a communication unit, a storage unit, the communication unit, and a processor operatively connected to the storage unit, wherein the processor obtains audio data including a user's voice, and uses the obtained audio data to obtain the audio data. Converts a user's voice into text, extracts guide text corresponding to a predefined intent from among the text, searches guide content corresponding to the guide text, and uses the searched guide content and the extracted text configured to provide to the user's device.

In order to solve the above problems, there is provided a conference recording method according to another embodiment of the present invention. The method includes: obtaining audio data including a user's voice from a multi-channel microphone; separating an audio signal for each channel of a microphone from the obtained audio data; Acquiring an audio signal of a conversation partner with whom the user is conversing, converting the user's voice into text using the acquired audio signal, and extracting a guide text corresponding to a predefined intent among the converted text and transmitting at least one of the obtained audio data, the converted text, and the text for the guide to a conference recording recording server.

According to a feature of the present invention, the method may further include, before the acquiring of the audio data, the user's voice information and the other user's voice information from another meeting recording device adjacent to the meeting recording device.

According to another feature of the present invention, the step of separating the audio signal may include removing the audio signal of a user other than the user according to the size of the audio signal and the stored voice information of the user from the audio signal separated for each microphone channel. It may further include the step of

In order to solve the above problems, there is provided a conference recording apparatus according to another embodiment of the present invention. The apparatus includes an input unit, a storage unit, and a processor operatively connected to the input unit and the storage unit, wherein the processor obtains audio data including a user's voice from a multi-channel microphone, and the obtained audio data separates an audio signal for each channel of the microphone, obtains an audio signal of the user and a conversation partner talking with the user using the separated audio signal, and uses the obtained audio signal to text the user's voice , extracts guide text corresponding to a predefined intent among the converted text, and transmits at least one information of the acquired audio signal, the converted text, and the guide text to the conference recording server configured to do

Specific details of other embodiments are included in the detailed description and drawings.

According to the present invention, it is possible to discriminate the voices of conference participants in a meeting, and automatically and quickly record all utterances of the conference participants. In particular, since the meeting contents are automatically recorded by the meeting recording device or the meeting recording server that records the voices of the meeting participants, even if the meeting is held for a long time, it is possible to reduce the burden on the meeting participants for recording the meeting contents and making the meeting minutes. .

In addition, the present invention converts the contents of the meeting into text by understanding the intention (intent, Intent) of the meeting based on the contents provided by the meeting participants before the meeting is conducted or before the meeting contents are converted into text. During the meeting, various contents can be provided to attendees to understand the contents of the meeting and to communicate smoothly with the other party.

For example, the present invention can provide meeting participants with dictionary definitions of some words in the meeting contents converted into text, laws related to meeting contents, by-laws, guidance guidelines, etc. in various formats such as text, images, and videos, A checklist and various document forms that can be referred to in a meeting for each intent can be provided to meeting participants.

Furthermore, the present invention can provide various contents by job and qualification of meeting attendees, and provide additional content through desired keywords for each participant, thereby increasing the understanding of the meeting for each participant and improving the quality of the meeting and meeting minutes. .

The effect according to the present invention is not limited by the contents exemplified above, and more various effects are included in the present invention.

1A and 1B are schematic diagrams of a conference recording system according to an embodiment of the present invention.
2 is a block diagram illustrating the configuration of a conference recording apparatus according to an embodiment of the present invention.
3 is a block diagram illustrating the configuration of a conference recording server according to an embodiment of the present invention.
4 is a schematic flowchart of a conference recording method of the conference recording apparatus according to an embodiment of the present invention.
5 is a schematic flowchart of a conference recording recording method of a conference recording recording server according to an embodiment of the present invention.
6 and 7 are schematic diagrams for explaining an interface in which conference recordings are recorded according to an embodiment of the present invention.
8 and 9 are schematic diagrams exemplarily illustrating an interface for providing content for a meeting guide according to an embodiment of the present invention.

Advantages and features of the present invention, and a method for achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims. In connection with the description of the drawings, like reference numerals may be used for like components.

In this document, expressions such as "has," "may have," "includes," or "may include" refer to the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.

In this document, expressions such as "A or B," "at least one of A and/and B," or "one or more of A or/and B" may include all possible combinations of the items listed together. . For example, "A or B," "at least one of A and B," or "at least one of A or B" means (1) includes at least one A, (2) includes at least one B; Or (3) it may refer to all cases including both at least one A and at least one B.

As used herein, expressions such as "first," "second," "first," or "second," may modify various elements, regardless of order and/or importance, and refer to one element. It is used only to distinguish it from other components, and does not limit the components. For example, the first user equipment and the second user equipment may represent different user equipment regardless of order or importance. For example, without departing from the scope of the rights described in this document, the first component may be named as the second component, and similarly, the second component may also be renamed as the first component.

A component (eg, a first component) is "coupled with/to (operatively or communicatively)" to another component (eg, a second component) When referring to "connected to", it will be understood that the certain element may be directly connected to the other element or may be connected through another element (eg, a third element). On the other hand, when it is said that a component (eg, a first component) is "directly connected" or "directly connected" to another component (eg, a second component), the component and the It may be understood that other components (eg, a third component) do not exist between other components.

As used herein, the expression "configured to (or configured to)" depends on the context, for example, "suitable for," "having the capacity to ," "designed to," "adapted to," "made to," or "capable of." The term “configured (or configured to)” may not necessarily mean only “specifically designed to” in hardware. Instead, in some circumstances, the expression “a device configured to” may mean that the device is “capable of” with other devices or parts. For example, the phrase “a processor configured (or configured to perform) A, B, and C” refers to a dedicated processor (eg, an embedded processor) for performing the operations, or by executing one or more software programs stored in a memory device. , may mean a generic-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

Terms used in this document are only used to describe specific embodiments, and may not be intended to limit the scope of other embodiments. The singular expression may include the plural expression unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meanings as commonly understood by one of ordinary skill in the art described in this document. Among the terms used in this document, terms defined in a general dictionary may be interpreted with the same or similar meaning as the meaning in the context of the related art, and unless explicitly defined in this document, ideal or excessively formal meanings is not interpreted as In some cases, even terms defined in this document cannot be construed to exclude embodiments of this document.

Each feature of the various embodiments of the present invention can be partially or wholly combined or combined with each other, and as those skilled in the art can fully understand, technically various interlocking and driving are possible, and each embodiment may be implemented independently of each other, It may be possible to implement together in a related relationship.

For clarity of interpretation of the present specification, terms used herein will be defined below.

1A and 1B are schematic diagrams of a conference recording system according to an embodiment of the present invention.

1A and 1B , the conference recording system 1000 may record and record a multi-party conference as shown in FIG. 1A or record and record a one-to-one conference as shown in FIG. 1B according to the configuration of the conference participants. Specifically, in FIG. 1A , the conference recording recording system 1000 includes conference recording apparatuses 100-1, 100-2, 100-3, ( 100-4) (hereinafter referred to as (100)) and the conference recording server 200 for receiving and converting audio data including the user's voice acquired by the conference recording apparatus 100 into text and the converted text It may include a display device 300 for outputting .

In addition, in FIG. 1B , the conference recording and recording system 100 provides audio including the user's voice acquired by the conference recording apparatus 100 and the conference recording apparatus 100 disposed between the users E and F talking to each other. It may include a conference recording recording server 200 that receives the data and converts it into text.

The conference recording apparatus 100 is a device that obtains audio data including the user's voice during a meeting and converts the user's voice into text. It may include various electronic devices that can

According to an embodiment, the conference recording apparatus 100 may be disposed in front of or near the user to obtain audio data including the user's voice, and separately obtain audio signals of each of a plurality of users. For example, the conference recording apparatus 100 obtains an audio signal of a user specified in the conference recording apparatus 100 before proceeding with the conference, and based on this, the user, the conversation partner with whom the user is talking, or other users other than the user Audio signals may be separately acquired. As another example, the conference recording apparatus 100 may separate and obtain audio signals of two or more users by classifying and grouping audio signals according to waveforms analyzed from audio data recorded after the conference is over.

Specifically, the conference recording apparatus 100 may separately acquire audio signals of users by using a multi-channel microphone. In an embodiment of the present invention, the conference recording apparatus 100 may have a multi-channel microphone for acquiring the user's voice, and the conference recording apparatus 100 has a difference in the size of an audio signal according to the arrangement direction of each of the multi-channel microphones. , it is possible to separately acquire the audio signals of users in consideration of the acquisition time difference.

Also, the conference recording apparatus 100 may obtain voice information of another user, that is, an audio signal, from another conference recording apparatus 100 adjacent to the conference recording apparatus 100 . However, here, the conference recording apparatus 100 may recognize or receive information about the arrangement direction of the other conference recording apparatus 100 together with the voice information of the other user, so that the audio signals obtained for each channel may be classified for each user.

The conference recording apparatus 100 may classify the voice of one designated user or classify the voices of a plurality of users and convert them into text. If the conference recording apparatus 100 intends to distinguish the voice of one designated user, audio signals of other users other than the user may be removed according to pre-stored voice information (audio signals) of the user.

According to an embodiment, the conference recording apparatus 100 may convert a user's voice into text and extract a guide text corresponding to a predefined intent from the converted text. Here, the intent means a topic of a meeting to be recorded or a meeting on which the recording is completed, and the guide text means a reference text for providing necessary information to the user according to the intent. For example, the intent may include “Guide to verbal abuse counseling”, “Guide to sexual harassment in public places”, “Guide to budget meeting”, etc. The text for the guide is “How to ask for help”, “Supplementary budget” It is not defined among meeting participants in the course of the meeting, but it may contain words and phrases that need to be understood through supplementary explanations.

That is, the conference recording apparatus 100 finally acquires the audio signal, converts the user's voice into text, and can extract the guide text from among them, and includes three pieces of information (audio signal, text, guide use). text) may be transmitted to the conference recording server 200 .

In addition, when the conference recording apparatus 100 is implemented as an electronic device including a visual output device such as a smart phone, a tablet PC, or a PC, the conference recording apparatus 100 is used for the user's voice conversation contents and guides converted into text Text may be output, and conversely, when an output device is not included, dialogue content and text for a guide may be output through a separate display device 300 as shown in FIG. 1A .

Meanwhile, although one display device 300 is illustrated in FIG. 1A , a display device 300 may be provided for each meeting participant A, B, C, and D, and the meeting participant A, B )(C)(D), different guide contents for each job and position may be provided by the conference recording server 200 .

The conference recording recording server 200 is a server for recording conference contents stored in the form of audio data as text, and may include various electronic devices such as a PC, a tablet PC, and a data server. According to an embodiment, the conference recording server 200 may provide a web or mobile application, program, widget, etc. for conference recording and conference recording that is installed in the conference recording apparatus 100 and is executable.

According to an embodiment, the conference recording recording server 200 receives the audio data including the user's voice, the converted text, and the guide text from the conference recording apparatus 100, and searches for guide content corresponding to the guide text. can be provided. Here, guide content is a generic term for information that must be provided to users according to the guide text. It includes a dictionary definition of guide text, laws related to guide text, bylaws, guide guidelines, etc., and includes text, images, and video It may be provided in various formats, such as

In addition, the guide content is content retrieved from a database corresponding to a predefined intent, and the conference recording server 200 further confirms user personal information such as job, qualification, gender, age, etc. of a user who will provide the guide content and, based on this, it is possible to quickly search for guide content suitable for the user.

According to another embodiment, the conference recording server 200 may receive only audio data including the user's voice from the conference recording apparatus 100 , convert it into text, and then extract the guide text.

Meanwhile, the conference recording recording server 200 may complete text conversion and guide content provision for a recorded conference or a conference in which recording is completed in real time, and may generate a summary of the conference minutes. Specifically, the conference recording recording server 200 may generate conference minutes summarizing the corresponding recording (audio data) by combining information of any one of the extracted text, guide text, and guide content. Here, the meeting minutes may be formed in a combination of various types such as text, image, and video.

So far, the conference recording and recording system 1000 according to an embodiment of the present invention has been described. According to the present invention, users attending a conference through the conference recording and recording system 1000 can conveniently record the conference contents without the effort of directly writing the conference contents. In addition, content suitable for each characteristic of the user who will view the minutes may be provided to help the understanding of the contents of the meeting.

Hereinafter, the conference recording apparatus 100 arranged in a conference hall to record the contents of a multi-party conference will be described in detail.

2 is a block diagram illustrating the configuration of a conference recording apparatus according to an embodiment of the present invention.

Referring to FIG. 2 , the conference recording apparatus 100 may include an input unit 110 , a storage unit 120 , a communication unit 130 , and a processor 140 .

The input unit 110 may obtain audio data for the user's voice. According to an embodiment, the input unit 110 may receive a user's voice through a multi-channel microphone, or may receive two or more user's voices. The audio data obtained by the input unit 110 may be divided into two or more audio signals by the processor 140 .

The storage unit 120 may store voice information (audio signal) of a user who possesses the conference recording apparatus 100 . Specifically, when the conference recording device 100 is a device used only by a designated user, the storage unit 120 may store the user's voice information and user personal information (eg, the user's job, qualification, gender, age). In addition, the user's personal information may be used to extract the guide text.

Also, the storage unit 120 may store candidate reference texts for extracting the guide text from the text converted by the processor 140 . Here, the intent means a topic of a meeting to be recorded or a meeting on which the recording is completed, and the guide text means a reference text for providing necessary information to the user according to the intent. For example, the intent may include “Guide to verbal abuse counseling”, “Guide to sexual harassment in public places”, “Guide to budget meeting”, etc. The text for the guide is “How to ask for help”, “Supplementary budget” It is not defined among meeting participants in the course of a meeting, but it may contain words and phrases that need to be understood through supplementary explanations.

That is, since the amount of information stored in the storage 220 of the conference recording recording server 200 is greater than that of the storage 120 of the conference recording apparatus 100, refer to candidates based on the information stored in the storage 120 After the text is extracted, the text to be used as the reference text may be specified by the conference recording server 200 , while actually providing the guide content among the reference texts.

In various embodiments, the storage unit 120 may include a volatile or non-volatile recording medium capable of storing various data, commands, and information. For example, the storage unit 120 is a flash memory type, hard disk type, multimedia card micro type, card type memory (SD, XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, network storage storage, cloud , may include at least one type of storage medium among the blockchain database.

Also, the storage unit 120 may record commands for the operation of the conference recording apparatus 100 . In various embodiments, an application (not shown) for converting meeting contents implemented as audio data into text and extracting text for a guide may be recorded in the storage unit 120 .

The communication unit 130 may send and receive data to and from the conference recording server 200 through a wired/wireless network. For example, the communication unit 130 may transmit the audio data, the converted text, and the extracted guide text to the conference recording server 200, and according to the presence of a visual output device, the conference recording recording server ( 200) may receive guide content corresponding to the guide text.

Also, the communication unit 130 may receive other user's voice information from another conference recording apparatus 100 adjacent to the conference recording apparatus 100 and store it in the storage unit 120 . For example, the communication unit 130 may receive an identification number, user information, and user voice information (audio signal) of the conference recording apparatus 100 adjacent to the local area wireless network.

The processor 140 is operatively connected to the input unit 110 , the storage unit 120 , and the communication unit 130 , and can control the overall operation of the conference recording apparatus 100 , and the application stored in the storage unit 120 . Alternatively, by running the program, the contents of the meeting may be converted into text, and various commands for extracting the text for the guide may be performed.

The processor 140 may correspond to a computing device such as a central processing unit (CPU) or an application processor (AP). In addition, the processor 140 may be implemented in the form of an integrated chip (IC) such as a system on chip (SoC) in which various computing devices are integrated.

According to an embodiment, the processor 140 separates the audio signal for each channel from the audio data obtained through the input unit 110, and uses the separated audio signal to obtain the user and the audio signal of the conversation partner. can For example, the processor 140 may obtain an audio signal of a user specified in the conference recording apparatus 100 before proceeding with the conference, and based on this, the user and the audio signal of the conversation partner to talk with the user may be separately obtained. there is. For another example, the processor 140 may separate and acquire audio signals of two or more users by classifying and grouping audio signals according to a waveform analyzed from audio data that has been recorded after a meeting is completed.

Specifically, the processor 140 may separate the audio signals of multiple users in consideration of the difference in the size of the audio signal and the difference in acquisition time according to the arrangement direction of the multi-channel microphone, and if the audio signal for a specific user is stored, Users can be classified for each audio signal.

That is, the processor 140 may distinguish one designated user's voice or a plurality of user's voices, and may convert them into text. At the same time, the processor 140 may extract the guide text corresponding to the predefined intent among the converted. Here, the extraction may be understood as highlighting or attaching a tag to the guide text in the conversation content of the user and the conversation partner converted into text.

So far, a conference recording apparatus according to an embodiment of the present invention has been described. According to the present invention, the conference recording apparatus 100 can separate the voices of a plurality of users and convert them into text instead of human handwriting input, thereby reducing the hassle of recording the conference contents in the conference process, and the conference contents can be left on the record without omission.

Hereinafter, the conference recording server 200 that records the conference contents and provides necessary information to conference participants will be described in detail.

3 is a block diagram showing the configuration of a conference recording server according to an embodiment of the present invention.

Referring to FIG. 3 , the conference recording server 200 may include a communication unit 210 , a storage unit 220 , an output unit 230 , and a processor 240 .

The communication unit 210 may exchange data with the user's device through a wired/wireless network. For example, when the user's device is a microphone-type conference recording apparatus 100 provided in a conference room and does not include a visual output device, the communication unit 210 includes the user's voice from the conference recording apparatus 100 . Received audio data or text converted from a user's voice, and guide content obtained as a result of analyzing the audio data is provided in the conference room and may be transmitted to the display device 300 connected to the conference recording apparatus 100 .

For another example, when the user's device is the conference recording apparatus 100 including a visual output device, the communication unit 210 may receive audio data including the user's voice or the user's voice from the corresponding conference recording apparatus 100 . The converted text may be received, and guide content may be transmitted to the corresponding conference recording apparatus 100 .

Meanwhile, the audio data received by the communication unit 210 may be already recorded audio data or audio data being recorded in real time.

The communication unit 210 may receive the meeting minutes written by the user, user personal information and voice information input by the user from the user's device. Here, the meeting minutes may be a meeting-related memo written by the user during the meeting, or a document used for the meeting, which may be utilized by the meeting recording server 200 to designate the subject of the meeting, that is, the intent. . In addition, the user personal information is information for receiving suitable guide content for each user, and includes the user's job, qualification, gender, age, etc., and the voice information is an audio signal for the user's voice, and the conference recording server 200 ) can be used to separate audio signals for users other than the user.

Also, in order to separate the audio signal, the communication unit 210 may additionally receive an audio signal of another user included in the audio data in addition to the user who is a target of providing the guide content. For example, the communication unit 210 may receive audio signals for the voices of users from a plurality of conference recording apparatuses 100 arranged in a conference room.

The storage 220 may store various guide contents that can be provided to the user for each intent. To this end, the storage 220 may include a plurality of databases corresponding to predefined intents. Here, the intent refers to the subject of a meeting in which recording is being conducted or a meeting in which recording has been completed. , “regular sale planning”, etc. may include various intents depending on the type of company.

In addition, the storage unit 220 may store matters to be checked by the user for each intent, that is, information on a checklist and a document form required for conducting a meeting for each intent. For example, when the intent is a performance production meeting, the checklist may include a date and time, a place, a future schedule, and the like. This checklist may be selectively provided together with guide content provided by the processor 240 according to the type of intent, and according to implementation, the checklist may be added/deleted by the user.

In addition, the storage 220 may store words similar to the intent, ie, synonyms for each intent, and the synonyms may be used to expand the database in the process of searching for guide content.

According to an embodiment, when the conference recording apparatus 100 is a device assigned to one user, the storage 220 may match and store the corresponding conference recording apparatus 100 and user information. Specifically, the user information may include user personal information and voice information.

In various embodiments, the storage 220 may include a volatile or non-volatile recording medium capable of storing various data, commands, and information. For example, the storage unit 220 is a flash memory type, hard disk type, multimedia card micro type, card type memory (SD, XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, network storage storage, cloud , may include at least one type of storage medium among the blockchain database.

Also, the storage 220 may record commands for the operation of the conference recording server 200 . In various embodiments, an application (not shown) for converting meeting contents implemented as audio data into text and providing guide contents for each meeting participant may be recorded in the storage 220 .

The output unit 230 may convert the user's voice into text and display it. Specifically, the output unit 230 may display the reproduction graphic of the audio data, and at the same time, the real-time converted text may be displayed separately for each user. For example, the audio signal of the user and other users separated by the processor 240 can be displayed in graphic form according to time, and in addition, the text for the guide extracted by the processor 240 is displayed as a separate highlight, Guide content corresponding to the guide text may be displayed together.

According to an embodiment, the user can directly contact the user's device 100 or the conference recording server 200 to check the user's voice and the converted text while moving the audio signal graphic displayed on the output unit 230. And it is possible to correct the part where there is an error in text conversion.

The processor 240 is operatively connected to the communication unit 210 , the storage unit 220 , and the output unit 230 to control the overall operation of the conference recording server 200 , and to the storage unit 220 . By driving a stored application or program, various commands for converting meeting contents into text and providing guide contents may be performed.

The processor 240 may correspond to a computing device such as a central processing unit (CPU) or an application processor (AP). In addition, the processor 240 may be implemented in the form of an integrated chip (IC) such as a system on chip (SoC) in which various computing devices are integrated.

The processor 240 may convert the user's voice into text. For this, one It is possible to separate the audio signals of multiple users from the audio data. For example, the processor 240 may separate audio signals for the user and other users in a state in which voice information of a user who is a target for providing the guide content is stored in advance. Specifically, the processor 240 identifies a plurality of microphone channels in the user's device (the conference recording apparatus 100) that has obtained the audio data, and compares and analyzes the audio signals for each channel to obtain audio signals for the user and other users. can be separated. That is, the audio data consists of audio signals obtained from a plurality of microphone channels, and the processor 240 can separate the audio signals for a plurality of users by comparing the size of the audio signal for each channel and the acquisition time. In addition, the processor 240 may convert the user's voice to text only for one designated user, and accordingly, may delete audio signals of other users in addition to the pre-stored user's audio signals.

In addition, when there is no pre-stored voice information of the voice user, the processor 240 may separate the audio signals of two or more users by classifying and grouping the audio signals according to the waveform analyzed from the recorded audio data.

On the other hand, so that the processor 240 can more easily separate the audio signals of the users, the device of the user who obtained the audio data (the conference recording apparatus 100) and the device of another user (the other conference recording apparatus 100) adjacent to the user's device (the other conference recording apparatus 100) Audio data may be obtained from the , and information on a direction in which another user's device is arranged may be obtained together with the audio data. That is, the processor 240 may check the size and acquisition time of the audio signal for each microphone channel disposed in different directions, and may separate the audio signal for each designated user according to the acquired audio signal.

According to an embodiment, the processor 240 may convert the user's voice into text using the obtained audio data, and extract the guide text corresponding to the predefined intent. The guide text refers to reference text for providing necessary information to a user according to an intent. For example, the intent may include “Volunteer Counseling Guide”, “Guide to Public Sexual Abuse Investigation”, “Guide to Budget Meeting”, etc. The text for the guide is “How to ask for help”, “Supplementary Budget Proposal” ”, etc., are not defined among meeting participants in the course of the meeting, but may contain words and phrases that need to be understood through supplementary explanations.

Meanwhile, the processor 240 may check whether an intent is defined in order to extract the guide text. Here, the predefined intent may be input by the user or may be designated by the processor 240 through meeting minutes uploaded by the user. The processor 240 may check whether a file input or uploaded by the user exists, and may designate an intent before extracting text for a guide.

For example, the processor 240 may designate an intent through meeting minutes uploaded by the user. Specifically, the processor 240 may extract text for designating an intent from the meeting minutes created by the user, and audio data and meeting minutes for a previous meeting stored in the storage 220 may be used for extracting the text. there is.

As another example, when the intent is not designated because data input by the user or meeting minutes do not exist, the processor 240 may designate the intent using audio data. Specifically, the processor 240 may designate an intent using the user's voice in the initial part of the audio data, and more specifically, the processor 240 may designate the user's voice within a predetermined time (eg, within 1 minute). can be converted into text and a word to be defined as an intent can be extracted by analyzing the converted text. In other words, at the beginning of the meeting, there are utterances related to the topic of the meeting such as “We will start the meeting on Agenda A”, “We will start the investigation on Case B”, and “In relation to the marketing of this C product~” As a result, the processor 240 may convert the user's voice into text within a predetermined time, and extract a word to be defined as an intent from the text through various analysis methods. For example, the processor 240 may extract a word using a morpheme analysis, an entity name analysis, and a semantic role analysis method.

Meanwhile, as an intent is designated, the processor 240 may provide a document form necessary for a meeting for each intent to the user's device before searching and providing the guide content. For example, when the user's device is implemented as an electronic device including an output device, the processor 240 may control to transmit an intent-related document (such as an estimate, a pledge, etc.) through the communication unit 210 .

The processor 240 may search for guide content corresponding to the guide text, and provide the guide content together with the text converted in real time to the user's device. Specifically, the processor 240 may search the guide content corresponding to the text for the guide in the database corresponding to the predefined intent. For example, the processor 240 projects the guide text into a space vector of a neural network-based language model corresponding to a specific intent category, and then calculates and calculates the cosine similarity between the guide text and the content. It may be determined that content having a similarity level greater than or equal to a specified value is guide content to be provided to the user.

In addition, the guide content searched by the processor 240 is a generic name of information that must be provided to the user according to the guide text, and includes a dictionary definition of the guide text, laws related to the guide text, bylaws, guide guidelines, etc. and may be provided in various formats such as text, image, and video.

Here, the processor 240 may check user personal information such as job, qualification, gender, age, etc. of the user who will provide the guide content, and quickly search for guide content suitable for the user based thereon.

According to an embodiment, the processor 240 may search for and provide a matter to be confirmed in the meeting according to the previously designated intent along with the guide content. For example, if the intent is a regular sale planning meeting, the check list may include a date and time, a place, a sale price, and the like.

After converting all audio signals included in the audio data into text, the processor 240 may combine information on any one of text, guide text, and guide content to generate meeting minutes summarizing the audio data. The minutes may include text and highlighted text for a guide along with graphics that reproduce audio data, and may include guide content implemented in various formats. The processor 240 may utilize text and content having the highest similarity to an intent among guide texts and guide content as summary information of audio data.

So far, the conference recording server 200 according to an embodiment of the present invention has been described. According to the present invention, meeting contents can be quickly converted into text through the meeting recording server 200, and additional information suitable for the user to be provided with the meeting minutes is provided, so that the user can obtain personalized meeting minutes.

Hereinafter, a method of recording the conference contents through the conference recording apparatus 100 and recording the conference contents through the conference recording recording server 200 will be described.

4 is a schematic flowchart of a conference recording method of a conference recording apparatus according to an embodiment of the present invention, and FIG. 5 is a schematic flowchart of a conference recording recording method of a conference recording recording server according to an exemplary embodiment of the present invention.

Referring to FIG. 4 , the conference recording apparatus 100 acquires audio data including the user's voice from the multi-channel microphone ( S110 ). Specifically, the conference recording apparatus 100 includes multi-channel microphones arranged in different directions, and through the multi-channel microphone, audio data including voices of other users other than the user located in front of the conference recording apparatus 100 are recorded. can be obtained

After step S110, the conference recording apparatus 100 separates the audio signal for each microphone channel from the obtained audio data (S120). For example, the conference recording apparatus 100 may separate several audio signals in consideration of a difference in amplitudes of audio signals and a difference in acquisition time depending on the arrangement direction of each of the multi-channel microphones.

After step S130, the conference recording apparatus 100 obtains the audio signal of the user and the conversation partner talking to the user by using the separated audio signal (S130). Here, if there is preset user's voice information (audio signal), the conference recording apparatus 100 may separate the conversation partner's audio signal based on the user's voice information. Also, the conference recording apparatus 100 may receive location information and other user's audio information (audio signal) from another adjacent conference recording apparatus 100 , and may obtain an audio signal of the user and the conversation partner based thereon.

According to an embodiment, the conference recording apparatus 100 may convert the separately obtained audio signal into text, and extract text for a guide corresponding to a predefined intent from the converted text. Here, the intent means a subject of a meeting to be recorded or a meeting on which recording has been completed, and the guide text means a reference text for providing necessary information to the user according to the intent, and the conference recording device ( 100) can highlight text for guides in the converted text, or tag text.

After step S130, the conference recording apparatus 100 transmits the obtained audio signal to the conference recording recording server (S140). Specifically, the conference recording apparatus 100 may transmit the converted text and guide text together with the audio signal to the conference recording recording server 200 . In addition, the conference recording apparatus 100 may transmit personal information such as the user's job, qualification, gender, age, etc. to the conference recording recording server 200 together, which will be utilized when creating conference minutes and providing individual guide content. can

Thereafter, the conference recording apparatus 100 may receive guide content corresponding to the guide text from the conference recording recording server 200 according to whether a visual output device is provided, and provides the guide content to the user through the output device. can provide

On the other hand, the conference recording apparatus 100 may perform only conference recording. In this case, the conference contents are recorded in the conference recording recording server 200, and content suitable for each user may be provided.

Hereinafter, with reference to FIG. 5 , a method for recording meeting contents of the meeting recording server 200 will be described.

Referring to FIG. 5 , the conference recording server 200 obtains audio data including the user's voice ( S210 ). At this time, the conference recording recording server 200 may check the number of microphone channels of the user's device (conference recording apparatus 100) that has obtained the audio data, and separate the audio signals of the user and other users for each confirmed channel. . In addition, the conference recording recording server 200 may obtain audio data from the device of the user who obtained the audio data (the conference recording apparatus 100) and the device of another user (the other conference recording apparatus 100) adjacent to it, In addition to the audio data, information on a direction in which another user's device is arranged may be acquired. That is, the conference recording server 200 may check the size and acquisition time of the audio signal for each microphone channel disposed in different directions, and separate the audio signal for each designated user according to the acquired audio signal.

Through such separation of audio signals, the conference recording server 200 may separate audio signals of each of a plurality of users or may delete audio signals of other users in addition to previously stored audio signals of users.

After step S210, the conference recording server 200 converts the user's voice into text using the obtained audio data (S220), and the conference recording recording server 200 provides a guide corresponding to a predefined intent among the texts. The text is extracted for use (S230). However, here, the conference recording server 200 may check whether an intent is defined in order to extract the guide text. Here, the predefined intent may be input by the user or may be designated through the meeting minutes uploaded by the user by the meeting recording server 200 . The conference recording server 200 may check whether a file input or uploaded by a user exists, and may designate an intent before extracting text for a guide.

For example, the conference recording recording server 200 may designate an intent through the conference minutes uploaded by the user. Specifically, the conference recording recording server 200 may extract text for designating an intent from the meeting minutes written by the user, and to the conference recording recording server 200 for text extraction of the conference recording recording server 200 . Audio data and minutes of stored previous meetings may be used.

As another example, the conference recording server 200 may designate an intent by using audio data when the intent is not specified because data input by the user or the meeting minutes do not exist. Specifically, the conference recording recording server 200 may designate an intent using the user's voice in the initial part of the audio data, and more specifically, the conference recording recording server 200 may be configured within a predetermined time (eg, 1 minute). ) converts the user's voice into text, and analyzes the converted text to extract a word to be defined as an intent. In other words, at the beginning of the meeting, there are utterances related to the topic of the meeting, such as “We will start the meeting on Agenda A”, “We will start the investigation on Case B”, and “In relation to the marketing of this product C~”. As such, the processor 240 may convert the user's voice into text within a predetermined time period, and extract a word to be defined as an intent from the text through various analysis methods. For example, the conference recording server 200 may extract a word using a morpheme analysis, an entity name analysis, and a semantic role analysis method.

After step S230, the conference recording server 200 searches for guide content corresponding to the guide text (S240). Specifically, the conference recording server 200 may search for guide text and corresponding guide content from a database corresponding to a predefined intent. For example, the conference recording server 200 projects the guide text into a space vector of a neural network-based language model corresponding to a specific intent category, and then calculates the cosine similarity between the guide text and the content. Accordingly, it may be determined that content having a calculated similarity greater than or equal to a specified value is guide content to be provided to the user.

In addition, the guide content searched by the conference recording server 200 is a generic term for information that must be provided to users according to the guide text, and includes a dictionary definition of the guide text, laws related to the guide text, bylaws, and guidance. Including instructions, and the like, may be provided in various formats such as text, image, video, and the like.

In addition, the conference recording server 200 may check user personal information such as job, qualification, gender, age, etc. of the user who will provide the guide content, and quickly search for guide content suitable for the user based on this.

After step S240, the conference recording server 200 provides the searched guide content and the extracted text to the user's device (S250). In addition, the conference recording server 200 may combine any one information among text, guide text, and guide content to generate conference minutes summarizing audio data, and provide them to the user's device. The minutes may include text and highlighted text for a guide along with graphics that reproduce audio data, and may include guide content implemented in various formats. The conference recording server 200 may utilize text and content having the highest similarity to the intent among guide texts and guide content as summary information of audio data.

So far, a method for recording a conference recording according to an embodiment of the present invention has been described. Hereinafter, an interface screen for recording a conference recording and a result thereof will be described with reference to FIGS. 6 to 9 .

6 and 7 are schematic diagrams for explaining an interface in which meeting recordings are recorded according to an embodiment of the present invention, and FIGS. 8 and 9 are for providing content for a meeting guide according to an embodiment of the present invention It is a schematic diagram illustrating an interface by way of example.

Referring to FIG. 6 , when the conference recording apparatus 100 is a PC including a visual output device, the conference recording content may be displayed as follows while the conference is in progress. Specifically, the conference recording apparatus 100 displays the audio signal 61 together with the user information 61 in graphic form, and the conference contents converted into text by the conference recording apparatus 100 or the conference recording recording server 200 . (Content of conversation) may be displayed at the bottom, and the meeting contents may be displayed separately from the questioner (G) and the answerer (H).

In addition, in a situation where text is converted while recording a meeting in real time, if the intent is not defined, the intent is to be designated in the initial recording by the meeting recording device 100 or the meeting recording recording server 200 . can Accordingly, based on the conversation content (63) and user information (61) of “I came to make a statement of damage in a sexual harassment case” by the conference recording device 100 or the conference recording recording server 200, Intent may be specified.

After that, when the guide text “① There was nowhere to go” is extracted from the meeting content (conversation content) by the meeting recording device 100 or the meeting recording recording server 200, the questioner (G) responds to the answerer (H) ② Guide content 64 related to the description of the shelter that can be explained to the user is provided by the conference recording recording server 200 , and the corresponding content may be displayed by the conference recording apparatus 100 .

Referring to FIG. 7 , when the conference recording apparatus 100 is a PC including a visual output device, the recorded conference contents may be displayed as follows. Specifically, the conference recording apparatus 100 provides the guide content 72 (eg, definition of NTTP training, host organization site link) corresponding to the guide text 71 (eg, 1. NTTP training) among the text-converted contents. ) may be provided and displayed. Also, for example, if “Budget Deliberation Guide” is a predefined intent, guide content 73 (eg, 8 in favor) corresponding to the text for guide (eg 2. Supplementary Budget - 3. 8 in favor) person's name/title) may be provided and displayed, and the guide content here may be information retrieved from a database corresponding to the intent or information retrieved from meeting materials uploaded to the meeting recording server 200 by the meeting participants.

In addition, the meeting recording apparatus 100 may provide and display summarized meeting minutes 74 at the bottom of the area providing guide content, and the meeting minutes may be generated using text, guide text, and guide content.

Referring to FIG. 8 , a user using the conference recording apparatus 100 selects an “intent” item from a category 81 including statistics, authority management, intent, synonym, and recognition rate, and records You can check the meeting details. Specifically, the user can select the type of intent by selecting the category ① by selecting the category, and accordingly, the intent 82 related to sexual harassment and the contents of the meeting recorded by each intent 83 are titled in the meeting recording device 100 . and may be displayed in the form of subheadings.

Finally, referring to FIG. 9 , a user using the conference recording device 100 selects a “synonym” item from the category 91 including statistics, authority management, intent, synonym, and recognition rate, and enters the intent. Keywords 92 to be added may be added or deleted. Here, the keyword may be used to expand the database for each intent for searching guide content.

So far, the conference recording system 1000 and the conference recording and recording method according to an embodiment of the present invention have been described. According to the present invention, the conference recording server 200 can provide various contents for understanding the conference contents to the conference participants while converting the conference contents into text by identifying the intention (intent) of the conference. .

Although one embodiment of the present invention has been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and various modifications may be made within the scope without departing from the technical spirit of the present invention. . Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

1000: conference recording recording system
100: conference recording device
110: input unit
120: storage
130: communication department
140: processor
200: conference recording recording server
210: communication unit
220: storage
230: output unit
240: processor
300: display device

Claims (18)

obtaining audio data including a user's voice;
converting the user's voice into text using the acquired audio data;
extracting a guide text corresponding to a predefined intent from among the text and corresponding to a reference text for providing necessary information to the user according to the intent;
searching for guide content corresponding to the text for the guide and including content for supplementary information different for each user; and
providing the searched guide content and the extracted text to the user's device; including,
The providing step is
A method of recording a meeting, which is a step of providing, to the user's device, a first screen in which user information and conversation contents are converted into text and displayed and a second screen in which different guide contents are displayed for each user. According to claim 1,
Prior to obtaining the audio data,
receiving the meeting minutes prepared by the user from the user's device, and
and extracting text for designating the intent from the received meeting minutes. According to claim 1,
Prior to obtaining the audio data,
checking whether the intent is defined, and
As a result of checking, if the intent is not defined, the user's voice is converted into text within a predetermined time based on the initial part of the audio data, and morpheme analysis, entity name analysis, and semantic role analysis method and extracting a word to be defined as an intent from the converted text through any one method. According to claim 1,
Converting the voice to text comprises:
Separating audio signals for the user and other users other than the user by using pre-stored audio signals grouped according to the user's voice information or waveform. 5. The method of claim 4,
Prior to obtaining the audio data,
and receiving the user's personal information or the voice information for defining the intent. 5. The method of claim 4,
Separating the audio signal comprises:
and confirming the number of microphone channels of the device that obtained the audio data, and collecting audio signals for two or more users for each of the checked channels. 5. The method of claim 4,
Separating the audio signal comprises:
and confirming the number of microphone channels of the device that obtained the audio data, and deleting audio signals of other users other than the audio signals for the user for each of the checked channels. 8. The method of claim 7,
The audio signals of the other users are
The method of claim 1, which is collected through audio data obtained from a device of another user adjacent to the device of the user. 5. The method of claim 4,
The step of providing to the user's device comprises:
The method further comprising the step of displaying and providing the separated audio signals of the user and the other user in a graphic form according to time. According to claim 1,
The step of searching the guide content is,
and searching for guide content corresponding to the guide text in a database corresponding to the intent. 11. The method of claim 10,
The guide content is
A conference recording recording method comprising at least one of a dictionary definition of the guide text, laws related to the guide text, bylaws, and guide guidelines. According to claim 1,
Acquiring the audio data includes:
A method for recording conference recording, the step of acquiring recorded audio data or audio data being recorded in real time. According to claim 1,
After providing to the user's device,
and generating meeting minutes summarizing the audio data by combining information of any one of the converted text, the guide text, and the searched guide content. communication department;
storage;
a processor operatively connected to the communication unit and the storage unit; including,
The processor is
Acquire audio data including the user's voice,
A guide that converts the user's voice into text using the obtained audio data, corresponds to a predefined intent among the text, and corresponds to a reference text for providing necessary information to the user according to the intent Doedoe configured to extract a text for a guide, search for guide content that is the text for the guide, and include content for supplementary information different for each user, and provide the searched guide content and the extracted text to the user's device,
The processor is configured to provide, to the user's device, a first screen in which user information and conversation contents are converted into text and displayed and a second screen in which guide content different for each user is displayed together. obtaining audio data including a user's voice from a multi-channel microphone;
separating an audio signal for each channel of a microphone from the obtained audio data;
obtaining an audio signal of the user and a conversation partner talking with the user by using the separated audio signal;
Converts the user's voice into text using the obtained audio signal, corresponds to a predefined intent among the converted text, and corresponds to a reference text for providing necessary information to the user according to the intent extracting text for a guide; and
transmitting at least one of the acquired audio data, the converted text, and the guide text to a conference recording server; A method of recording a meeting comprising a. 16. The method of claim 15,
Prior to obtaining the audio data,
and storing the user's voice information and other user's voice information from another conference recording device adjacent to the conference recording device. 17. The method of claim 16,
Separating the audio signal comprises:
removing an audio signal of a user other than the user according to the size of the audio signal and the stored user's voice information from the audio signal separated for each microphone channel; input unit;
storage;
a processor operatively connected to the input unit and the storage unit; including,
The processor is
Acquires audio data including a user's voice from a multi-channel microphone, separates an audio signal for each channel of the microphone from the obtained audio data, and uses the separated audio signal to communicate with the user and the user obtains an audio signal of , converts the user's voice into text using the obtained audio signal, corresponds to a predefined intent among the converted text, and provides necessary information to the user according to the intent and extracting a guide text corresponding to a reference text for doing so, and transmitting at least one of the obtained audio signal, the converted text, and the guide text to a conference recording recording server.

Images