CN110705269A - Automatic construction method of multi-source information fusion new word library - Google Patents

Abstract

The invention discloses a method for automatically constructing a new word database with multi-source information fusion. Memory and long-term memory are two multi-level vocabularies; the invention integrates information such as user operation, mouth shape, voice, memory, etc. to automatically recognize new words and dynamically update the vocabularies, and realize the construction process of the vocabularies Compared with the previous method where users judge by themselves and then manually confirm, the learning efficiency can be improved; new words are divided into reading new words and phonetic new words, which is different from the previous method of memorizing unknown or unfamiliar words. It can promote the improvement of reading ability and listening and speaking level at the same time; divide the vocabulary library into two multi-level vocabulary libraries of short-term memory and long-term memory, which provides a means for the study of personalized memory rules.

Description

An automatic construction method of new vocabulary database based on multi-source information fusion

technical field

The invention relates to the technical field of computer applications, in particular to a method for automatically constructing a lexicon based on multi-source information fusion.

Background technique

Internationalization is an important feature of social development. In daily life and work, more and more people need to communicate in foreign languages. Memorizing new words is an effective means to improve foreign language ability. There are also many software tools on the market to assist the memory of new words. The deficiencies of these tools are mainly reflected in the following points:

(1) The words in the new vocabulary database first need users to judge whether they are new words, and then realize the update of the new vocabulary database by manual confirmation, which affects the improvement of learning efficiency;

(2) The comprehension of "new words" is limited to the words that are not known or unfamiliar. Memorizing such new words is beneficial to the improvement of reading ability, but does not directly promote the listening and speaking ability;

(3) Review new words according to the common group law of the Ebbinghaus memory-forgetting curve, ignoring the individual differences in memory-forgetting speed.

SUMMARY OF THE INVENTION

The object of the present invention is to overcome the shortcomings and deficiencies of the prior art, and to provide a method for automatically constructing a new word library with multi-source information fusion, which automatically recognizes and reads new words according to user operation information; Automatically recognize new words in speech; automatically and dynamically update the multi-level new word database according to the law of memory forgetting.

The object of the present invention is achieved through the following technical solutions:

A method for automatically constructing a lexicon based on multi-source information fusion. The lexicon includes two types: reading vocabularies and phonetic vocabularies. The vocabularies are divided into two multi-level vocabularies of short-term memory and long-term memory. The method includes the following steps :

S1, automatic recognition of new words;

S11, according to the user operation information, automatically recognize and read new words;

Described user operation information includes operation reaction time and the information whether the operation is correct, if the user does not make an operation within the specified time or the operation is wrong, the word is recognized as a new word for reading;

S12, according to the information of the user's mouth shape and voice, automatically recognize the new words in the voice;

S2, the new vocabulary is automatically updated;

S21, the recognized new reading words and new phonetic words are automatically stored in the new reading word database and the new phonetic word database respectively;

S22, the automatic dynamic update of the multi-level lexicon.

Preferably, the S12 further includes the following steps:

S121, if the voice recognition device does not receive the user's voice or receives an incorrect user's voice within the specified time, but the mouth shape recognition device recognizes that the user's pronunciation mouth shape is correct, then prompt the user to read it again;

S122, if the speech recognition device does not receive the user's voice within the specified time or receives an incorrect user's speech, and at the same time, the mouth shape recognition device recognizes that the user's pronunciation mouth shape is also wrong, then the word is recognized as a new speech word.

Preferably, the multi-level new vocabulary database in the S22 is divided according to the general law of memory forgetting, which can be used as the basis for the study of the personalized memory law. S22 also includes the following steps:

S221, the new words recognized for the first time are automatically stored in the new word database that needs to be reviewed in the shortest time;

S222, the new word is correctly memorized within the memory period, and is automatically transferred to the next-level new word database with a longer memory period;

S223, the new word has not been correctly memorized within the memory period, and is automatically transferred to the new word library with a shorter memory period at the previous level;

S224, the new word in the new word database with the longest memory period is deleted from the new word database after the correct memory is completed within the memory period.

Compared with the prior art, the present invention has the following beneficial effects:

(1) The present invention fuses information such as user operation, mouth shape, voice, memory, etc., performs automatic identification of new words and dynamic update of the new word database, and realizes the complete automation of the construction process of the new word database. Compared with the manual confirmation method, the learning efficiency can be improved;

(2) the present invention divides new words into reading new words and phonetic new words, compared with the way of being limited to memorizing unrecognized or unfamiliar words in the past, the improvement of reading ability and listening and speaking level can be promoted simultaneously;

(3) The present invention divides the lexicon into two multi-level lexicons of short-term memory and long-term memory, which provides a means for the study of individualized memory rules.

Description of drawings

Fig. 1 is the schematic flow chart of the present invention;

2 is a schematic diagram of a multi-source information acquisition device of the present invention;

Fig. 3 is the schematic flow chart of the automatic recognition of new words of the present invention;

4 is a schematic diagram of an automatic recognition interface for reading new words of the present invention;

Fig. 5 is the interface schematic diagram of the automatic recognition of speech new words of the present invention;

FIG. 6 is a schematic diagram of the structure of the vocabulary library of the present invention;

7 is a schematic diagram of the automatic updating process flow of the new vocabulary database of the present invention;

FIG. 8 is a schematic diagram of the automatic dynamic update process of the multi-level new vocabulary database of the present invention.

Detailed ways

The present invention will be described in further detail below with reference to the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

The invention proposes a method for automatically constructing a new word database with multi-source information fusion, which includes two processes of automatic recognition of new words and automatic update of the new word database; the method is oriented to two purposes of document reading and voice communication, and the new words are divided into reading new words. According to the law of memory and forgetting, the new word database is divided into two multi-level new word databases: short-term memory and long-term memory; this method automatically recognizes and reads new words according to user operation information; , automatically recognize the new words in speech; according to the law of memory forgetting, the multi-level new word database is automatically and dynamically updated.

Specifically, as shown in Figures 1 to 8, an automatic construction method of a new word database based on multi-source information fusion, the new words include two types of reading words and phonetic words, and the new word database is divided into two types: short-term memory and long-term memory. Multi-level lexicon, the method includes the following steps:

Step 1: Automatic recognition of new words.

(1) Automatically identify and read new words according to user operation information.

The user operation information includes operation response time and information on whether the operation is correct. If the user does not perform the operation within the specified time or the operation is wrong, the word is recognized as a new word for reading.

(2) According to the information of the user's mouth and voice, automatically recognize the new words in the voice.

If the voice recognition device does not receive the user's voice or receives the wrong user voice within the specified time, but the mouth-shape recognition device recognizes that the user's pronunciation is correct, it will prompt the user to read it again; If the user's voice is not received or an incorrect user's voice is received within the time, and at the same time, the mouth shape recognition device recognizes that the user's pronunciation mouth shape is also wrong, and the word is recognized as a new voice word.

Step 2, the new vocabulary is automatically updated.

(1) The recognized new reading words and new phonetic words are automatically stored in the new reading word database and the new phonetic word database, respectively.

(2) Automatic dynamic update of multi-level lexicon.

The multi-level lexicon is divided according to the general law of memory forgetting and can be used as the basis for the study of personalized memory law, among which:

The new words recognized for the first time are automatically stored in the new word library that needs to be reviewed in the shortest time; the new words are correctly memorized within the memory cycle, and automatically transferred to the next level new word database with a longer memory cycle; the new words are not completed within the memory cycle. For correct memorization, it will be automatically transferred to the vocabulary library with the shorter memory period at the previous level; the new words in the vocabulary library with the longest memory period will be deleted from the vocabulary library after the correct memory is completed within the memory period.

As shown in FIG. 1 , this embodiment proposes a method 100 for automatically constructing a new word database by fusion of multi-source information, which includes two processes of automatic word recognition 300 and automatic update 700 of the new word database.

The multi-source information includes user operation, mouth shape, voice, memory and other information. As shown in FIG. 2 , the multi-source information acquisition apparatus 200 includes, but is not limited to, a camera 201 , a touch screen 202 , a speaker 203 , a microphone 204 , and software and hardware systems for computing and data storage. Among them, the camera 201 is used to obtain mouth shape information, the touch screen 202 is used to obtain user operation information, and the microphone 204 is used to obtain voice information. Memorized information is gradually forgotten over time and follows the general pattern of fast first and then slow. Setting up multi-level lexicons with different memory periods can record the user's personalized memory information, which provides an effective way to analyze the characteristics of personalized memory and form personalized memory rules.

The invention is oriented to two types of uses of document reading and voice communication, and divides new words into two types: reading new words and phonetic new words. As shown in FIG. 3 , the automatic recognition of new words 300 includes: automatically recognizing and reading new words 301 according to user operation information;

As shown in FIG. 4 , an acquisition interface 400 for automatic recognition information for reading new words includes word 401 , option one 403 , option two 404 , option three 405 , and option four 406 . The relationship between the word 401 and the four options 403, 404, 405, and 406 is similar to a multiple-choice question, and only one option is correct. When the word 401 is in Chinese, the four options 403, 404, 405, and 406 are in foreign languages; when the word 401 is in foreign languages, the four options 403, 404, 405, and 406 are in Chinese. The word 401 is randomly selected from the word bank and displayed on the screen, and the corresponding four options appear in the positions indicated by the four options 403, 404, 405, and 406 respectively. The interface 400 may further include prompt information 402 , which is used to display user operation information such as whether the selection is correct, the duration used, and the like; the related prompt information can also be played through the speaker 203 . When the word 401 appears on the screen, if the user makes a correct choice from the four options 403, 404, 405, and 406 within the specified time, prompt information such as the correct choice and the time used will appear at 402; if the user makes the correct choice within the specified time If no selection is made or the selection is wrong, a prompt message of selection error or timeout appears in 402, and the word 401 is recognized as a new word for reading.

The present invention does not limit the specified time for the user's selection operation, which may be a certain time length within 10 seconds, or may be a time length calculated according to the user's personalized memory characteristics.

The interface 400 may also include game characters 407 to overcome the tediousness of word memory with the entertainment of the game. If the user makes three correct choices in a row, the game character 407 starts dancing; otherwise, the game character 407 stops dancing.

As shown in FIG. 5 , an acquisition interface 500 for automatic recognition information of new words in speech includes words 401 . The word 401 is randomly drawn from the word bank and displayed on the screen. The interface 500 may also include prompt information 402 for displaying information such as whether the pronunciation is correct, whether it needs to be re-read aloud, the duration of use, etc.; the related prompt information can also be played through the speaker 203 . If the microphone 204 receives the user's correct word reading voice within the specified time, prompt information such as the correct pronunciation and the time used will appear in 402; 201 recognizes that the user's pronunciation and mouth shape is correct, then 402 appears a prompt message of re-reading, and the timing restarts; if the microphone 204 does not receive the user's voice within the specified time or receives an incorrect user's voice, at the same time, the camera 201 recognizes If the pronunciation of the user is also wrong, a prompt message 402 of a pronunciation error or a timeout appears, and the word 401 is recognized as a new voice word.

The present invention does not limit the specified time for the microphone 204 to receive the user's correct word reading voice, which may be a certain time within 10 seconds, or a time calculated according to the user's personalized memory characteristics.

The interface 500 may also include game characters 407 to overcome the tediousness of word memory with the entertainment of the game. The user makes three consecutive correct pronunciations, and the game character 407 starts dancing; otherwise, the game character 407 stops dancing.

The present invention does not limit the speech and mouth shape recognition methods, and can use artificial intelligence methods to judge whether the pronunciation of the word and the mouth shape are correct through audio and mouth shape comparison.

As shown in FIG. 6 , the new word database 600 includes a new reading word database 601 and a new speech word database 602 . Since the retention of memory is different in time, there are two kinds of short-term memory and long-term memory. After the input information is learned through the human attention process, it becomes human short-term memory, but if it is not reviewed in time, these memories will be forgotten, and after timely review, these short-term memories will become human. A type of long-term memory that remains in the brain for a long time. Therefore, the present invention subdivides 601 and 602 into a multi-level short-term memory lexicon 603 and a multi-level long-term memory lexicon 604 respectively.

The present invention does not limit the number of levels of the multi-level lexicon. According to the general law of memory forgetting, the short-term memory multi-level lexicon may include a 5-minute lexicon, a 30-minute lexicon and a 12-hour lexicon; The multi-level lexicon of time memory can include 1-generation lexicon, 2-generation lexicon, 4-generation lexicon, 7-generation lexicon and 15-generation lexicon.

As shown in FIG. 7 , the automatic update 700 of the new vocabulary includes: automatically storing the recognized new reading words and new voice words in the new reading vocabulary and the new pronunciation vocabulary 701 respectively; automatic dynamic updating 800 of the multi-level new vocabulary and other steps.

As shown in FIG. 8 , the automatic dynamic update 800 of the multi-level new word database includes: the new words recognized for the first time are automatically stored in the new word database 801 that needs to be reviewed in the shortest time; The new word bank 802 with a longer memory period at the next level; if the new word is not correctly memorized within the memory period, it is automatically transferred to the new word bank 803 with a shorter memory period at the previous level and the new word in the new word library with the longest memory period After completing the correct memorization in the memory cycle, the steps such as 804 are deleted from the new vocabulary database. For example, a new word recognized for the first time is automatically stored in the 5-minute new word database, and the storage time is t. If the user completes the correct memory within t + 5 minutes, the new word will be transferred from the 5-minute new word database to the 30-minute new word database; If the user completes the correct memorization within t+30 minutes, the new word will be transferred from the 30-minute vocabulary database to the 12-hour vocabulary library; otherwise, the new word will be adjusted from the 30-minute vocabulary library back to the 5-minute vocabulary library. Generally speaking, when a new word has been correctly memorized 8 times in the 3-level short-term memory lexicon and 5-level long-term memory lexicon within 15 days, the new word is no longer a new word for the user, and finally from 15 Deleted from the natural thesaurus.

The present invention further describes the process of automatic recognition of new words and automatic update of the new word database of a method for automatically constructing a new word database based on multi-source information fusion, and does not involve a method for obtaining a user's personalized memory rule. However, dividing the new word database into two multi-level word databases of short-term memory and long-term memory can provide support for the research on the user's personalized memory rules. For example, the new words recognized for the first time are automatically stored in the 5-minute new word database, and the storage time is t. If user A does not memorize within t+5 minutes, the new word will remain in the 5-minute new vocabulary database; if user A has memorized it within [t+5, t+30] minutes, but it is incorrect, the New words are kept in the 5-minute vocabulary. If user B does not memorize within t+5 minutes, the new word will be kept in the 5-minute vocabulary database; if user B has memorized correctly within [t+5, t+30] minutes, the new word will be changed from 5 to 5 minutes. The minute vocabulary is directly transferred to the 12-hour vocabulary. If the probability of the above occurrence is high enough, it can be concluded that user B's short-term memory is better than user A's. When user A has memorized correctly within the range of t+5 minutes, the memorized new words will be transferred from the 5-minute vocabulary database to the 30-minute vocabulary library; if user B has memorized correctly within the range of t+5 minutes, then Memorized words can be directly transferred from the 5-minute lexicon to the 12-hour lexicon. In this way, the differences in the individualized memory rules are finally reflected in the dynamic update of the multi-level lexicon, which can further improve the learning efficiency.

The invention integrates information such as user operation, mouth shape, voice, memory, etc., to automatically recognize new words and dynamically update the new word database, and realizes the complete automation of the construction process of the new word database. In the past, users first judged themselves and then manually confirmed. Compared with the way of memorizing unfamiliar or unfamiliar words in the past, the new words can be divided into reading new words and phonetic new words, which can improve reading ability and listening level at the same time. It plays a promoting role; divides the vocabulary database into two multi-level vocabulary libraries of short-term memory and long-term memory, which provides a means for the study of personalized memory rules.

The above are the preferred embodiments of the present invention, but the embodiments of the present invention are not limited by the above-mentioned contents, and any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention are all Should be equivalent replacement manners, all are included within the protection scope of the present invention.

Claims (3)

1. the automatic construction method of a new word bank of multi-source information fusion, it is characterized in that, new word comprises two classes of reading new word and phonetic new word, and new word bank is divided into two multi-level new word banks of short-term memory and long-term memory, The method includes the following steps: S1, automatic recognition of new words; S11, according to the user operation information, automatically recognize and read new words; Described user operation information includes operation reaction time and the information whether the operation is correct, if the user does not make an operation within the specified time or the operation is wrong, the word is recognized as a new word for reading; S12, according to the information of the user's mouth shape and voice, automatically recognize the new words in the voice; S2, the new vocabulary is automatically updated; S21, the recognized new reading words and new phonetic words are automatically stored in the new reading word database and the new phonetic word database respectively; S22, the automatic dynamic update of the multi-level lexicon. 2. the method for automatically constructing the new vocabulary library of multi-source information fusion according to claim 1, is characterized in that, described S12 also comprises the steps: S121, if the voice recognition device does not receive the user's voice or receives an incorrect user's voice within the specified time, but the mouth shape recognition device recognizes that the user's pronunciation mouth shape is correct, then prompt the user to read it again; S122, if the speech recognition device does not receive the user's voice within the specified time or receives an incorrect user's speech, and at the same time, the mouth shape recognition device recognizes that the user's pronunciation mouth shape is also wrong, then the word is recognized as a new speech word. 3. the automatic construction method of the new vocabulary library of multi-source information fusion according to claim 1, is characterized in that, in described S22, the multi-level new vocabulary library is divided according to the general rule of memory forgetting, can be used as personalized memory rule Based on the research, S22 also includes the following steps: S221, the new words recognized for the first time are automatically stored in the new word database that needs to be reviewed in the shortest time; S222, the new word is correctly memorized within the memory period, and is automatically transferred to the next-level new word database with a longer memory period; S223, the new word is not correctly memorized within the memory period, and it is automatically transferred to the new word bank with a shorter memory period at the previous level; S224, the new word in the new word database with the longest memory period is deleted from the new word database after the correct memory is completed within the memory period.

Images