CN102509483A - Distributive automatic grading system for spoken language test and method thereof - Google Patents

Abstract

A distributed automatic scoring system for oral exams, comprising a scoring management device to manage several test client devices, the scoring management device communicates with at least one scoring task scheduling device, and one scoring task scheduling device manages several scoring computing devices, The modules used for automatic scoring include: a recording module, which is used to detect and record the candidate's voice; a voice processing module, which is used to process the candidate's voice signal, extract acoustic features, recognize the text corresponding to the voice, and obtain the time boundary of the acoustic unit; The pronunciation evaluation module is used to analyze and evaluate different aspects of the user's pronunciation according to the topic information and the output results of the speech processing module; the speech flow feature extraction module is used to extract the comprehensive speech flow features that reflect the candidate's oral level, including acoustics, Grammatical and semantic features; Scoring sub-module: use the comprehensive features of language flow to comprehensively score the user's voice; except for the recording module, other scoring modules can be based on the calculation load and network load of the test client device and the scoring calculation device in the background to dynamically schedule deployments.

Description

A distributed automatic scoring system and method for speaking test

technical field

This patent relates to the fields of computer speech signal processing and analysis, language assessment, pattern recognition and computer-aided education, especially the field of computer-aided distributed large-scale oral examination automatic scoring method and system.

Background technique

Traditional examinations can be divided into two categories, one is written examinations; the other is oral examinations (focusing on the oral language ability of students). With the deepening of the concept of quality education, more and more attention has been paid to oral English ability. Correspondingly, oral English teaching and examinations are also becoming more popular. For example, oral English ability has been tested in both national and provincial examinations. In addition, in various professional level examinations, oral English ability has also become the focus of the test, such as Putonghua proficiency test, college English level test, IELTS, TOEFL, etc.

Different from written exams, there are usually two ways of oral exams: one is face-to-face communication between candidates and examiners; the other is to record the spoken voice of candidates in a specific context for the examiner to analyze. There are also two ways to score the speaking test. One is that the examiner gives the score on the spot based on the communication experience; the other is to score by listening and analyzing the candidates' speech. It can be seen that no matter which method is used, the speaking test needs to consume a lot of manpower and material resources, especially in large-scale tests. On the other hand, in large-scale examinations, it is usually required to complete all scoring within a limited time, and the scoring efficiency needs to be improved. In addition, the speaking test is a subjective question, and its scoring workload is huge, and the subjectivity is relatively high, and the consistency is poor. Therefore, in a large-scale oral test, a test system and method that can provide highly consistent scores and have high efficiency and low cost are needed.

Computer technology has been applied in the field of educational examinations, which can reduce examination costs and improve examination efficiency. Typical applications include: general networked test management system [CN101964152A, CN1801244A, CN1732490B, CN101827124A], networked test system for oral English [CN101419795B, Design and Implementation of English Spoken Test System (Master’s Thesis of University of Science and Technology of China)], online marking Scoring system [CN1700200A, CN1598845A, CN1987889A], etc. However, none of these works dealt with the organizational management and automatic scoring of large-scale speaking tests.

Aiming at the subjectivity problem of expert scoring in the current speaking test, the present invention uses computer and voice analysis technology to automatically score candidates' voices with high consistency. Aiming at the huge workload of scoring tasks in large-scale oral exams, the invention utilizes computer networks and computing servers to perform distributed parallel scoring on the voices of large-scale candidates.

Contents of the invention

The invention discloses a set of distributed automatic scoring system and its scoring method for speaking tests, which solves the problems of low scoring efficiency and poor consistency in current speaking tests, and especially provides a set of reliable, high-efficiency and low-cost scoring for large-scale speaking tests solution.

The present invention provides a distributed automatic scoring system for oral exams, which includes a scoring management device to manage several test client devices, the scoring management device communicates with at least one scoring task scheduling device, and one scoring task scheduling device manages several scoring Computing means, where modules for automatic scoring include:

The recording module is used to detect and record the examinee's voice;

The speech processing module is used to process the examinee's speech signal, extract the acoustic feature, recognize the text corresponding to the speech, and obtain the time boundary of the acoustic unit;

The pronunciation evaluation module is used to analyze and evaluate different aspects of the user's pronunciation according to the topic information and the output results of the speech processing module;

The speech flow feature extraction module is used to extract the speech flow comprehensive features reflecting the examinee's oral level, including acoustic, grammatical and semantic features;

Comprehensive scoring module: use the comprehensive features of speech flow to comprehensively score the user's voice;

In addition to the recording module, other scoring modules can be dynamically scheduled and deployed according to the computing load and network load of the test client device and the background scoring calculation device.

Also provided is a distributed automatic scoring method for oral exams, wherein the specific steps of scoring include:

Step 1. The recording device detects the examinee's voice and records it;

Step 2, extracting speech features, using the acoustic model and language model to recognize the speech, and obtaining the corresponding text and its phoneme unit time boundary and likelihood score;

Step 3. Using the phoneme unit time boundary and likelihood score and topic-related knowledge, analyze and evaluate the speech, and obtain the pronunciation accuracy, prosodic accuracy, pronunciation completeness and pronunciation fluency of different phonetic units;

Step 4, integrating speech processing results, evaluation results and topic information, and extracting comprehensive characteristics of speech flow in terms of acoustics, grammar and semantics that reflect the examinee's oral level;

Step 5. Based on the pre-trained model and design rules, use the comprehensive features of speech flow to comprehensively score the oral proficiency of candidates.

A distributed automatic scoring method for an oral test is further provided, which specifically includes the following steps:

Step 1. Candidates obtain the test questions and answer according to the requirements. The test client device records the candidate's voice, performs the first-stage scoring calculation on the testee's voice according to the question information and requirements, and sends the testee's voice and scoring results to the scoring management device according to the data exchange standard ;

Step 2. The scoring management device receives the examinee's answer sent by the test client device and the scoring result of the first stage, analyzes the scoring result, and if the scoring result is not the final scoring, organizes the candidate's voice, the intermediate scoring result and the topic information according to the data exchange standard The scoring task is encrypted and sent to the scoring task scheduling device for further scoring;

Step 3. The scoring task scheduling device receives the scoring task sent by the scoring management device, analyzes the scoring task to obtain the scoring data and task priority, selects the required scoring module and algorithm according to the setting of the scoring module field, generates a scoring calculation program, and prioritizes according to the task Submit to the scoring calculation device in the background for calculation;

Step 4. The scoring calculation device receives the scoring program sent by the task scheduling device, executes the second-stage scoring, and sends the final scoring result to the scoring task scheduling device, and then the scoring task scheduling device forwards the scoring result to the scoring management device.

Description of drawings:

Figure 1 Schematic diagram of the organizational structure of the distributed automatic scoring system for the oral test.

Fig. 2 is a schematic diagram of the scoring process of the distributed automatic scoring system and method for the speaking test.

Figure 3 Schematic diagram of the workflow of the distributed automatic scoring system for the oral test.

Figure 4 is a schematic diagram of the deployment of a typical distributed automatic scoring module for an oral test.

Figure 5 is a schematic diagram of the organizational structure of the distributed automatic scoring system for the oral test connected to the voice cloud device.

Fig. 6 is a schematic structural diagram of a distributed automatic scoring system environment for a simplified speaking test.

Detailed ways

Below in conjunction with accompanying drawing and specific embodiment the present invention is described in further detail:

Fig. 1 shows a schematic diagram of the organizational structure of the system disclosed in the present invention. A distributed automatic scoring system for oral exams consists of four devices: a test client device 101 , a scoring management device 102 , a scoring task scheduling device 103 and a scoring calculation device 104 .

The test client device 101 is a terminal device for test takers to provide test services, usually a computer or an embedded mobile device. It has at least the following hardware configuration: audio playback unit, audio recording unit, computing unit, and network communication unit.

Optionally, GUI and touch operation are supported.

It displays the content of the test questions to candidates through a graphical interface, plays the audio of the test questions through an audio playback unit (such as: headphones or speakers), records the voice of candidates through an audio recording unit (such as: a microphone), and uses the local computing unit to analyze the answers (that is, the voice of candidates) ) to perform preprocessing and certain scoring calculations (referred to as the first stage of scoring), and send candidates’ answers and scoring results (usually intermediate scoring results) to the scoring management device through the network or other means. And it can temporarily store the candidate's voice for offline asynchronous transmission.

It is also possible to configure a dedicated test client program or a browser with a test plug-in on the test client device to complete the above-mentioned recording and scoring work.

Scoring management device 102 is a computer for acquiring candidates' answers and first-stage scoring results, organizing and submitting scoring tasks.

It obtains the examinee's answer and the scoring result of the first stage from the test client device, uses the scoring result and topic information to construct a scoring task, sends it to the scoring task scheduling device, and receives the final scoring result from the scoring task scheduling device.

It has a reliable large-capacity storage device and can generally be configured with high data transmission bandwidth. Save candidates' answers and final scoring results, and temporarily save intermediate scoring results from the first stage of scoring.

The scoring task scheduling device 103 is a computer for receiving scoring tasks, analyzing and constructing scoring calculation tasks, submitting calculation tasks and managing scoring calculation devices.

It receives the scoring task sent by the scoring management device, analyzes the task, extracts the scoring data, automatically organizes the scoring calculation program according to the topic information and the modules required for scoring, generates the scoring calculation task, submits the calculation task to the scoring calculation device through the scheduling algorithm, and finally Obtain final scoring results from the computing device.

In particular, the scoring task scheduling device has a reliable large-capacity storage device and generally can have a high data transmission bandwidth.

In particular, the scoring task scheduling device reserves a section of memory space and maintains a queue of computing tasks sorted by priority.

In particular, the task scheduling algorithm of the scoring task scheduling device is: if the task queue is not full, then insert the task into the corresponding position of the queue according to its priority; otherwise, if the task priority is higher than some task priority in the queue, then insert the task The task with the lowest priority is moved out to the cache area or external memory, and the task is inserted into the corresponding position of the queue; otherwise, the task is stored in the cache area or external memory; the task queue is queried regularly, and if there is space, the task in the cache area is prioritized Level joins the queue; regularly polls each scoring calculation device, if there is a CPU idle, then fetches tasks from the task queue to the CPU for calculation.

The score calculation device 104 is a calculation cluster composed of several high-performance CPUs, all of which can work in parallel. It executes the scoring calculation program submitted by the scoring task scheduling device, obtains the scoring result through calculation, and outputs it to the scoring task scheduling device.

The distributed automatic scoring system for the speaking test is characterized in that the scoring task scheduling device can organize multiple scoring tasks together to form a batch-processing computing task, which is submitted to the background scoring computing device.

The distributed automatic grading system for the speaking test is characterized in that the test client device can deploy multiple grading modules. In the case of ensuring that the test recording process is not affected, the first stage of scoring is completed by using the local CPU. The first stage of grading work can be carried out simultaneously with the examination, and in order to avoid any impact on the examination, it can also be run regularly during the free time outside the examination.

The distributed automatic scoring system for the speaking test is characterized in that the scoring task scheduling device and the scoring management device can be connected through a wide area network or a local area network.

The distributed automatic scoring system for the speaking test is characterized in that the scoring calculation device in the background can be an ordinary computer or a high-performance computing server. These scoring calculation devices are connected through a local area network to form a computing cluster.

The structure of the distributed automatic scoring system of the oral test is: a scoring management device manages several test client devices, the scoring management device communicates with at least one scoring task scheduling device, and a scoring task scheduling device manages A number of score calculation devices in the background.

The network structure of the distributed automatic scoring system of the oral exam is: between the scoring management device and the test client device, connect through a local area network or a dedicated network; connect between the scoring task scheduling device and the scoring calculation device at the background; The data communication between the management device and the scoring task scheduling device can be carried out not only through any encrypted network but also through other confidential data transmission methods.

Specifically, data transmission between the scoring management device and the test client device and between the scoring management device and the scoring task scheduling device can also be performed through a mobile storage device.

Fig. 2 shows the scoring module and process of the distributed automatic scoring system and method for the speaking test. The core modules of the distributed automatic scoring system and method for speaking test include: recording module 201 , speech processing module 202 , pronunciation evaluation module 203 , speech flow feature extraction module 204 and comprehensive scoring module 205 .

Recording module 201: detect and record the candidate's voice.

The recording module automatically detects the examinee's voice by using the voice endpoint detection method, or starts recording by receiving the examinee's manual operation command.

Speech processing module 202: process the examinee's speech signal, extract the acoustic feature, recognize the text corresponding to the speech, and obtain the time boundary of the acoustic unit.

The voice processing module is characterized in that it supports real-time online processing and offline batch processing. The voice recognition process in the voice processing module can be completed by using local services, or by accessing voice cloud services through the network.

Pronunciation evaluation module 203: according to the topic information and the output result of the speech processing module 202, analyze and evaluate different aspects of the user's pronunciation.

The feature of the pronunciation evaluation module is that the pronunciation evaluation includes at least four aspects: pronunciation accuracy of different units, prosody accuracy (such as stress, tone, intonation, etc.), pronunciation completeness and pronunciation fluency.

Speech flow feature extraction module 204: extract the comprehensive features of speech flow that reflect the examinee's spoken language level, including acoustic, grammatical and semantic features.

The speech flow feature extraction module is characterized in that part of the extracted features come from the pronunciation evaluation module 203 , part of it comes from topic information, and part of it comes from the speech processing module 202 .

The information about the test questions at least includes question types, question requirements, question content, examination focus, standard answer texts, and the like.

Scoring sub-module 205: using the comprehensive features of the speech flow to perform comprehensive scoring on the user's voice.

The scoring sub-module is characterized in that the speech flow comprehensive feature comes from the speech flow feature extraction module 204, and the comprehensive scoring algorithm is a combination of statistics and rules.

The scoring method of the distributed automatic scoring method of the speaking test comprises the following steps:

Step 1: Detect and record the examinee’s voice through the recording module;

Step 2, extracting speech features, using the acoustic model and language model to recognize the speech, and obtaining the corresponding text and its acoustic unit time boundary and likelihood score;

Step 3: Using the acoustic unit time boundary, likelihood score and topic-related knowledge to analyze and evaluate the speech, and obtain the scores of pronunciation accuracy, prosodic accuracy, pronunciation completeness and pronunciation fluency of different speech units;

Step 4, extracting from the output results of the speech processing module and the pronunciation evaluation module and the information of the test questions, the integrated features of the acoustic, grammatical and semantic aspects of the oral language level of the candidates are extracted;

Step five, based on the pre-trained model and designed rules, use the comprehensive features of speech flow to comprehensively score the oral level of candidates.

The distributed automatic scoring method of the speaking test is characterized in that the scoring modules are sequentially dependent, the input information of each scoring module comes from the calculation output of the previous module, and the calculation complexity of the speech processing module 202 is the highest.

Figure 3 shows the workflow of the distributed automatic scoring system for oral exams:

In the first step, candidates obtain the test questions and answer them as required. The test client device records the candidate’s voice, performs the first-stage scoring calculation on the candidate’s voice according to the question information and requirements, and sends the testee’s voice and scoring results to the scoring management system according to the data exchange standard. device;

In the second step, the scoring management device receives the examinee's answers and scoring results sent by the test client device, and analyzes the scoring results. If the scoring result is not the final scoring, it organizes the candidate's voice and scoring intermediate results into a scoring task according to the data exchange standard. After encryption Sent to scoring task scheduling device for further scoring;

In the third step, the scoring task scheduling device receives the scoring task sent by the scoring management device, analyzes the scoring task to obtain the scoring data and task priority, selects the required scoring module and algorithm according to the setting of the scoring module field, generates a scoring calculation program, and calculates the score according to the task. The priority is submitted to the scoring calculation device in the background for calculation;

In the fourth step, the scoring calculation device receives the scoring program sent by the scoring task scheduling device, executes the second stage of scoring, and sends the final scoring result to the scoring task scheduling device, and then the scoring task scheduling device forwards the scoring result to the scoring management device.

The key feature of the disclosed distributed automatic scoring system for oral exams is that the automatic scoring process can be divided into two stages: the first stage of scoring on the test client device and the second stage of scoring on the background scoring calculation device.

The key feature of the disclosed distributed automatic grading system for oral exams is that the automatic grading modules can be assigned to the first grading stage and the second grading stage respectively.

The key feature of the disclosed distributed automatic scoring system for oral exams is that the first-stage scoring is deployed on the test client device, and the second-stage scoring is deployed on the scoring calculation device.

The key feature of the disclosed distributed automatic scoring system for oral exams is that the second-stage scoring deployed on the scoring calculation device supports batch calculation of large-scale scoring tasks, and multiple scoring calculation tasks are executed independently and in parallel.

The key feature of the disclosed distributed automatic scoring system for oral exams is that the scoring calculation task in the second stage can simultaneously include the scoring of multiple questions of multiple examinees. The same scoring modules for different candidates and test questions in the same scoring calculation task can share the same scoring resources, reducing the waste of time and space for repeated resource loading.

The disclosed distributed automatic scoring system for the speaking test is also characterized in that it supports both large-scale offline batch scoring and small-scale online scoring.

The distributed automatic grading system for the speaking test is also characterized in that it supports small-scale online grading, that is, each grading task only processes the answer to one test question of one examinee, and directly feedbacks the grading after the test is completed.

The key feature of the disclosed distributed automatic scoring system for oral exams is that other scoring modules except the recording module can be dynamically scheduled and deployed according to the computing load and network load of the test client device and the scoring calculation device in the background.

The key feature of the disclosed distributed automatic scoring system for oral exams is that among the scoring modules deployed on the test client device, the recording module has the highest priority, ensuring that the execution of other modules will not affect the recording function.

According to the location where the scoring module is deployed, the distributed automatic scoring system for the speaking test has the following four scoring module deployment schemes:

Table 1 Four scoring module deployment schemes for the distributed automatic scoring system of the speaking test

Specifically, in solution 4, the scoring task no longer needs to be sent to the background scoring calculation device for calculation.

Figure 4 shows a schematic diagram of the scoring module deployment of scheme 1. The distributed automatic scoring system of the entire speaking test can be divided into two major modules: the front-end module and the back-end module.

The front-end module includes an examination client device and a scoring management device. A recording module and a voice processing module are deployed on the test client device.

The back-end module includes scoring task scheduling means and background scoring calculation means. The score calculation device in the background is equipped with a pronunciation evaluation module, a speech flow feature extraction module and a comprehensive scoring module.

Typically, in Solution 1, the recording module and voice processing module are deployed on the front-end module, and other modules are deployed on the back-end module.

The advantage of deployment scheme 1 is that the computing power of the test client device can be fully used to complete the voice processing function, which can reduce the pressure on the score calculation device in the background, and at the same time, it will not cause too heavy load on the test client device and affect the recording work.

The distributed automatic scoring system of the speaking test is also characterized in that the test client device can be connected to the voice cloud system through the network, and the cloud computing resources can be fully utilized.

The deployment plan 1 of the distributed automatic grading system for the speaking test is also characterized in that the speech recognition calculation of the speech processing module can be submitted to the speech cloud for completion, thereby reducing the computing load of the test client.

Fig. 5 is a schematic diagram of the organizational structure of the distributed automatic scoring system of the speaking test connected to the voice cloud system. The test client device 101 accesses the voice cloud system 105 through the network.

Powerful computing resources and accurate speech model resources are deployed on the speech cloud system 105 . The speech recognition process in the speech processing module consumes the most computing resources in the entire scoring process. By using the computing power and model resources of the speech cloud device, not only can the computing load of the test client be reduced, but also more accurate recognition results can be obtained.

In the case of scoring module deployment scheme 1, the test client device 101 first records the examinee's voice; then performs voice processing, sends the examinee's voice or the acoustic features of the examinee's voice to the voice cloud system for voice recognition, and obtains the voice recognition result, After completing other speech processing tasks, the topic information, examinee's speech and speech processing results are sent to the scoring management device 102, and the scoring management device 102 performs subsequent scoring processing.

The distributed automatic scoring system of the oral exam is characterized in that if the scoring equipment and the scoring management device in the background are in the same local area network, or the data transmission passes through the mobile storage device, the scoring in the distributed automatic scoring system of the oral exam The management means and scoring task scheduling means can be combined. The system is a distributed automatic scoring system for a streamlined oral exam.

Fig. 6 shows a structural schematic diagram of a distributed automatic scoring system environment for a simplified speaking test.

The distributed automatic grading system for the simplified speaking test includes a test client device 301 , a task management device 302 and a score calculation device 303 .

Examination client device 301 has the same function and configuration as module 101 in FIG. 1 .

The score calculating means 303 has the same function and configuration as the module 104 in FIG. 1 .

Task management device 302: Receive candidates' answers and scoring results, analyze the scoring results, use the intermediate scoring data and topic information to construct a scoring calculation program, submit scoring calculation tasks, and receive the final scoring results.

Specifically, the examinee's answers and scoring results on each test client device can be moved to the scoring management device by using a mobile storage device. The scoring management program on the scoring management device analyzes the scoring results, extracts the intermediate scoring data, examinee voice and question information, and connects with the scoring program of the required scoring module, aggregates multiple scoring tasks to generate batch scoring calculation tasks, and then submits them to the background The score calculation device performs the calculation.

The distributed automatic scoring system of the speaking test is also characterized in that it has a set of calculation fault tolerance mechanism to ensure the stability of scoring calculation. Specific measures include:

1) Configure backup computing equipment: configure redundant CPUs for the scoring computing equipment in the background to deal with unexpected hardware failures. If any scoring calculation equipment breaks down, the scoring task scheduling device automatically activates the standby scoring calculation equipment, resubmits the scoring task on the computer, and sends out an alarm message. The standby scoring calculation device is always in a waiting state, and the scoring task scheduling device can enable the device at any time. Usually one-tenth of the available CPU of the scoring computing device is set as a backup computing device.

2) Record the log of the calculation process: keep a log file for the scoring task of each question, from the beginning of the recording to the final scoring, record the execution of each step in each scoring sub-module, at least including the executed instructions, Data directory, execution time, execution status and other information.

3) Detect recording abnormalities: Before the exam officially starts, the recording module requires candidates to try recording. By analyzing the audio signal obtained from the candidate's recording device, detect the abnormality of the recording device, such as: no voice, too much volume or too low volume and other abnormal conditions will prompt candidates.

4) Detect abnormal scoring calculation tasks: the scoring task scheduling device of the back-end module monitors the calculation process of each scoring task, if any task calculation failure is found, if it is caused by a computing hardware problem, then enable the backup computing resource and resubmit the calculation task; otherwise, log an error message and issue a warning message.

5) Abnormal detection of data transmission: the test client device and scoring management device monitor the data transmission process, if any data transmission fails, retransmit at intervals, record error messages, and issue warning messages.

6) Data backup: Temporarily save the examinee's voice and the results of the first stage of scoring on the test client device until the data is successfully transmitted. The examinee's speech, scoring tasks and final scoring results are saved on the front-end scoring management device. The scoring task and the final scoring result are temporarily saved on the back-end scoring task scheduling device.

Certainly, the present invention also can have other multiple embodiments, without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and deformations according to the present invention, but these corresponding Changes and deformations should belong to the scope of protection of the appended claims of the present invention.

Claims (14)

1. the distributed automatic scoring system of a SET; Comprise one the scoring management devices manage some examination client terminal devices; Scoring management devices and at least one scoring task scheduling apparatus communicate; A scoring task scheduling apparatus is managed some score calculation devices, and the module that wherein is used for automatic scoring comprises:

Recording module is used for detecting and recording examinee's voice;

Speech processing module is used for examinee's voice signal is handled, and extracts acoustic feature, and the text that recognizing voice is corresponding obtains the time border of acoustic elements;

The pronunciation evaluation module is used for the result according to exercise question information and speech processing module output, and the different aspect of user pronunciation is analyzed and assessed;

The flow characteristic extracting module is used to extract the flow comprehensive characteristics that reflects examinee's spoken language proficiency, comprises the characteristic of acoustics, syntax and semantics;

Comprehensive grading module: utilize the flow comprehensive characteristics, user speech is carried out comprehensive grading; Wherein

Except that recording module, other grading module can be come the dynamic dispatching deployment according to the computational load and the offered load of the score calculation device on examination client terminal device and backstage.

2. the distributed automatic scoring system of SET as claimed in claim 1, wherein

The examination client terminal device; Be used for recording examinee's voice through said recording module; Through said local computing unit pre-service is carried out in examinee's answer; Calculate certain middle appraisal result, send to the scoring management devices to examinee's answer and middle appraisal result thereof through network communication unit or other modes;

The scoring management devices; Be used to obtain examinee's answer and middle appraisal result thereof from the examination client terminal device; Appraisal result and exercise question information structuring scoring task sends to the scoring task scheduling apparatus in the middle of utilizing, and receives the final appraisal result from the scoring task scheduling apparatus;

The scoring task scheduling apparatus; Be used to receive the scoring task that said scoring management devices sends; Resolve and extract score data, organize the score calculation program automatically, generate the score calculation task according to information such as exercise question information and the required modules of scoring; Submit to calculation task to the score calculation device through dispatching algorithm, obtain final appraisal result from the score calculation device at last;

The score calculation device is used to receive and carry out the scoring procedures that said scoring task scheduling apparatus sends, and sends to said scoring task scheduling apparatus to final appraisal result.

3. the distributed automatic scoring system of SET as claimed in claim 2, the client terminal device of wherein taking an examination can be disposed a plurality of grading module, is guaranteeing to utilize local cpu to accomplish certain scoring work under the impregnable situation of Recording Process; Said scoring work can be carried out with examination simultaneously, for avoiding that examination is had any impact, and also can be at the free time timing operation outside the examination.

4. the distributed automatic scoring system of SET as claimed in claim 2, the management devices of wherein marking disposes reliable mass-memory unit with the scoring task scheduling apparatus, generally can have high data transfer bandwidth; Said scoring task scheduling apparatus is safeguarded the calculation task formation of a prioritization in internal memory.

5. the distributed automatic scoring system of SET as claimed in claim 2; The network data transmission mode of wherein taking an examination between client terminal device, scoring management devices and the scoring task scheduling apparatus can use other data transfer mode to substitute, and all the other devices and workflow still remain unchanged; Consider from data confidentiality, transmit data and can carry out encryption.

6. the distributed automatic scoring system of SET as claimed in claim 4, the task scheduling algorithm of wherein said scoring task scheduling apparatus is: if task queue is discontented, then insert the formation relevant position to task according to its priority; Task priority is higher than some task priority in the formation else if, then is moved out to buffer area or external memory storage to the minimum task of priority, inserts the formation relevant position to this task; Otherwise store this task into buffer area or external memory storage; Regularly query task formation if having living space, then adds formation to the buffer area task by its priority; Each score calculation device of automatic regular polling, if there is CPU idle, the task of then from task queue, getting is carried out computing to this CPU.

7. the distributed automatic scoring system of SET as claimed in claim 4, but wherein said scoring task scheduling apparatus is submitted to the score calculation device to the calculation task that a plurality of scoring taskings constitute a batch processing together.

8. the distributed automatic scoring system of SET as claimed in claim 1, wherein the flow characteristic extracting module combines exam question information to extract the flow comprehensive characteristics according to the output result of speech processing module and pronunciation evaluation module; Said exam question information comprises exercise question type, exercise question requirement, item content, investigation emphasis and model answer text at least.

9. the distributed automatic scoring system of SET as claimed in claim 1, its automatic scoring process is divided into two stages: the phase one scoring is deployed on the examination client terminal device; The subordinate phase scoring is deployed on the score calculation device; Wherein: the phase one scoring comprises recording module.

10. the distributed automatic scoring system of SET as claimed in claim 1, its automatic scoring module is successively to rely on; Except that recording module, other modules can be on phase one scoring and subordinate phase be marked flexible allocation; Concrete scheme is: the phase one scoring also comprises speech processing module; The subordinate phase scoring is made up of pronunciation evaluation module, flow characteristic extracting module and comprehensive grading module.

11. the distributed automatic scoring system of SET as claimed in claim 9, the subordinate phase scoring that is deployed on the score calculation device supports the batch of extensive scoring task to calculate, and a plurality of score calculation task independent parallels are carried out; Each score calculation task of subordinate phase scoring can comprise the scoring of answer of a plurality of examinees' multiple tracks exercise question or a whole set of examination question.

12. the distributed automatic scoring method of a SET, wherein the concrete steps of scoring comprise:

Step 1, detect examinee's voice and record through recording module;

Step 2, extraction phonetic feature utilize acoustic model and language model, and voice are discerned, and obtain corresponding text and phoneme unit time border and likelihood score score;

Step 3, utilize phoneme unit time border and likelihood score score and exercise question relevant knowledge, voice are carried out analysis and evaluation, obtain pronunciation accuracy, rhythm accuracy, pronunciation integrity degree and the fluent degree of pronunciation etc. of different phonetic unit;

Step 4, integrated voice result, assessment result and exercise question information are extracted the acoustics of reflection examinee spoken language proficiency, the flow comprehensive characteristics of syntax and semantics aspect;

Step 5, based on the model and the designed regular of training in advance, use the flow comprehensive characteristics that examinee's spoken language proficiency is carried out comprehensive grading.

13. the distributed automatic scoring method of a SET specifically comprises the steps:

Step 1, examinee obtain exam question and answer on request; The examination client terminal device is recorded examinee's voice; With requiring examinee's voice are carried out the phase one score calculation according to exercise question information, send to the scoring management devices to the phase one appraisal result of examinee's voice and acquisition according to the self-defining data exchange standard;

Step 2, scoring management devices receive examinee's answer and the phase one appraisal result that the examination client terminal device sends; Resolve appraisal result; If appraisal result is not final scoring; Then be made into the scoring task to examinee's voice and scoring intermediate result and exercise question message block, send to the scoring task scheduling apparatus and carry out the subordinate phase scoring according to the self-defining data exchange standard;

Step 3, scoring task scheduling apparatus receive the scoring task that the scoring management devices sends; Resolve the scoring task and obtain score data and task priority; Select required grading module and algorithm according to being provided with of grading module territory; Generate the score calculation program, the score calculation device that is submitted to the backstage according to task priority calculates;

Step 4, score calculation device receive the scoring procedures that task scheduling apparatus sends, and carry out the subordinate phase scoring, send to the scoring task scheduling apparatus to final appraisal result, and the task scheduling apparatus of marking then is forwarded to the scoring management devices to appraisal result again.

14. the distributed automatic scoring system of SET as claimed in claim 1, said automatic scoring system comprises that a cover calculates fault tolerant mechanism, specifically comprises:

1) dispose subsequent use computational resource: for the CPU of backstage score calculation device configuring redundancy with reply burst hardware fault; If there is the score calculation device to break down, said task scheduling apparatus is launched subsequent use score calculation device CPU automatically, and resubmits the scoring task on this computing machine, and sends a warning; Subsequent use score calculation device is in waiting status always, and said task scheduling apparatus can be launched the CPU of this device at any time;

2) daily record of record calculation process: the scoring task of inscribing for per pass all keeps a journal file; Up to obtaining last scoring, write down the implementation status of each step in each grading module from the recording beginning, comprise the order of execution at least; Data directory, execution time and execution state information;

3) detect recording unusually: before examination formally began, said recording module required examinee's rehearsal sound.Through examinee's sound signal that analysis is obtained, it is unusual to detect sound-recording function, and prompting examinee adjustment;

4) detect the score calculation task abnormity: the computation process of said each scoring task of scoring task scheduling apparatus monitoring, if find to have the task computation failure,, then launch subsequent use computational resource, and resubmit task if the computing hardware problem causes; Otherwise misregistration information, information gives a warning;

5) detect data transmission exception: said examination client terminal device and scoring management devices monitor data transmission course, if data transmission fails is arranged, the some time transmits again at interval, and misregistration information, information gives a warning;

6) data backup: said examination client terminal device is preserved the result of user speech and phase one scoring temporarily, successfully transmits up to these data; Said scoring management devices is preserved examinee's voice, scoring task and final appraisal result; Said scoring task scheduling apparatus is preserved scoring task and final appraisal result temporarily.

Images