CN116258390B - A cognitive support quality evaluation method and system for teachers' online teaching feedback - Google Patents

Abstract

The invention relates to the field of computer information processing, and provides a cognitive support quality evaluation method and a cognitive support quality evaluation system for teacher online teaching feedback. The method comprises the following steps: (1) Constructing a cognitive support quality evaluation model, dividing the cognitive support characteristic types, and constructing a corresponding cognitive support quality evaluation model; (2) Determining a cognitive support feature recognition processing algorithm, extracting on-line teaching feedback text information of an excavating teacher by adopting a natural language processing technology, and realizing effective recognition and processing of information sharing, query answering, discussion guidance and evaluation guidance features; and (3) measuring and calculating the comprehensive evaluation result. The method and the system can comprehensively diagnose the cognitive support quality of the online teaching feedback of the teacher, identify the cognitive support characteristics of the online teaching feedback of the teacher, promote the teacher in the online teaching environment to provide high-quality teaching feedback, and further provide reference basis for the online teaching quality improvement.

Description

A cognitive support quality evaluation method and system for teachers' online teaching feedback

Technical Field

The present invention relates to the field of computer information processing, and in particular to a cognitive support quality evaluation method and system for teacher online teaching feedback.

Background technique

The important role of online teaching is becoming increasingly prominent. In the online teaching environment with time and space separation, teacher feedback can enhance students' understanding, construction and creation of knowledge, provide cognitive support for students, and is an indispensable force for improving the quality of online teaching. Conducting a study on the cognitive support quality evaluation of teacher online teaching feedback is conducive to accurately understanding the status of teacher online teaching feedback, which is of great significance to improving the effectiveness of online teaching.

At present, the research on the cognitive support quality of teachers' online teaching feedback is mainly based on status analysis and questionnaire surveys. There is a lack of a unified evaluation model, and there is a problem that the analysis data is mainly subjective data. The current difficulties in the evaluation of the cognitive support quality of teachers' online teaching feedback are: (1) There is a lack of comprehensive analysis of multi-dimensional and complex process data, which makes it difficult to fully reflect the cognitive support quality of teachers' online teaching feedback; (2) There is a lack of a unified evaluation model, which makes it difficult to standardize the evaluation focus and process methods; (3) The evaluation means and methods are subjective, and the validity of the data and the accuracy of the results are difficult to guarantee. It is difficult to achieve the scale and automation of the evaluation of the cognitive support quality of teachers' online teaching feedback, and it cannot fully meet the actual needs of online education in my country.

Summary of the invention

In view of the above defects or improvement needs of the prior art, the present invention provides a method and system for evaluating the quality of cognitive support for teachers' online teaching feedback, which aims to comprehensively diagnose the cognitive support quality of teachers' online teaching feedback, identify the cognitive support characteristics of teachers' online teaching feedback, and promote teachers in the online teaching environment to provide high-quality teaching feedback, thereby providing a reference basis for improving the quality of online teaching.

The purpose of the present invention is achieved through the following technical measures.

A cognitive support quality evaluation method for teachers' online teaching feedback includes the following steps:

(1) Construct a cognitive support quality evaluation model. Cognitive support characteristics are divided into four categories: information sharing, question answering, discussion guidance, and evaluation guidance, and a corresponding cognitive support quality evaluation model is constructed.

(2) Determine the cognitive support feature recognition and processing algorithm. In view of the massive process data generated by teachers’ online teaching feedback, natural language processing technology is used to extract and mine the text information of teachers’ online teaching feedback, so as to effectively identify and process the features of information sharing, question answering, discussion guidance, and evaluation guidance.

(3) Calculate the comprehensive evaluation results. Based on the feature recognition processing results, an evaluation score matrix is formed, the comprehensive evaluation results of cognitive support quality are calculated, and the comprehensive evaluation results are displayed in a visual way.

The present invention also provides a cognitive support quality evaluation system for teacher online teaching feedback, comprising the following modules:

Evaluation model module, used to establish a cognitive support quality evaluation model consisting of information sharing, question answering, discussion guidance, and evaluation guidance;

The basic information module is used to set the basic information of teachers, students and online teaching courses, including teacher nickname, teacher ID, teacher age, teacher gender, student nickname, student ID, student gender, student grade, course name, and course category;

The data mining module is used to collect and mine the process data related to teachers' online teaching feedback, including teacher ID, student ID, behavior operation, related content, and time;

The semantic analysis module is used to analyze the semantic information of the text content related to the teacher's online teaching feedback, including text content annotation and cognitive support index matching calculation;

Model training and testing module, used to train and test the cognitive support feature processing model, including model training, loss function value calculation and accuracy calculation;

Evaluation result calculation module, used to calculate the score, weight value and comprehensive evaluation value of each evaluation indicator;

The visualization presentation module is used to retrieve data from the above modules and visualize the comprehensive evaluation results of cognitive support quality.

The beneficial effects of the present invention are:

Combining the theories and characteristics of multiple disciplines such as pedagogy and data science, a cognitive support quality evaluation model for teachers' online teaching feedback is constructed, and a cognitive support feature identification and processing algorithm based on process data is proposed to overcome several defects of the existing cognitive support quality evaluation, realize the effective identification of cognitive support features, modeling analysis and accurate evaluation of cognitive support quality, and promote the further development and application of relevant theories and methods for research on teachers' online teaching feedback, which will help improve the effectiveness of online teaching and promote the development of online education.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a general flow chart of a cognitive support quality evaluation method according to an embodiment of the present invention.

FIG. 2 is a flowchart of establishing a cognitive support feature recognition processing algorithm according to an embodiment of the present invention.

FIG. 3 is a schematic diagram of a cognitive support feature prediction result according to an embodiment of the present invention.

FIG. 4 is a schematic diagram of the correlation relationship between cognitive support evaluation index scores according to an embodiment of the present invention.

FIG5 is a radar chart of the teacher group cognitive support quality evaluation results according to an embodiment of the present invention.

FIG6 is a bar chart of cognitive support quality evaluation results for teacher groups of different genders according to an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

As shown in FIG1 , an embodiment of the present invention provides a cognitive support quality evaluation method for teacher online teaching feedback, comprising the following steps:

(1) Construct a cognitive support quality evaluation model, divide cognitive support characteristics into four categories: information sharing, question answering, discussion guidance, and evaluation guidance, and construct a corresponding cognitive support quality evaluation model. The details are as follows:

(1-1) Classification of cognitive support characteristics. Based on the conceptual connotation of cognitive support of teaching feedback and the characteristics of the online teaching environment, the cognitive support characteristics of teachers' online teaching feedback are divided into four categories: information sharing, question answering, discussion guidance, and evaluation guidance.

(1-2) Evaluation model construction. According to the characteristics of cognitive support, four corresponding evaluation indicators are formed to construct a cognitive support quality evaluation model for teachers' online teaching feedback. The specific contents of each evaluation indicator included in this evaluation model are as follows:

Information sharing indicators: used to evaluate whether teachers provide and share ideas or resources related to learning topics or tasks;

Question answering indicator: used to evaluate whether the teacher explains and answers the questions raised by students;

Discussion guidance indicator: used to evaluate whether the teacher guides the topic and direction of students' participation in online discussions;

Evaluation and guidance indicators: used to evaluate whether teachers give evaluations on students’ learning methods, learning outcomes, etc., and provide guidance suggestions.

(2) Determine the cognitive support feature recognition and processing algorithm. In view of the massive process data with complex structure and diverse types generated by teachers' online teaching feedback, natural language processing technology is used to extract and mine the text information of teachers' online teaching feedback, and realize the effective recognition and processing of information sharing, question answering, discussion guidance, and evaluation guidance features. The details are as follows:

(2-1) Multi-source process data collection. Focusing on the basic situation of teachers, we collected information on teachers’ gender, age, teaching experience, and teaching grade; focusing on teachers’ online teaching feedback, we collected text content and behavior log data related to teachers’ feedback.

(2-2) Cognitive support feature recognition and processing. Integrate the semantic analysis method and the meaning of the evaluation index, establish a specific problem model, build a semantic information processing model, and conduct model training, optimization, and testing through text content extraction, vocabulary sentence representation, and text semantic matching to determine the cognitive support feature recognition and processing algorithm and implement cognitive support feature recognition and processing. As shown in Figure 2, the details are as follows:

(2-2-1) Problem model establishment. Based on the information sharing, question answering, discussion guidance, and evaluation guidance indicators included in the evaluation model, key text information is extracted from process data such as text content. The degree of matching between key text information and indicators is used as the main feature basis for cognitive support feature recognition processing. The cognitive support feature recognition processing problem is determined to be a text multi-label classification problem, which can be expressed as the mathematical model shown in F(content _i ).

Among them, f represents the function of calculating the matching degree between text content and cognitive support indicators, P ₁ , P ₂ , P ₃ , and P ₄ represent the meanings of information sharing, question answering, discussion guidance, and evaluation guidance indicators, respectively, and content _i represents the content of the i-th feedback text. The value range of f is defined as 0 or 1, where 0 means that the meaning of the indicator does not match the feedback content, and 1 means that the meaning of the indicator matches the feedback content.

(2-2-2) Analysis model construction. In response to the above-mentioned text multi-label classification problem, natural language processing technology and deep learning methods are used to combine BERT with TextCNN, BiLSTM, TextRCNN, and DPCNN into multiple analysis models to complete the key information extraction, content understanding and topic classification processes, and perform cognitive support feature recognition processing. The analysis models include BERT, BERT+TextCNN, BERT+BiLSTM, BERT+TextRCNN, and BERT+DPCNN.

(2-2-3) Determination of recognition processing algorithm. BERT, BERT+TextCNN, BERT+BiLSTM, BERT+TextRCNN, and BERT+DPCNN were trained and tested respectively. The prediction effect of each model was tested and compared using the model prediction accuracy index Accuracy (i.e. the proportion of samples whose predicted results are consistent with the actual results) and the loss function value L. The model with the best effect was used as the cognitive support feature recognition processing algorithm for teachers' online teaching feedback.

Accuracy = (TP ₁ +TP ₂ +TP ₃ +TP ₄ +TP ₅ )/N

Where Accuracy represents the classification accuracy, TP ₁ , TP ₂ , TP ₃ , TP ₄ , TP ₅ represent the number of samples accurately predicted as Label 1, Label 2, Label 3, Label 4, Label 5, respectively, and N represents the total number of predicted samples. L represents the value of the model loss function, K represents the number of classification labels, y _ij represents the true probability that the i-th sample belongs to the j-th class label (the probability is 1 if it belongs to the j-th class label, otherwise it is 0), and p _ij represents the model's predicted probability that the i-th sample belongs to the j-th class label.

The accuracy results of cognitive support feature analysis of each model are shown in Table 1. Specifically, the BERT+DPCNN model has the highest accuracy, reaching 0.791, and is used as the cognitive support feature recognition processing algorithm model. The prediction results of the model for various features such as information sharing, question answering, discussion guidance, and evaluation guidance are shown in Figure 3.

Table 1. Accuracy of cognitive support feature analysis of each model

(3) Calculate the comprehensive evaluation results. Based on the feature recognition processing results, form an evaluation score matrix, calculate the comprehensive evaluation results of cognitive support quality, and use radar chart visualization to display the comprehensive evaluation results. The details are as follows:

(3-1) Determine the method for calculating the index evaluation score. Use the index dimensionless processing method to standardize and dimensionlessly process the eigenvalues, calculate the scores l _ij of the online teaching feedback provided by teachers to each student in terms of information sharing, question answering, discussion guidance, and evaluation guidance, and form the evaluation score matrix L for cognitive support quality evaluation;

Where l _ij represents the score of the teacher's feedback to student i in indicator j, e _ij represents the characteristic value of the teacher's feedback to student i in indicator j, n represents the number of students, and m is the number of indicators.

(3-2) Determine the indicator weight. Use the entropy method to calculate the entropy value H _j of each evaluation indicator based on the score of each indicator l _ij , and finally obtain the weight w _i of each indicator based on the entropy value;

(3-3) Calculate the comprehensive evaluation value. Apply the linear weighted method to calculate the comprehensive evaluation value of cognitive support quality based on the indicator evaluation score l _ij and the indicator weight w _i .

(3-4) Visualization of results. Different visualization techniques and graphics were used to display the overall performance and quality difference analysis of teachers’ online teaching feedback quality evaluation. Correlation analysis was used to process the scores of each evaluation indicator in the cognitive support of teachers’ feedback, and the correlation between the scores of each evaluation indicator of cognitive support was visualized (as shown in Figure 4). Radar charts and bar charts were used to process the results of the cognitive support quality evaluation of the teacher group, and the overall performance of teachers’ cognitive support (as shown in Figure 5) and the performance of teachers of different genders (as shown in Figure 6) were visualized.

The embodiment of the present invention also provides a cognitive support quality evaluation system for teacher online teaching feedback, including the following modules:

Evaluation model module, used to establish a cognitive support quality evaluation model consisting of information sharing, question answering, discussion guidance, and evaluation guidance;

The basic information module is used to set the basic information of teachers, students and online teaching courses, including teacher nickname, teacher ID, teacher age, teacher gender, student nickname, student ID, student gender, student grade, course name, and course category;

The data mining module is used to collect and mine the process data related to teachers' online teaching feedback, including teacher ID, student ID, behavior operation, related content, and time;

The semantic analysis module is used to analyze the semantic information of the text content related to the teacher's online teaching feedback, including text content annotation and cognitive support index matching calculation;

Model training and testing module, used to train and test the cognitive support feature processing model, including model training, loss function value calculation and accuracy calculation;

Evaluation result calculation module, used to calculate the score, weight value and comprehensive evaluation value of each evaluation indicator;

The visualization presentation module is used to retrieve data from the above modules and visualize the comprehensive evaluation results of cognitive support quality.

The contents not described in detail in this specification belong to the prior art known to those skilled in the art.

It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims (3)

1. A cognitive support quality evaluation method for teacher-oriented online teaching feedback is characterized by comprising the following steps:

(1) Constructing a cognitive support quality evaluation model, dividing cognitive support characteristics into four categories of information sharing, query solution, discussion guidance, evaluation guidance, and constructing a corresponding cognitive support quality evaluation model; the specific process is as follows:

the method comprises the following steps of (1-1) dividing cognitive support characteristics into four types of information sharing, query solution, discussion guidance and evaluation guidance according to the cognitive support concept connotation of teaching feedback and the on-line teaching environment characteristics;

(1-2) constructing an evaluation model, forming four corresponding evaluation indexes according to the cognitive support characteristics, and constructing a cognitive support quality evaluation model for online teaching feedback of a teacher, wherein the evaluation model comprises the following specific contents of each evaluation index:

information sharing index: for evaluating whether a teacher provides and shares views or resources related to a learning topic or task;

query and answer index: for evaluating whether the teacher interprets and answers questions posed by the student;

discussion of guide indicators: the method is used for evaluating whether the teacher guides the topics and directions of the students participating in the online discussion;

evaluation guide index: the method is used for evaluating whether a teacher gives out evaluation for the learning method and learning effect performance of students and giving out guiding advice;

(2) Determining a cognitive support feature recognition processing algorithm, extracting and mining text information of the online teaching feedback of a teacher by adopting a natural language processing technology aiming at massive process data generated by the online teaching feedback of the teacher, and realizing effective recognition and processing of information sharing, query solution, discussion guidance and evaluation guidance features; the specific process is as follows:

(2-1) multisource process data acquisition, namely acquiring sex, age, teaching age and teaching grade information of a teacher around the basic condition of the teacher; collecting text content and behavior log data related to teacher feedback around the relevant conditions of the teacher online teaching feedback;

(2-2) cognition support feature recognition processing, namely, establishing a specific problem model by integrating a semantic analysis method and evaluation index meanings, establishing a semantic information processing model, performing model training, optimization and inspection by text content extraction, vocabulary statement representation and text semantic matching, and determining a cognition support feature recognition processing algorithm to realize cognition support feature recognition processing, wherein the method comprises the following steps of:

(2-2-1) establishing a question model, extracting key text information from process data based on information sharing, query solving, discussion guiding and evaluation guiding indexes contained in the evaluation model, and determining that the cognitive support feature recognition processing question is a text multi-label classification question based on the matching degree of the key text information and the indexes as main feature basis of the cognitive support feature recognition processing, wherein the text multi-label classification question can be expressed as F (content) _i )

Wherein f represents a function for calculating the matching degree of text content and cognitive support indexes, and P ₁ ，P ₂ ，P ₃ ，P ₄ Index meanings, content of information sharing, query solution, discussion guidance and evaluation guidance are respectively expressed _i The ith feedback text content is represented, the value range of f is defined as 0 or 1,0 represents that the index meaning is not matched with the feedback content, and 1 represents that the index meaning is matched with the feedback content;

(2-2-2) constructing analysis models, combining BERT and TextCNN, biLSTM, textRCNN, DPCNN into a plurality of analysis models by using a natural language processing technology and a deep learning method, completing the processes of key information extraction, content understanding and topic classification, and performing cognitive support feature recognition processing;

(2-2-3) determining by an identification processing algorithm, and checking and comparing the prediction effect of each model by model training and checking by using an Accuracy index Accurcy of model prediction and a loss function value L; the model with the best effect is used as a cognitive support characteristic recognition processing algorithm for the online teaching feedback of a teacher;

Accuracy＝(TP ₁ +TP ₂ +TP ₃ +TP ₄ +TP ₅ )/N

wherein Accuracy represents classification Accuracy, namely sample proportion of predicted result to actual result, TP ₁ ,TP ₂ ,TP ₃ ,TP ₄ ,TP ₅ Respectively, the number of samples accurately predicted as Label 1, label 2, label 3, label 4, label 5, N is the total number of samples predicted, L is the model loss function value, K is the classification Label number, y _ij Representing the true probability that the ith sample belongs to the jth class of labels, wherein the probability is 1 when the ith sample belongs to the jth class of labels, otherwise, the probability is 0, and p _ij Representing the prediction probability of the model for the ith sample belonging to the jth class of labels;

(3) And measuring and calculating the comprehensive evaluation result, forming an evaluation score matrix based on the characteristic recognition processing result, measuring and calculating the comprehensive evaluation result of the cognitive support quality, and displaying the comprehensive evaluation result in a visual mode.

2. The method for evaluating the cognitive support quality for the online teaching feedback of the teacher according to claim 1, wherein the specific process of measuring and calculating the comprehensive evaluation result in the step (3) is as follows:

(3-1) determining an index evaluation score measuring and calculating method, carrying out standardization and dimensionless processing on characteristic values by using an exponential dimensionless processing method, and calculating the score l of each index of online teaching feedback provided by teachers for each student in information sharing, query answering, discussion guidance and evaluation guidance _ij Forming an evaluation score matrix L for evaluating the cognitive support quality;

wherein l _ij Score value e representing feedback of teacher to student i at index j _ij The characteristic value of feedback of a teacher to a student i in an index j is represented, n represents the number of students, and m represents the number of indexes;

(3-2) determining index weights according to the index scores l by adopting an entropy method _ij Calculating entropy value H of each evaluation index _j Finally, obtaining each index weight w according to the entropy value _i ；

(3-3) measuring and calculating the comprehensive evaluation value, applying a linear weighting method, and evaluating the score l according to the index _ij And index weight w _i Calculating a comprehensive evaluation value of the cognitive support quality;

and (3-4) visually displaying the result, processing the evaluation score and the comprehensive evaluation value of each index of the cognitive support quality by using a radar chart, and displaying the comprehensive evaluation result in a visual mode.

3. A teacher-oriented online teaching feedback cognitive support quality evaluation system, which is characterized by being used for realizing the cognitive support quality evaluation method in any one of claims 1-2, comprising:

the evaluation model module is used for establishing a cognitive support quality evaluation model consisting of information sharing, query solution, discussion guidance and evaluation guidance;

the basic information module is used for setting basic information of teachers, students and online teaching courses, and comprises teacher nicknames, teacher IDs, teacher ages, teacher sexes, student nicknames, student IDs, student sexes, grades of students, course names and course categories;

the data mining module is used for collecting and mining teacher online teaching feedback related process data, including teacher ID, student ID, behavior operation, related content and time;

the semantic analysis module is used for analyzing semantic information of text content related to online teaching feedback of teachers, and comprises text content labeling and cognitive support index matching degree calculation;

the model training and checking module is used for training and checking the cognition support characteristic processing model and comprises model training, loss function value calculation and accuracy calculation;

the evaluation result calculation module is used for calculating the score, the weight value and the comprehensive evaluation value of each evaluation index;

and the visual presentation module is used for retrieving data from the modules and visually presenting the comprehensive evaluation result of the cognitive support quality.