CN110321291A - Test cases intelligent extraction system and method - Google Patents
Test cases intelligent extraction system and method Download PDFInfo
- Publication number
- CN110321291A CN110321291A CN201910627829.7A CN201910627829A CN110321291A CN 110321291 A CN110321291 A CN 110321291A CN 201910627829 A CN201910627829 A CN 201910627829A CN 110321291 A CN110321291 A CN 110321291A
- Authority
- CN
- China
- Prior art keywords
- test case
- test
- test cases
- cases
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 458
- 238000000605 extraction Methods 0.000 title claims abstract description 27
- 238000000034 method Methods 0.000 title abstract description 28
- 238000013528 artificial neural network Methods 0.000 claims abstract description 19
- 230000000306 recurrent effect Effects 0.000 claims abstract description 19
- 238000001914 filtration Methods 0.000 claims abstract description 17
- 238000004364 calculation method Methods 0.000 claims description 20
- 238000004422 calculation algorithm Methods 0.000 claims description 13
- 230000008859 change Effects 0.000 claims description 11
- 238000013461 design Methods 0.000 abstract description 16
- 238000009825 accumulation Methods 0.000 abstract description 4
- 238000012216 screening Methods 0.000 abstract description 4
- 230000000694 effects Effects 0.000 abstract 1
- 230000006870 function Effects 0.000 description 25
- 230000006399 behavior Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000003058 natural language processing Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 230000011218 segmentation Effects 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 5
- 230000001537 neural effect Effects 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 101100481876 Danio rerio pbk gene Proteins 0.000 description 1
- 101100481878 Mus musculus Pbk gene Proteins 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Prevention of errors by analysis, debugging or testing of software
- G06F11/3668—Testing of software
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention provides a kind of test cases intelligent extraction system and methods.The system comprises: information module is for storing various information；Policy module is according to the information stored in information module, test cases and program, problem, demand and the model of user feedback are established respectively, test cases, which is generated, using depth collaborative filtering and recurrent neural network recommends sequence, it determines the similarity of each test cases, generates test cases recommendation results；Data module screens test cases and is sorted according to the essential information and its similarity of test cases, determines consequently recommended test cases；Display module shows consequently recommended test cases, receives field feedback.The present invention can sufficiently promote the effect in test cases library, promote the accumulation and multiplexing of test assets, and tester is helped to carry out Test Case Design, reduce Test Case Design and omit, while reducing the time of tester's screening test case.
Description
Technical Field
The invention relates to the technical field of computer software, in particular to an intelligent test case extraction system and method.
Background
A test case is a set of test inputs, execution conditions, and expected results tailored for a particular purpose to test a certain program path or verify that a certain requirement is met. The importance of the test case is mainly shown in the following aspects: the test cases form the basis for designing and formulating the test process; the depth of the test is proportional to the number of the test cases, and the time arrangement of each stage of the test period can be more accurately estimated according to the comprehensive and refined test cases; the type of test design and development, and the resources required, are largely controlled by the test case.
The test cases are mainly designed by testers according to the content of the project or selected from an inventory test case library to be subjected to new data combination. Because the testing personnel do not know the contents of the project or are unfamiliar with the inventory testing cases, the situations of missing of the testing cases, insufficient matching with the requirements of the project and the like are easily caused, and the inventory testing case assets cannot be fully utilized. On the other hand, test case design omissions tend to result in the program changes after project delivery not being adequately tested. Currently, there is no method and system for extracting test cases according to project contents and an existing test case library.
Disclosure of Invention
In order to solve the above problem, an embodiment of the present invention provides an intelligent test case extraction system, where the system includes: the system comprises an information module, a strategy module, a data module and a display module;
the information module is used for storing a test case library, a system program node flow chart, a system test question library, project requirement function description information and user test case selection behavior information;
the strategy module respectively establishes a test case and a model of a program, a problem, a demand and user feedback according to the information stored in the information module, generates a test case recommendation sequence by using a depth collaborative filtering algorithm and a recurrent neural network, determines the similarity of each test case in the test case recommendation sequence, and generates a test case recommendation result;
the data module screens and sorts the test cases in the test case recommendation result according to the basic information and the similarity of the test cases in the test case recommendation result to determine the final recommended test case;
the display module is used for displaying the final recommended test case and receiving user feedback information.
Optionally, in an embodiment of the present invention, the policy module includes a test case and program model unit, and is configured to perform text classification on text descriptions of test cases in the test case library, match the text classification of the test cases with text descriptions of node flow charts of the system program, determine a matching relationship between the test cases and the program, and obtain the test cases and the program model.
Optionally, in an embodiment of the present invention, the policy module includes a test case and problem model unit, and is configured to perform text classification on the text descriptions of the test cases in the test case library, perform text classification on the text descriptions of the test problems in the system test problem library, match the text classification of the test cases with the text classification of the test problems, determine a matching relationship between the test cases and the problems, and obtain the test cases and the problem models.
Optionally, in an embodiment of the present invention, the policy module includes a test case and requirement model unit, and is configured to perform text classification on text descriptions of test cases in the test case library, perform text classification on text descriptions of software requirement specifications in the project requirement function description information, match the text classification of the test cases with the text classification of the software requirement specifications, determine a matching relationship between the test cases and requirements, and obtain the test cases and the requirement model.
Optionally, in an embodiment of the present invention, the policy module includes a test case and user feedback model unit, and is configured to perform text classification on text descriptions of test cases in the test case library, analyze user feedback behaviors in the user test case selection behavior information, obtain user feedback features, match the text classification of the test case with the user feedback features, determine a matching relationship between the test case and the user feedback features, and obtain the test case and the user feedback model.
Optionally, in an embodiment of the present invention, the policy module includes a policy calculation unit, configured to generate a test case recommendation sequence by using a deep collaborative filtering algorithm and a recurrent neural network according to the test case and a model of a program, a problem, a requirement, and user feedback, determine a similarity of each test case in the test case recommendation sequence, and generate a test case recommendation result.
Optionally, in an embodiment of the present invention, the policy calculation unit determines the similarity of each test case in the test case recommendation sequence according to a test case similarity formula, where the test case similarity formula is:
wherein,is a time decay function, alpha is a regulation factor, ti、tjRespectively representing the time of selecting the test case i and the test case j by the user, NiRepresenting the number of users, N, selecting test case ijRepresenting the number of users who selected test case j.
Optionally, in an embodiment of the present invention, the data module is further configured to fuse and rearrange the test case recommendation results by using a recurrent neural network according to the program change information, generate an online recommendation result, filter and sort the test case recommendation result and the test cases in the online recommendation result, and determine a final recommended test case.
Optionally, in an embodiment of the present invention, the data module sorts the test cases in the test case recommendation result according to a sorting formula, where the sorting formula is:
wherein N isuIs a set of user-selected test cases, S (j, K) is a set of K test cases most similar to test case j, WijIs the similarity of the test cases j and i,is the probability that the user u selects the test case i, beta is a weight factor, and beta is more than or equal to 0 and less than or equal to 1.
The embodiment of the invention also provides an intelligent test case extraction method, which utilizes the intelligent test case extraction system to carry out intelligent test case extraction.
The invention can fully promote the function of the test case library, promote the accumulation and reuse of test assets, help the testers to design the test cases, reduce the omission of the design of the test cases and reduce the time for the testers to screen the test cases.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive labor.
Fig. 1 is a schematic structural diagram of an intelligent test case extraction system according to an embodiment of the present invention;
FIG. 2 is a block diagram of a policy module according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a test case and program model unit according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a test case and problem model unit according to an embodiment of the present invention:
FIG. 5 is a schematic structural diagram of a test case and requirement model unit according to an embodiment of the present invention:
FIG. 6 is a schematic diagram of a recurrent neural network and deep collaborative filtering according to an embodiment of the present invention:
FIG. 7 is a flowchart illustrating generation of a recommended test case set according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides an intelligent test case extraction system and method.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic structural diagram of an intelligent test case extraction system according to an embodiment of the present invention, where the system includes: the system comprises an information module 1, a strategy module 2, a data module 3 and a display module 4;
the information module 1 is used for storing a test case library, a system program node flow chart, a system test question library, project requirement function description information and user test case selection behavior information;
the strategy module 2 respectively establishes a test case and a model of a program, a problem, a demand and user feedback according to the information stored in the information module, generates a test case recommendation sequence by using a depth collaborative filtering algorithm and a recurrent neural network, determines the similarity of each test case in the test case recommendation sequence, and generates a test case recommendation result;
the data module 3 screens and sorts the test cases in the test case recommendation result according to the basic information and the similarity of the test cases in the test case recommendation result to determine the final recommended test case;
the display module 4 is used for displaying the final recommended test case and receiving user feedback information.
In this embodiment, the system of the present invention adds a natural language processing method and a deep learning method of a recurrent neural network on the basis of a collaborative filtering recommendation algorithm, effectively combines the respective advantages of the collaborative filtering method and the deep learning method, provides more accurate test case recommendation for a user, improves the efficiency of test case design to a certain extent, and saves the time cost of test case design.
As an embodiment of the invention, the strategy module comprises a test case and program model unit, which is used for performing text classification on the text description of the test case in the test case library, matching the text classification of the test case with the text description of the system program node flow chart, and determining the matching relation between the test case and the program to obtain the test case and the program model.
As an embodiment of the invention, the strategy module comprises a test case and problem model unit, which is used for performing text classification on the text description of the test case in the test case library, performing text classification on the text description of the test problem in the system test problem library, matching the text classification of the test case with the text classification of the test problem, and determining the matching relation between the test case and the problem to obtain the test case and the problem model.
As an embodiment of the invention, the strategy module comprises a test case and demand model unit, which is used for performing text classification on the text description of the test case in the test case library, performing text classification on the text description of the software demand specification in the project demand function description information, matching the text classification of the test case with the text classification of the software demand specification, and determining the matching relation between the test case and the demand to obtain the test case and the demand model.
As an embodiment of the invention, the strategy module comprises a test case and user feedback model unit, which is used for performing text classification on the text description of the test case in the test case library, analyzing the user feedback behavior in the user test case selection behavior information to obtain the user feedback characteristics, matching the text classification of the test case with the user feedback characteristics, and determining the matching relationship between the test case and the user feedback characteristics to obtain the test case and the user feedback model.
As an embodiment of the invention, the strategy module comprises a strategy calculation unit which is used for generating a recommendation sequence of the test cases by utilizing a depth collaborative filtering algorithm and a recurrent neural network according to the test cases and models of programs, problems, requirements and user feedback, determining the similarity of each test case in the recommendation sequence of the test cases and generating a recommendation result of the test cases.
In this embodiment, the policy calculation unit determines the similarity of each test case in the test case recommendation sequence according to a test case similarity formula, where the test case similarity formula is:
wherein,is a time decay function, alpha is a regulation factor, ti、tjRespectively representing the time of selecting the test case i and the test case j by the user, NiRepresenting the number of users, N, selecting test case ijRepresenting the number of users who selected test case j.
As an embodiment of the present invention, the data module is further configured to fuse and rearrange the test case recommendation results by using a recurrent neural network according to the program change information, generate an online recommendation result, and screen and sort the test case recommendation result and the test cases in the online recommendation result to determine a final recommended test case.
As an embodiment of the present invention, the data module sorts the test cases in the test case recommendation result according to a sorting formula, where the sorting formula is:
wherein N isuIs a set of user-selected test cases, S (j, K) is a set of K test cases most similar to test case j, WijIs the similarity of the test cases j and i,is the probability that the user u selects the test case i, beta is a weight factor, and beta is more than or equal to 0 and less than or equal to 1.
The invention overcomes the defects in the prior art, and provides a system for recommending test cases for testers by using a natural language processing method and a deep learning method based on a recurrent neural network. The invention does not depend on the experience of testers and the particularity of the software system, and can be universally applied to the test of all iteratively developed software systems. The business requirements and the test problems are associated with the test cases through natural language processing, so that an association model of the test cases and the test problems and the test cases and the requirement items is established, the test asset function of the test cases is fully exerted, the hidden features of the test cases and the hidden features of users are mined through a recurrent neural network method, the test cases with more comprehensive and higher relevancy are recommended for the testers, an efficient solution is provided for the design of the test cases in the iterative development process, and the test risk caused by insufficient experience of the testers is made up.
In an embodiment of the present invention, as shown in fig. 1, the information module 1 mainly stores information such as a test case library, a system program node flow chart, a system test question library, a project requirement function description, and a user test case selection behavior. The test case library is a test case library of the system full function and is a test case set compiled by testers in history. The system program node flow chart refers to a flow chart between system programs and program and between program internal components generated by a code static analysis method. The system test question library refers to test question information of system stock, and comprises information of question correlation test cases, question descriptions, question proposers, question resolvers, question types and the like. The project requirement function description refers to the requirement description of the project and the implementation information of the function. The user test case selection behavior refers to the selection of the test case by the user under different project and function implementations.
The strategy module 2 mainly provides decision basis for generation of a result set of recommended test cases in a model calculation mode. As shown in fig. 2, the module includes a test case-program model unit 21, a test case-problem model unit 22, a test case-requirement model unit 23 and a test case-user feedback model unit 24, and learns test case hidden features and user hidden features, i.e. time sequence information hidden in the above data, from a test case, program, test problem and project requirement and user feedback feature matrix through a strategy calculation unit 25. Since in practice the user often prefers to select a new test case and it is also important to correct the deviations to take into account the age of the test case and the age of the user, the system converts the test case selection prediction problem into a sequence prediction problem by means of the strategy calculation unit 25.
The test case-program model unit 21 mainly establishes a many-to-many matching relationship between the flow chart between the system program and the program, and between the flow chart between the program internal components and the test case. As shown in fig. 3, the test cases 1-211, 2-212, and N-213 represent a series of test cases formulated by the tester according to the content of the project, and the programs 1-215, 2-216, and N-217 represent a program set for implementing a specific function of the project.
One test case can correspond to a plurality of programs, and different test cases can also correspond to the same program, so that the test case-program model not only establishes the mapping relation between the test cases and the programs, but also provides support for the integrity of the test. The manner of establishing the mapping relationship between the test case and the program may be as follows. Firstly, preprocessing the text description of the testing case stored in the testing case library, removing words which are frequently appeared in the text description of the testing case and are meaningless for text classification, then carrying out word segmentation processing on the text description of the testing case according to a certain strategy, and automatically distributing predefined class labels to a result set after word segmentation processing according to the content or theme of a given document, thereby realizing text classification of the text description of the testing case, then matching with the text description of a system program node flow chart, and finally obtaining a testing case-program matching relation 214.
The test case-problem model unit 22 mainly performs text classification on the test problems by a natural language processing method, and constructs an association relationship between the problem description semantics and the test cases. As shown in FIG. 4, the test questions 1-221, 2-222, and N-223 represent program defects discovered by the tester during the project testing process, and the test cases 1-225, 2-226, and N-227 represent a series of test cases formulated by the tester according to the project content. Firstly, preprocessing the text description of the test problem stored in the test problem library, removing words which are frequently appeared in the text description of the test problem and are meaningless for text classification, then carrying out word segmentation processing on the text description of the test problem according to a certain strategy, and automatically distributing predefined class labels to a result set after word segmentation processing according to the content or theme of a given document, thereby realizing text classification of the text description of the test problem, and then matching with the text classification of the test case to finally obtain a test problem-case matching relation 224. Therefore, the test case-problem model is a semantic model for adding problem and test case combinations to the test cases from the perspective of historical problems, and many-to-many association relations between the test problems and the test cases can be constructed through the semantic model.
The test case-requirement model unit 23 is mainly used for constructing the association relationship between the requirement description and the test case by a natural language processing-text summarization method. As shown in FIG. 5, the requirements 1-231, 2-232 and N-233 represent specific requirement information of the project, and the test cases 1-235, 2-236 and N-237 represent a series of test cases formulated by the tester according to the content of the project. Firstly preprocessing a software requirement specification, removing words which are frequently appeared in the software requirement specification and are meaningless for text classification, then performing word segmentation processing on the software requirement specification according to a certain strategy, automatically distributing predefined class labels to a result set after word segmentation processing according to the content or theme of a given document, thereby realizing text classification of the software requirement specification, then matching with test case text classification, and finally obtaining a requirement-test case matching relation 234. The test case-requirement model ensures semantic association of test cases with project requirements and function implementation content.
The test case-user feedback model unit 24 mainly analyzes the user's behavior and constructs the user's characteristics according to the actual results selected by the user feedback cases. Selecting the test case of the history of the testers according to certain keywords, such as the test case selection rate, the test case recent selection time, the test case selection preference and the like, screening out the features with high occurrence frequency, and then matching the features with the test case text classification, thereby establishing the incidence relation between the features and the test cases.
According to the test case-program model unit 21, the test case-problem model unit 22, the test case-demand model unit 23 and the test case-user feedback model unit 24, the strategy calculation unit 25 adopted by the system is a deep collaborative filtering algorithm, a recursive neural network is used as an unsupervised learning method and a collaborative filtering algorithm for training, the hidden features of the test case and the hidden features of the user are automatically learned from the test case, program, problem and demand and user feedback feature matrixes, the specific test case is subjected to differentiation processing, the similarity calculation of the test case is performed according to the attributes and features of the model, and the test case candidate set with high correlation is generated.
FIG. 6 is a diagram illustrating a recurrent neural network-depth collaborative filtering in an embodiment of the present invention. As shown in FIG. 6, the recursive neural network-deep collaborative filtering graph comprises an input layer X-253, a hidden layer S-252 and an output layer O-251, and an expanded form S of the hidden layer S-252t-1-255、St-256、St+1-257. The input set of input layer X-253 is labeled { X0,X1,...,Xt,Xt+1,., the output set of hidden layers S-252 is labeled S0,S1,...,St,St+1,., the output set of output layer O-251 is labeled as { O }0,O1,...,Ot,Ot+1,...}，St-1-255、St-256、St+1-257 represents the output state of the hidden layer at time t-1, time t, and time t +1, respectively, where the hidden layer is responsible for performing the most important tasks. From FIG. 6, a single direction can be seenThe streaming information flows from the input layer X-253 to the hidden layer S-252, while a unidirectional streaming information flow flows from the hidden layer S-252 to the output layer O-251, where W-254 is a circular layer. In some cases, however, the recurrent neural network breaks the latter constraint, directing information from the output layer back to the hidden layer, and the input to the hidden layer also contains the state of the previous hidden layer, i.e. the nodes within the hidden layer may be self-connected or interconnected. Taking the time t as an example, the test case set which is possibly selected by the tester at the time t can be predicted according to the main content of the project at the time and the test case selection history of the tester at the previous time under the background of the similar project.
In the intelligent test case extraction system, the problem to be solved is to predict the test cases to be selected by the tester in the future according to the test cases selected by the tester historically. To solve the above problem, the training data of the model may be structured as his (X) ═ X according to the selection history of the test case by the tester1,X2,...,Xn}. Wherein X represents a test case selected by a tester, X1,X2A time series of test cases is selected for the tester. Meanwhile, for the accuracy of data, the historical data of the test case selection of the tester needs to be cleaned first. With the above input data, training is required to be performed according to the model of fig. 6, so that the test case selection prediction problem is converted into a sequence prediction problem, and finally, sorting is performed according to the prediction probability to obtain a final recommendation result.
Meanwhile, the system also introduces a cosine similarity calculation method of the strategy calculation unit 25, and according to the test case-program model, the test case-problem model, the test case-demand model and the test case-user feedback model, cosine similarity matching item codes, test problems, item demands and user feedback feature matrices are calculated, and test cases with high correlation are recommended. Therefore, the system considers the influence of the content of the test case on the recommendation and the influence of the user characteristics on the recommendation, thereby better improving the relevance of the recommendation of the test case.
As shown in formula (1), the test case similarity formula proposed by the system is:
wherein,is a time decay function, alpha is a regulation factor, ti、tjRespectively represents the time when the user selects the case i and the case j, and the time attenuation function represents that the closer the time when the user selects the case i and the case j, the higher the correlation between the cases. N is a radical ofiRepresenting users who select test case i, NjRepresents the number of users who selected test case j, so the meaning of the numerator is the number of users who selected test case i and test case j simultaneously. The above formula can be understood as how many proportion of the users who select test case i also select test case j.
The data module 3 mainly stores an offline recommended test case result set and an online calculation recommended test case result set. The offline recommended test case result set mainly refers to a recommended result obtained through calculation by the policy calculation unit 25. The off-line recommendation mainly comprises an algorithm set and an algorithm engine, and is responsible for integration of test case data, extraction of features, training of a model and off-line evaluation. And calculating a result set of the recommended test case on line, extracting corresponding features mainly according to the project change program set, fusing and grading and rearranging the result set by using a recurrent neural network algorithm, and finally generating the recommended test case for on-line calculation. In the case where no program change list is provided, the set of recommended test cases calculated online is empty.
FIG. 7 is a flowchart of a recommended test case set generation for an intelligent test case design extraction system. As shown in FIG. 7, the online recommended test case set 307 is calculated by a recurrent neural network algorithm according to a change program set such as a change program set 1-301, a change program set 2-302, and a change program set N-303, and the offline recommended test case set 308 is calculated by a recommendation engine such as a recommendation engine 1-304, a recommendation engine 2-305, and a recommendation engine N-306 through a policy. After the offline recommended test case result set and the online recommended test case result set obtained through the strategy calculation are screened 309 according to indexes such as the similarity of the test cases, the ages of the test cases, the correlation between the test cases and the items and the like, the system provides the following formulas to perform TopK sorting recommendation 310 and 311 on the test case result set, and obtain a final recommended test case 312.
Wherein N isuIs a set of user-selected test cases, S (j, K) is a set of K test cases most similar to test case j, WijIs the similarity of the test cases j and i,is the probability that the user u selects the test case i, beta is a weight factor, and beta is more than or equal to 0 and less than or equal to 1. The meaning of this formula is that the more similar the test cases have historically been selected by the user, the more likely it is to get a higher ranking in the user's recommendation list.
The display module 4 mainly provides an interface for a user to view and recommend test case information, test case management problem information, select a test case by operating a button, import and display a change program and the like. The module provides an entrance for information display for a user and provides a test case selection and feedback channel for the user.
The system establishes the correlation model of the test cases and the test problems and the test cases and the requirement items by using a natural language processing method, provides a powerful decision basis for the subsequent test case recommendation, and fully plays the test asset role of the test cases; the system provides a correlation method for constructing the test cases and the test programs, assists testers in designing the test case combination of the change programs, improves the coverage rate and accuracy rate of the test cases, and meets the requirements of rapid iteration and rapid delivery of projects; the system provides a deep learning-based recommendation system, hidden features of test cases and hidden features of users are mined by a recurrent neural network method, respective advantages of a traditional collaborative filtering recommendation technology and a deep learning method are effectively utilized, long-term (global) and short-term (local) associations between test personnel and the test cases are obtained, and complementarity of the test personnel and the test cases on the recommendation system is fully exerted.
The invention can fully promote the function of the test case library, promote the accumulation and reuse of test assets, help the testers to design the test cases, reduce the omission of the design of the test cases and reduce the time for the testers to screen the test cases. Has the following application prospect and advantages: analyzing the text semantics of the function modification description by a natural language processing method of text similarity calculation, establishing semantic association between the function description and the stock test case library, and associating test cases with the same or similar functions; the test cases can be associated comprehensively and reasonably according to the existing program, so that the test cases are prevented from being omitted in design; when the recommendation calculation is carried out on the test case set, a recursive neural network-deep collaborative filtering method is mainly used, and the test case set for recommendation reference is given by analyzing various attribute characteristics of the test cases influencing the project, establishing attribute combination and screening. When the project is delivered and the patch needs to be sent, a proper test case can be provided in the shortest time, and the test efficiency is improved.
The embodiment of the invention also provides an intelligent test case extraction method, which utilizes the intelligent test case extraction system to carry out intelligent test case extraction.
Based on the same application concept as the test case intelligent extraction system, the invention also provides the test case intelligent extraction method. The principle of solving the problems of the intelligent test case extraction method is similar to that of the intelligent test case extraction system, so the implementation of the intelligent test case extraction method can be referred to the implementation of the intelligent test case extraction system, and repeated parts are not described again.
The invention can fully promote the function of the test case library, promote the accumulation and reuse of test assets, help the testers to design the test cases, reduce the omission of the design of the test cases and reduce the time for the testers to screen the test cases. Has the following application prospect and advantages: analyzing the text semantics of the function modification description by a natural language processing method of text similarity calculation, establishing semantic association between the function description and the stock test case library, and associating test cases with the same or similar functions; the test cases can be associated comprehensively and reasonably according to the existing program, so that the test cases are prevented from being omitted in design; when the recommendation calculation is carried out on the test case set, a recursive neural network-deep collaborative filtering method is mainly used, and the test case set for recommendation reference is given by analyzing various attribute characteristics of the test cases influencing the project, establishing attribute combination and screening. When the project is delivered and the patch needs to be sent, a proper test case can be provided in the shortest time, and the test efficiency is improved.
It will be understood by those skilled in the art that all or part of the steps in the method for implementing the above embodiments may be implemented by relevant hardware instructed by a program, and the program may be stored in a computer readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. An intelligent test case extraction system, comprising: the system comprises an information module, a strategy module, a data module and a display module;
the information module is used for storing a test case library, a system program node flow chart, a system test question library, project requirement function description information and user test case selection behavior information;
the strategy module respectively establishes a test case and a model of a program, a problem, a demand and user feedback according to the information stored in the information module, generates a test case recommendation sequence by using a depth collaborative filtering algorithm and a recurrent neural network, determines the similarity of each test case in the test case recommendation sequence, and generates a test case recommendation result;
the data module screens and sorts the test cases in the test case recommendation result according to the basic information and the similarity of the test cases in the test case recommendation result to determine the final recommended test case;
the display module is used for displaying the final recommended test case and receiving user feedback information.
2. The system of claim 1, wherein the policy module comprises a test case and program model unit, and is configured to perform text classification on text descriptions of test cases in the test case library, match the text classification of the test cases with text descriptions of node flow charts of the system program, and determine a matching relationship between the test cases and the program to obtain the test cases and the program model.
3. The system of claim 1, wherein the policy module comprises a test case and problem model unit for performing text classification on the text descriptions of the test cases in the test case library and the text descriptions of the test problems in the system test problem library, matching the text classifications of the test cases with the text classifications of the test problems, and determining a matching relationship between the test cases and the problems to obtain the test cases and the problem models.
4. The system of claim 1, wherein the policy module comprises a test case and requirement model unit, and is configured to perform text classification on text descriptions of test cases in a test case library, perform text classification on text descriptions of software requirement specifications in the project requirement function description information, match the text classifications of the test cases with the text classifications of the software requirement specifications, and determine a test case and requirement matching relationship to obtain the test cases and the requirement model.
5. The system of claim 1, wherein the policy module comprises a test case and user feedback model unit, configured to perform text classification on text descriptions of test cases in the test case library, analyze user feedback behavior in user test case selection behavior information, obtain user feedback characteristics, match the text classification of the test cases with the user feedback characteristics, determine a matching relationship between the test cases and the user feedback characteristics, and obtain the test cases and the user feedback model.
6. The system of claim 1, wherein the policy module comprises a policy calculation unit configured to generate a recommended sequence of test cases by using a deep collaborative filtering algorithm and a recurrent neural network according to the test cases and models of programs, problems, requirements, and user feedback, determine similarity of each test case in the recommended sequence of test cases, and generate a recommended result of test cases.
7. The system of claim 6, wherein the policy computation unit determines the similarity of each test case in the test case recommendation sequence according to a test case similarity formula, the test case similarity formula being:
wherein,is a time decay function, alpha is a regulation factor, ti、tjRespectively representing the time of selecting the test case i and the test case j by the user, NiRepresenting the number of users, N, selecting test case ijRepresenting the number of users who selected test case j.
8. The system of claim 1, wherein the data module is further configured to fuse and rearrange the test case recommendation results by using a recurrent neural network according to program change information to generate online recommendation results, and screen and sort the test case recommendation results and the test cases in the online recommendation results to determine a final recommended test case.
9. The system of claim 1, wherein the data module ranks the test cases in the test case recommendation according to a ranking formula, wherein the ranking formula is:
wherein N isuIs a set of user-selected test cases, S (j, K) is a set of K test cases most similar to test case j, WijIs the similarity of the test cases j and i,is the probability that the user u selects the test case i, beta is a weight factor, and beta is more than or equal to 0 and less than or equal to 1.
10. An intelligent test case extraction method, which is characterized in that the intelligent test case extraction system of any one of claims 1-9 is used for intelligent test case extraction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910627829.7A CN110321291A (en) | 2019-07-12 | 2019-07-12 | Test cases intelligent extraction system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910627829.7A CN110321291A (en) | 2019-07-12 | 2019-07-12 | Test cases intelligent extraction system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110321291A true CN110321291A (en) | 2019-10-11 |
Family
ID=68123280
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910627829.7A Pending CN110321291A (en) | 2019-07-12 | 2019-07-12 | Test cases intelligent extraction system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110321291A (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111104329A (en) * | 2019-12-20 | 2020-05-05 | 光大兴陇信托有限责任公司 | Self-service system docking case guide test method |
| CN111488278A (en) * | 2020-04-08 | 2020-08-04 | 中国银行股份有限公司 | Test method and device |
| CN111506724A (en) * | 2020-07-02 | 2020-08-07 | 北京梦天门科技股份有限公司 | Standard phrase recommendation method and device |
| CN111639024A (en) * | 2020-05-18 | 2020-09-08 | 四川新网银行股份有限公司 | Software quality prediction method based on characteristic frequency data mining |
| CN111813690A (en) * | 2020-07-23 | 2020-10-23 | 深圳壹账通智能科技有限公司 | Test strategy setting method and device based on machine learning |
| CN112181825A (en) * | 2020-09-26 | 2021-01-05 | 建信金融科技有限责任公司 | Test case library construction method and device, electronic equipment and medium |
| CN112256566A (en) * | 2020-09-28 | 2021-01-22 | 建信金融科技有限责任公司 | Test case preservation method and device |
| CN112487293A (en) * | 2020-11-30 | 2021-03-12 | 增长引擎（北京）信息技术有限公司 | Method, device and medium for extracting safety accident case structured information |
| CN112783758A (en) * | 2019-11-11 | 2021-05-11 | 阿里巴巴集团控股有限公司 | Test case library and feature library generation method, device and storage medium |
| CN112882934A (en) * | 2021-02-24 | 2021-06-01 | 中国工商银行股份有限公司 | Test analysis method and system based on defect growth |
| CN112925723A (en) * | 2021-04-02 | 2021-06-08 | 上海复深蓝软件股份有限公司 | Test service recommendation method and device, computer equipment and storage medium |
| CN113010430A (en) * | 2021-03-30 | 2021-06-22 | 中信银行股份有限公司 | Test mode recommendation method and device and electronic equipment |
| CN113886235A (en) * | 2021-09-29 | 2022-01-04 | 上海浦东发展银行股份有限公司 | Test plan determination method, test plan determination device, test plan determination medium, and electronic device |
| CN114461530A (en) * | 2022-02-11 | 2022-05-10 | 国泰君安证券股份有限公司 | Intelligent test case recommendation processing system, method, device, processor and computer-readable storage medium for APP |
| US11327873B2 (en) | 2020-03-27 | 2022-05-10 | Wipro Limited | System and method for identification of appropriate test cases using artificial intelligence for software testing |
| WO2022127393A1 (en) * | 2020-12-15 | 2022-06-23 | International Business Machines Corporation | Reinforcement learning for testing suite generation |
| CN115730914A (en) * | 2022-12-06 | 2023-03-03 | 广州天维信息技术股份有限公司 | Performance assessment scheme recommendation method and device, storage medium and computer equipment |
| CN115906539A (en) * | 2023-01-10 | 2023-04-04 | 联友智连科技有限公司 | System for realizing automatic driving simulation based on Django |
| CN116719996A (en) * | 2023-06-13 | 2023-09-08 | 深圳上翼技术有限公司 | Mobile phone application adding, screening and managing system suitable for APP interaction platform |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102063376A (en) * | 2011-02-16 | 2011-05-18 | 哈尔滨工程大学 | Test case selection method |
| CN102364449A (en) * | 2011-10-24 | 2012-02-29 | 中兴通讯股份有限公司 | Generation method and system for minimum test case set |
| CN103473175A (en) * | 2013-09-11 | 2013-12-25 | 江苏中科梦兰电子科技有限公司 | Extraction method for software testing case set |
| CN106326122A (en) * | 2016-08-23 | 2017-01-11 | 北京精密机电控制设备研究所 | A Software Unit Test Case Management System |
| CN107678951A (en) * | 2017-09-21 | 2018-02-09 | 平安科技（深圳）有限公司 | Test exemple automation management method, device, equipment and storage medium |
| CN108959184A (en) * | 2018-06-26 | 2018-12-07 | 武汉理工大学 | A kind of Collaborative Filtering Recommendation Algorithm and device based on similitude and similitude confidence level |
-
2019
- 2019-07-12 CN CN201910627829.7A patent/CN110321291A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102063376A (en) * | 2011-02-16 | 2011-05-18 | 哈尔滨工程大学 | Test case selection method |
| CN102364449A (en) * | 2011-10-24 | 2012-02-29 | 中兴通讯股份有限公司 | Generation method and system for minimum test case set |
| CN103473175A (en) * | 2013-09-11 | 2013-12-25 | 江苏中科梦兰电子科技有限公司 | Extraction method for software testing case set |
| CN106326122A (en) * | 2016-08-23 | 2017-01-11 | 北京精密机电控制设备研究所 | A Software Unit Test Case Management System |
| CN107678951A (en) * | 2017-09-21 | 2018-02-09 | 平安科技（深圳）有限公司 | Test exemple automation management method, device, equipment and storage medium |
| CN108959184A (en) * | 2018-06-26 | 2018-12-07 | 武汉理工大学 | A kind of Collaborative Filtering Recommendation Algorithm and device based on similitude and similitude confidence level |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112783758A (en) * | 2019-11-11 | 2021-05-11 | 阿里巴巴集团控股有限公司 | Test case library and feature library generation method, device and storage medium |
| CN112783758B (en) * | 2019-11-11 | 2024-02-27 | 阿里云计算有限公司 | Test case library and feature library generation method, device and storage medium |
| CN111104329A (en) * | 2019-12-20 | 2020-05-05 | 光大兴陇信托有限责任公司 | Self-service system docking case guide test method |
| US11327873B2 (en) | 2020-03-27 | 2022-05-10 | Wipro Limited | System and method for identification of appropriate test cases using artificial intelligence for software testing |
| CN111488278A (en) * | 2020-04-08 | 2020-08-04 | 中国银行股份有限公司 | Test method and device |
| CN111488278B (en) * | 2020-04-08 | 2023-09-22 | 中国银行股份有限公司 | Test method and device |
| CN111639024A (en) * | 2020-05-18 | 2020-09-08 | 四川新网银行股份有限公司 | Software quality prediction method based on characteristic frequency data mining |
| CN111506724A (en) * | 2020-07-02 | 2020-08-07 | 北京梦天门科技股份有限公司 | Standard phrase recommendation method and device |
| CN111813690A (en) * | 2020-07-23 | 2020-10-23 | 深圳壹账通智能科技有限公司 | Test strategy setting method and device based on machine learning |
| CN112181825A (en) * | 2020-09-26 | 2021-01-05 | 建信金融科技有限责任公司 | Test case library construction method and device, electronic equipment and medium |
| CN112256566B (en) * | 2020-09-28 | 2024-03-05 | 中国建设银行股份有限公司 | Fresh-keeping method and device for test cases |
| CN112256566A (en) * | 2020-09-28 | 2021-01-22 | 建信金融科技有限责任公司 | Test case preservation method and device |
| CN112487293A (en) * | 2020-11-30 | 2021-03-12 | 增长引擎（北京）信息技术有限公司 | Method, device and medium for extracting safety accident case structured information |
| WO2022127393A1 (en) * | 2020-12-15 | 2022-06-23 | International Business Machines Corporation | Reinforcement learning for testing suite generation |
| GB2617737A (en) * | 2020-12-15 | 2023-10-18 | Ibm | Reinforcement learning for testing suite generation |
| CN112882934A (en) * | 2021-02-24 | 2021-06-01 | 中国工商银行股份有限公司 | Test analysis method and system based on defect growth |
| CN112882934B (en) * | 2021-02-24 | 2024-02-13 | 中国工商银行股份有限公司 | Test analysis method and system based on defect growth |
| CN113010430A (en) * | 2021-03-30 | 2021-06-22 | 中信银行股份有限公司 | Test mode recommendation method and device and electronic equipment |
| CN112925723A (en) * | 2021-04-02 | 2021-06-08 | 上海复深蓝软件股份有限公司 | Test service recommendation method and device, computer equipment and storage medium |
| CN113886235A (en) * | 2021-09-29 | 2022-01-04 | 上海浦东发展银行股份有限公司 | Test plan determination method, test plan determination device, test plan determination medium, and electronic device |
| CN113886235B (en) * | 2021-09-29 | 2025-06-24 | 上海浦东发展银行股份有限公司 | Test plan determination method, device, medium and electronic equipment |
| CN114461530A (en) * | 2022-02-11 | 2022-05-10 | 国泰君安证券股份有限公司 | Intelligent test case recommendation processing system, method, device, processor and computer-readable storage medium for APP |
| CN115730914A (en) * | 2022-12-06 | 2023-03-03 | 广州天维信息技术股份有限公司 | Performance assessment scheme recommendation method and device, storage medium and computer equipment |
| CN115906539A (en) * | 2023-01-10 | 2023-04-04 | 联友智连科技有限公司 | System for realizing automatic driving simulation based on Django |
| CN116719996A (en) * | 2023-06-13 | 2023-09-08 | 深圳上翼技术有限公司 | Mobile phone application adding, screening and managing system suitable for APP interaction platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110321291A (en) | Test cases intelligent extraction system and method | |
| CN111291266B (en) | Artificial intelligence based recommendation method and device, electronic equipment and storage medium | |
| US8498950B2 (en) | System for training classifiers in multiple categories through active learning | |
| US20200401948A1 (en) | Data sampling for model exploration | |
| US20230214679A1 (en) | Extracting and classifying entities from digital content items | |
| CN107844533A (en) | A kind of intelligent Answer System and analysis method | |
| Sharaf et al. | Promoting fairness in learned models by learning to active learn under parity constraints | |
| CN111177538A (en) | Unsupervised weight calculation-based user interest tag construction method | |
| US20230342555A1 (en) | Systems and methods for generating codes and code books using cosine proximity | |
| US20210173857A1 (en) | Data generation device and data generation method | |
| CN113032676A (en) | Recommendation method and system based on micro-feedback | |
| CN111930944B (en) | File label classification method and device | |
| Kuehl | Needmining: Towards analytical support for service design | |
| CN114722198B (en) | Product classification code determination method, system and related device | |
| CN114519406A (en) | Industrial data classification method and model training method and device thereof | |
| Allweyer et al. | Classification of Products in Retail using Partially Abbreviated Product Names Only. | |
| Roelands et al. | Classifying businesses by economic activity using web-based text mining | |
| CN113761193A (en) | Log classification method, apparatus, computer equipment and storage medium | |
| US20240345897A1 (en) | System and method for cloud architecture design and diagram generation using artificial intelligence | |
| Weingart et al. | A taxonomy for deriving business insights from user-generated content | |
| Shidaganti et al. | A comprehensive framework for big data analytics in education | |
| CN109299381B (en) | Software defect retrieval and analysis system and method based on semantic concept | |
| VK et al. | An Effective Book Recommendation System using Weighted Alternating Least Square (WALS) Approach. | |
| Witschel et al. | Case Model for the RoboInnoCase Recommender System for Cases of Digital Business Transformation: Structuring Information for a Case of Digital Change. | |
| Karamanolakis et al. | Interactive machine teaching by labeling rules and instances |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191011 |
|
| RJ01 | Rejection of invention patent application after publication |