CN102708052B - Software fault automatic positioning method in a kind of unit testing - Google Patents

Abstract

The present invention provides a method for automatically locating software faults in unit tests, comprising: A. using a group of test cases that meet the coverage requirements, executing the program to be tested, and judging whether the execution results of the test cases are consistent with the expected results; B. according to each The execution path corresponding to the test case, analyze the edge information on the path, and determine the suspicious space composed of the edges on the path; C, count the number of times each edge in the suspicious space is covered by the passed and failed test cases, and calculate the specified execution mode Next, the suspiciousness of each edge; D. Calculate the suspiciousness of its adjacent statement nodes from the suspiciousness of the edge; E. Sort the fault nodes according to the suspiciousness, and map them to the tested program in turn to generate fault location report, assisting users to check faults. By adopting the method, the calculation workload of statement analysis can be reduced, and the efficiency of fault location can be improved by taking into account the logical relationship between statement nodes.

Description

A method for automatically locating software faults in unit testing

technical field

The method relates to a fault location technology in software testing, in particular to a method for automatically locating software faults in unit testing, by executing a test case and according to whether the execution result is consistent with the expected test result, the fault in the software is automatically located.

Background technique

Software failure is a dynamic behavior. If there is no appropriate measure to deal with it in time when a software failure occurs, software failure will occur. Finding and eliminating as many faults as possible in unit testing as early as possible will reduce and avoid wasting too much unworthy time later.

At present, the research on software fault location technology is mostly based on whether the execution result is passed or not, and the program statement is used as the coverage node for statistical analysis. When there are many program codes, the amount of calculation for statement analysis is large, resulting in a lot of time-consuming, and the interference of fault propagation on location is not considered, which affects the accuracy of fault location.

Contents of the invention

In view of this, the main purpose of the present invention is to provide a method for automatically locating software faults in unit tests, to overcome the fact that the existing software fault locating methods take statement nodes as coverage units, have a large amount of calculation and do not take into account the relationship between statement nodes. It reduces the computational workload of statement analysis and improves the efficiency of fault location by taking into account the logical relationship between statement nodes.

In order to achieve the above object, technical solution of the present invention is achieved in that way:

A method for automatically locating software faults in unit testing, the method comprising:

A. Use a group of test cases that meet the coverage requirements, execute the program under test, and judge whether the execution results of the test cases are consistent with the expected results;

B. According to the execution path corresponding to each test case, analyze the edge information on the path, and determine the suspicious space composed of the edges on the path;

C. Count the number of times each edge in the suspicious space is covered by passed and failed test cases, and calculate the suspiciousness of each edge in the specified execution mode;

D, the suspicious degree calculation of its adjacent sentence node by the suspicious degree of the edge that above-mentioned steps obtains;

E. Sort the faulty nodes according to the suspiciousness of adjacent statement nodes, and map them to the tested program in turn to generate a fault location report to assist users in checking faults.

Wherein: described step A comprises:

A1. Fault location based on two execution modes, which are full use case execution mode and partial use case execution mode;

A2. When using the full-use-case execution mode, execute all the test cases in a group, and divide the use cases into two mutually incompatible parts according to whether the execution results are consistent with expectations, that is, the passed test cases and the failed test cases, and Record the side information contained in the corresponding execution path;

A3. When using the partial use case execution mode, randomly select a use case in the test case set to start execution, judge whether the execution result is consistent with the expected result, and stop the test until the execution result of the test case is inconsistent with the expected result. The test case of is divided into two parts, that is, the first m passed test cases and the m+1th failed test case, and the corresponding execution path is obtained;

A4. The execution results include the following content: the function has a return value, record its return result; key intermediate variables and global variables, record its return value; when it contains file operations, record the generated file directory and content information; when it contains database operations, Record the generated database directory and content information.

Described step B comprises:

B1. The suspicious space is composed of the edges included in the execution path corresponding to some test cases in the specified execution mode. It is the maximum range for judging the occurrence of faults. The suspicious space is taken from the edges of the suspicious space to calculate the suspiciousness of the subsequent steps. Different execution modes, suspicious space is different.

Wherein, the step B1 further includes:

B11. In the full use case execution mode, there must be faults in the edge sets contained in the execution paths corresponding to all failed test cases, and this is used as the largest suspicious space to calculate the suspiciousness of the execution mode in the subsequent steps;

B12. In the partial use case execution mode, there must be a fault in the execution path corresponding to the m+1th failed test case, and the set of edges on this path is taken as the suspicious space in this execution mode;

Described step C comprises:

C1. In the full use case execution mode, traverse the suspicious space and count the covered edges The number of test cases, combined with the total number of test cases that pass and fail, uses the Tarantula method to calculate the suspiciousness of the edge, and the calculation is as follows:

in: is the number of all failed test cases;

is the number of all passed test cases;

In the test case that is passed, its corresponding execution path contains the edge the number of use cases;

For the failed test case, its corresponding execution path contains the edge the number of use cases;

C2. In the partial use case execution mode, traverse the suspicious space obtained in step B12, that is, the edge set on the execution path corresponding to the failed test cases, and count the number of use cases that cover the specified edge among the first m passed test cases , using the following formula to calculate its suspiciousness:

;

in, is an edge on the execution path corresponding to the m+1th test case;

is the number of all passed test cases;

In the test case that is passed, its corresponding execution path contains the edge number of use cases.

Said step D comprises:

D1. In the full use case execution mode, analyze the adjacent nodes of the edges included in the execution path corresponding to all failed test cases, and take the maximum value of all incoming edge suspiciousness of the node and the maximum value of all outgoing edge suspiciousness Average value, as the suspicious degree of the node;

D2. In the partial use case execution mode, analyze the adjacent nodes of the edge on the execution path corresponding to the failed test case, and take the average value of the suspicious degree of the incoming edge and outgoing edge of the node as the suspicious degree of the node ;

D3. For a node with only outgoing edges or only incoming edges (such as program entry nodes or program exit nodes), its outgoing edge suspiciousness or incoming edge suspiciousness is taken as the suspicious degree of the node; for nodes that are not counted on the control flow graph, Suspiciousness is recorded as zero.

Described step E comprises:

E1. Fault location report, including the following content: sort the statement nodes according to the node suspicious degree value obtained by the above steps, each item includes the node suspicious degree value, node name and label, and the node corresponds to the statement line in the program under test No;

E2. Take the node with the highest suspicious degree each time, map it to the statement in the program under test, and display the statement in a highlighted color, which is convenient for the user to check until the fault location is found.

The method for automatically locating software faults in unit tests provided by the present invention has the following advantages:

The software fault location method of the present invention adopts the calculation of the suspicious degree with the edge as the coverage unit, and calculates the suspicious degree of the node by calculating the suspicious degree of the edge. The experiment proves that the calculation amount can be greatly reduced by the method of the present invention, and the efficiency of analysis can be improved. At the same time, considering the logical relationship of nodes, that is, the influence of fault propagation on program execution results, the accuracy of positioning is higher.

Description of drawings

Fig. 1 is a schematic diagram of fault location based on two execution modes in the present invention;

FIG. 2 is an overall flowchart of fault location based on two execution modes in the present invention.

detailed description

The method of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments of the present invention.

The method for automatically locating software faults in unit tests of the present invention proposes an automatic fault locating technology in unit tests based on two types of execution modes under multi-type coverage, and the two execution modes are considered from the perspective of users respectively. For the full-automatic mode, a group of test cases can be executed simultaneously, and the more common situation is that once a certain wrong test case is executed, the test is stopped to search for the fault, so the method of the present invention can respectively treat the two kinds of test cases. Execution mode for automatic location of software faults. Taking the edge on the control flow graph corresponding to the program as the coverage unit, if the program fails, the path execution trajectory will inevitably change. Using two test case execution methods, the edge on the path is counted on the execution of the test case, and the edge is calculated. , and then calculate the suspiciousness of adjacent statement nodes based on the suspiciousness of the edge. Represents the logic and dependency relationship between statement nodes, and the method of counting in units of edges considers the connection between statement nodes, thereby improving the positioning efficiency. At the same time, the test cases used in this method can meet various coverage requirements (statement coverage, branch coverage, MC/DC coverage, etc.) to generate, which provides a basis for the location efficiency analysis of various coverage, and has strong scalability.

Fig. 1 is a schematic diagram of fault location based on two execution modes in the present invention. As shown in Fig. 1, the basic idea of the automatic fault location method in the unit test proposed by the present invention is: first, generate a set of specified The test case of coverage criteria and coverage requirements, based on two execution modes, executes the program under test, then judges whether the execution result is consistent with the expected result, and marks the execution result as pass or fail. The suspicious space composed of the edges contained in the path counts the times of edge coverage, and calculates the suspicious degree of each edge in the suspicious space. Then, calculates the suspicious degree of adjacent statement nodes according to the suspicious degree of the edge, and finally, according to the suspicious degree The edge and statement nodes are sorted, and a fault location report is generated to assist users in troubleshooting in sequence.

As shown in Figure 1, the method mainly includes the following processes:

Step 1: Use a set of test cases that meet the coverage requirements (statements, branches, MC/DC, etc.), execute the program under test, and judge whether the execution results of the test cases are consistent with the expected results;

Step 2: According to the execution path corresponding to each test case, analyze the edge information on the path, and determine the suspicious space composed of the edges on the path;

Step 3: Count the number of times each edge in the suspicious space is covered by passed and failed test cases, and calculate the suspiciousness of each edge under the specified execution mode (full use case execution and partial use case execution);

Step 4: Calculate the suspiciousness of its adjacent sentence nodes by the suspiciousness of the edge obtained in the above steps;

Step 5: Sort the fault nodes according to the degree of suspiciousness, and map them to the program under test in turn to generate a fault location report to assist users in checking faults.

In conjunction with the above-mentioned process, the fault automatic location method in the unit test proposed by the present invention is further described in detail below. Fig. 2 is an overall flow chart of the present invention. As shown in Fig. 2, the method includes:

Step 1: According to different coverage criteria (such as statement, branch, MC/DC, etc.), generate a set of test cases that meet the coverage requirements.

Here, it is necessary to statically analyze the program under test and generate a control flow graph, and then instrument the program under test according to the specified coverage criteria, thereby generating a set of test cases that meet the coverage requirements, and the execution paths corresponding to each test case are not repeated , that is, the use case has no redundancy.

For the convenience of the following steps, the generated set of test cases is denoted as , the test case The corresponding execution path is denoted as , taking the edge on the control flow graph as the unit, the path can be expressed as .

Step 2: Use the test case generated in the above step 1 to execute the program under test, judge whether the execution result of the test case is consistent with the expected result, analyze the edge information on the path according to the execution path corresponding to each test case, and determine the edge information on the path The execution result is not only the return value of the program under test. If there are operations such as files, databases, and pointers, the file directory and content information generated by these operations are also part of the execution result.

Assume that the test case generated as described in step 1 is expressed as , this method performs fault location based on two execution modes, which are full use case execution mode and partial use case execution mode.

Step 21: When using the full use case execution mode, execute all the test cases obtained in step 1, and divide the use cases into two mutually incompatible parts according to whether the execution results are consistent with expectations, that is, the passed test cases and the failed test case , and record the side information contained in the corresponding execution path.

Step 22: When using partial use case execution mode, randomly select a use case in the test case set generated in step 1 to start execution, and judge whether the execution result is consistent with the expected result until a certain use case (assumed to be the m+1th use case) ) when the execution result is inconsistent with the expectation, the test is stopped. At this time, the executed test cases are divided into two parts, that is, the first m passed test cases, expressed as , and the m+1th failed test case, denoted as , to get their execution paths.

In the above steps, according to whether the test case is passed or not, we can get with The corresponding path information is as follows:

, indicating the set of paths corresponding to the passed test cases;

, representing the set of paths corresponding to the failed test cases.

The suspicious space is composed of the edges included in the execution paths corresponding to some test cases in the specified execution mode, which is the maximum range for judging the occurrence of faults, and the suspicious space is taken to calculate the suspiciousness in the following steps. Different execution modes have different suspicious spaces. details as follows:

Step 211: In the all-case execution mode, there must be faults in the execution paths corresponding to all failed test cases, and the intersection of edges on the paths constitutes a suspicious space in this execution mode, which is expressed as follows;

.

Step 221: In partial use case execution mode, assume the first m use cases The execution result of is correct, then the m+1th use case There must be a fault in the corresponding execution path, and the edges on the path constitute the suspicious space in the execution mode, which is expressed as follows;

.

Step 3: Count the number of times each edge in the suspicious space is covered by passed and failed test cases, and calculate the suspiciousness of each edge in the specified execution mode. The calculation formula of suspiciousness is different for different execution modes, as follows:

Step 31: In the full use case execution mode, traverse the suspicious space obtained in step 211, and count the covered edges The number of test cases, combined with the total number of test cases that pass and fail, uses the Tarantula method to calculate the suspiciousness of the edge, and the calculation is as follows:

;

in: Represents the set of test cases that failed The total number of use cases included;

Indicates the set of test cases that passed The total number of use cases included;

Indicates that in the passed test case, the corresponding execution path contains the edge the number of use cases;

Indicates that in the failed test case, the corresponding execution path contains the edge number of use cases.

Step 32: In partial use case execution mode, traverse the suspicious space obtained in step 221, that is, the set of edges on the execution path corresponding to the failed test cases, and count the number of use cases that cover the specified edge among the first m passed test cases , using the following formula to calculate its suspiciousness:

in, Indicates an edge on the execution path corresponding to the m+1th test case;

Indicates the number of all passed test cases;

Indicates that in the passed test case, the corresponding execution path contains the edge number of use cases.

Step 4: Calculate the suspiciousness of its adjacent statement nodes from the suspiciousness of the edge obtained in the above steps;

Step 41: In the full use case execution mode, analyze the adjacent nodes of the edges included in the execution path corresponding to all failed test cases, and take the maximum value of all incoming edge suspiciousness and the maximum value of all outgoing edge suspiciousness of the node The average value of , as the suspicious degree of the node.

Assume that the set of nodes is denoted as , for each node Suspiciousness is calculated as follows:

in, Indicates the maximum value of suspiciousness among all incoming edges of a node, ;

Indicates the maximum value of suspiciousness among all incoming edges of a node, .

Step 42: In the partial use case execution mode, analyze the adjacent nodes of the edge on the execution path corresponding to the failed test case, and take the average value of the suspiciousness of the incoming edge and outgoing edge of the node as the suspiciousness of the node. Spend.

Assume that the set of nodes is denoted as ,node entry , denoted as , the suspiciousness is calculated as follows:

.

Step 43: For nodes with only outgoing edges or incoming edges (such as program entry nodes or program exit nodes), the suspicious degree of outgoing edges or incoming edges is taken as the suspicious degree of the node; for the nodes that are not counted on the control flow graph Node, indicating that neither the incoming edge nor the outgoing edge is in the suspicious space, and the suspiciousness of this node is recorded as zero.

Step 5: Sort the fault nodes according to the degree of suspiciousness, and map them to the program under test in turn, display the statement in a highlighted color, and generate a fault location report to assist users in checking faults.

The fault location report can include the following content: sort the faulty nodes obtained in step 4 according to the degree of suspiciousness, and each item includes the value of the suspicious degree of the node, the name and label of the node, the line number of the statement corresponding to the node in the program under test, etc. .

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention.

Claims (6)

1. A method for automatically positioning software faults in unit test is characterized by comprising the following steps:

A. executing the tested program by adopting a group of test cases meeting the coverage requirement, and judging whether the execution result of the test cases is consistent with the expected result;

B. analyzing side information on the path according to the execution path corresponding to each test case, and determining a suspicious space formed by the sides on the path;

C. counting the times of covering each edge in the suspicious space by the passed test case and the failed test case, and calculating the suspicious degree of each edge in the specified execution mode;

D. calculating the suspicious degree of the adjacent statement nodes according to the suspicious degree of the edges obtained in the step;

E. sequencing the fault nodes according to the size of the suspicious degree of the adjacent statement nodes, sequentially mapping the fault nodes to a tested program to generate a fault positioning report, and assisting a user in checking the fault;

the step B comprises the following steps:

b1, the suspicious space is composed of edges contained in execution paths corresponding to some test cases under the appointed execution mode, the maximum range of the fault is judged, the edges of the suspicious space are taken to carry out suspicious degree calculation of the subsequent steps, and the suspicious space is different in different execution modes.

2. The method for automatically locating software faults in a unit test according to claim 1, wherein the step A comprises the following steps:

a1, fault location is carried out based on two execution modes, namely a full use case execution mode and a partial use case execution mode;

a2, when a full-case execution mode is adopted, a group of test cases are completely executed, the cases are divided into two mutually incompatible parts according to whether the execution result is consistent with the expectation, namely, the passed test cases and the failed test cases, and the side information contained in the corresponding execution path is recorded;

a3, when a partial case execution mode is adopted, randomly selecting a certain case in the test case set to start execution, judging whether an execution result is consistent with an expected result or not, stopping testing until the execution result of the test case is inconsistent with the expected result, dividing the test case executed at the moment into two parts, namely the first m passed test cases and the (m + 1) th failed test cases, and acquiring a corresponding execution path;

a4, the execution result comprises the following contents: the function has a return value and records the return result; recording return values of key intermediate variables and global variables; when the file operation is contained, recording the generated file directory and content information; and when the database operation is contained, recording the generated database catalog and content information.

3. The method for automatically locating software failure in unit test according to claim 2, wherein said step B1 further comprises:

b11, under the execution mode of all cases, the edge set contained in the execution path corresponding to all the failed test cases has certain faults, and the fault is used as the maximum suspicious space for carrying out the suspicious degree calculation of the execution mode in the following steps;

b12, under the partial case execution mode, the execution path corresponding to the m +1 th failed test case has a fault, and the edge set on the path is taken as a suspicious space under the execution mode.

4. The method for automatically locating software faults in a unit test according to claim 3, wherein the step C comprises the following steps:

c1, under the all-use-case execution mode, traversing the suspicious space and counting the covered edge e_iThe total number of the passed test cases and the failed test cases are combined, and the tarandula method is adopted to calculate the suspicion degree on the opposite side, and the calculation is as follows:

<mrow> <mi>s</mi> <mi>u</mi> <mi>s</mi> <mi>p</mi> <mi>i</mi> <mi>c</mi> <mi>i</mi> <mi>o</mi> <mi>u</mi> <mi>s</mi> <mi> </mi> <mi>deg</mi> <mi> </mi> <mi>r</mi> <mi>e</mi> <mi>e</mi> <mrow> <mo>(</mo> <msub> <mi>e</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mfrac> <mrow> <msubsup> <mi>N</mi> <mi>f</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>e</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> </mrow> <msub> <mi>N</mi> <mi>f</mi> </msub> </mfrac> <mrow> <mfrac> <mrow> <msubsup> <mi>N</mi> <mi>f</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>e</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> </mrow> <msub> <mi>N</mi> <mi>f</mi> </msub> </mfrac> <mo>+</mo> <mfrac> <mrow> <msubsup> <mi>N</mi> <mi>p</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>e</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> </mrow> <msub> <mi>N</mi> <mi>p</mi> </msub> </mfrac> </mrow> </mfrac> </mrow>

wherein: n is a radical of_fThe number of all failed test cases is counted;

N_pthe number of all passed test cases is counted;

in the passed test case, the corresponding execution path includes edge e_iThe number of use cases;

in the failed test case, the corresponding execution path includes the edge e_iThe number of use cases;

c2, under the execution mode of partial use cases, counting the number of use cases covering the appointed side in the previous m passed test cases by traversing the suspicious space obtained in the step B12, namely the edge set on the execution path corresponding to the failed test case, and calculating the suspicious degree by using the following formula:

<mrow> <mi>s</mi> <mi>u</mi> <mi>s</mi> <mi>p</mi> <mi>i</mi> <mi>c</mi> <mi>i</mi> <mi>o</mi> <mi>u</mi> <mi>s</mi> <mi> </mi> <mi>deg</mi> <mi> </mi> <mi>r</mi> <mi>e</mi> <mi>e</mi> <mrow> <mo>(</mo> <msub> <mi>e</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mn>1</mn> <mo>-</mo> <mfrac> <mrow> <msubsup> <mi>N</mi> <mi>p</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>e</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> </mrow> <msub> <mi>N</mi> <mi>p</mi> </msub> </mfrac> <mo>;</mo> </mrow>

wherein e is_iA certain edge on the execution path corresponding to the (m + 1) th test case;

N_pthe number of all passed test cases is counted;

in the passed test case, the corresponding execution path includes edge e_iThe number of use cases.

5. The method according to claim 4, wherein the step D comprises:

d1, under the execution mode of the all cases, analyzing adjacent nodes of edges contained in the execution paths corresponding to all failed test cases, and taking the average value of the maximum value of all incoming edge suspicions and the maximum value of all outgoing edge suspicions of the nodes as the suspicions of the nodes;

d2, under the execution mode of partial cases, analyzing the adjacent nodes of the edge on the execution path corresponding to the failed test case, and taking the average value of the incoming edge suspicion degree and the outgoing edge suspicion degree of the node as the suspicion degree of the node;

d3, regarding the node only having an outgoing edge or an incoming edge, the outgoing edge suspicion degree or the incoming edge suspicion degree is taken as the node suspicion degree; and for nodes which are not counted on the control flow graph, the suspicious degree is marked as zero.

6. The method for automatically locating software faults in a unit test according to claim 1, wherein the step E comprises the following steps:

e1, fault location report, containing the following: sequencing the sentence nodes according to the node suspicious degree values obtained in the steps, wherein each item comprises the node suspicious degree value, the node name and the label, and the node corresponds to the sentence line number in the tested program;

e2, each time, the node with the highest suspicious degree is taken and mapped to the sentence in the tested program, and the sentence is displayed in highlight color, so that the user can conveniently check the sentence until the fault position is found out.