CN104156308B - Mobile application testing method on basis of playback of execution trace of users - Google Patents

Abstract

The invention discloses a mobile application testing method on the basis of playback of execution trace of users. The mobile application testing method includes steps of recording an execution trace of a certain mobile application of a user; analyzing the execution trace and determining a plurality of stop points; playing back the execution trace of the application to a certain stop point; implanting an automatic testing method from the stop points. An automatic test process is guided by a handful of user execution trace information, the defect that automation method fails to process complicated interaction is overcome by a simple and feasible method, coverage rate of codes is greatly increased, and testing efficiency and quality are improved.

Description

A mobile application testing method based on user execution trace replay

technical field

The invention relates to the field of mobile application testing, in particular to a testing method based on user execution trace playback.

Background technique

Currently, testing methods for mobile applications mainly include two categories: manual testing and automatic testing. Manual testing relies entirely on manpower to generate various types of test cases. Although it can use human power to obtain valuable test cases and achieve better test results. But long-term testing will make testers feel boring and error-prone. Automated testing can be roughly divided into Monkey tools and model-based testing methods. Monkey is a tool that comes with the Android system to randomly generate input events. This tool cannot meet the need to generate a large number of valid test cases. The model-based method constructs a GUI-related model and sequentially traverses each interface that can be reached by simple interaction, which improves the test effect and efficiency. However, due to the large amount of complex interactions mixed with GUI events in mobile applications, the model-based automation approach encounters its own insurmountable obstacles.

Contents of the invention

Aiming at the problems existing in the prior art, the present invention provides a testing method based on user execution trace replay, which can make up for the deficiency of the fully automatic method, utilize a small amount of user information, handle complex interaction problems that cannot be solved by the automatic method, and enhance the code coverage, thereby improving test efficiency and quality.

The technical solution of the present invention is: a kind of testing method based on user execution trace playback, its steps are:

Step 1: User execution trace recording;

Step 2: stop point recognition;

Step 3: replay engine;

Step 4: Automated test engine.

In step 1, the recording of user execution trace refers to recording one execution trace of the mobile application by the user. The main function of the event recording part is to record various sensor input event sequences of the mobile device;

The user refers to a group of people who have certain experience in using mobile devices. Such users may not have experience in developing mobile platforms or even common programs; during the process of using mobile devices, all interactions are between the user and the sensor. The interaction events are finally collected into the input processing unit of the mobile device system;

The recording refers to going deep into the input processing unit of the mobile device, and saving all event sequences therein as user execution traces;

The mobile application refers to a program that runs on a mobile device and can complete various tasks, and generally utilizes various sensors of the mobile device to complete complex interactions;

The mobile device refers to a pocket-sized portable computing device that is generally equipped with an intelligent operating system and can help users complete various tasks. Typical examples of the mobile device include smart phones and tablet computers;

The execution trace refers to the operation input event sequence generated by the user's daily use of the mobile application without any prompt information, and is arranged in the order of event occurrence time.

In step 2, the stop point identification is to analyze the execution trace and determine several stop points. The main function of the stop point identification part is to identify the points at which several applications should stop in the input event sequence according to a given principle.

The stop point refers to the stop position when the user executes the trace playback; the stop position is a certain event in the user execution trace; on the basis of the user execution traces arranged in the order of event time, the determination principle of the stop point is : After executing an event in the user execution trace, a new GUI interface is generated, and this event is the stopping point; after executing an event in the user execution trace, an interface containing a large number of GUI elements is dynamically generated, and this event is is the stop point; compare the time interval between two adjacent events in the user execution trace, if the interval exceeds a certain threshold, the previous event of this interval is the stop point; the stop point can be identified and generated by some or all of the above principles. The threshold refers to the minimum value of several significantly larger time intervals.

In step 3, the replay engine refers to replaying the execution trace of the application until a certain stop point; specifically, it refers to the part of the sequence of events that the mobile device writes into the collected user execution trace, which includes: starting the application, sending the user The events contained in the execution trace are written to the input processing unit of the mobile device system to achieve the effect of the user re-operating the application, and stop writing at a certain stop point determined in the previous step; the input event refers to all sensor inputs received by the mobile device Information, including event time, event source, and event parameter information; the event parameter information includes event type, code, and value.

In step 4, the automated testing method refers to several methods that can be used for fully automated testing of mobile applications; in the present invention, it specifically refers to random methods and depth-first search methods, which can be existing methods or methods proposed by testers themselves , they generally include the following three steps: first, obtain the GUI dynamic layout information of the application at runtime, including various GUI elements; then, based on the obtained GUI element information, determine the next input action according to the automation strategy; here The automation strategy described above can be random selection or depth-first search; finally, the input action is sent to the mobile device and executed.

Compared with the prior art, the present invention has the advantages that: the present invention combines the manual testing method with the automatic testing method, and takes advantage of the manual method being good at exploring the program framework through complex interactions and the automatic method being better at traversing all states at a specific position Features, greatly enhance the coverage of the code, thereby improving the test efficiency and quality; From the perspective of the cost of manual participation, the present invention only needs to collect a small amount of user execution traces, generally requiring no more than 10 minutes of user participation.

Description of drawings

Fig. 1 is working principle diagram one of the present invention;

Fig. 2 is working principle diagram two of the present invention;

Fig. 3 is the working principle diagram three of the present invention;

Fig. 4 is a work flowchart of the present invention;

FIG. 5 is a schematic diagram of user execution traces in the present invention.

detailed description

The technical solutions of the present invention will be further described through specific embodiments below in conjunction with the accompanying drawings.

As shown in Fig. 1, Fig. 2 and Fig. 3, Fig. 1 is a working principle diagram of the present invention; Fig. 2 is a working principle diagram of the present invention; The state space of ; the circle represents the program state; the edge represents various interactions, including clicking buttons, drawing unlock patterns, etc. Figure 1 shows a collected user execution, and 101-106 in the figure represent a user execution trace; the shaded part 107 indicates that the user execution covers five points; Figure 2 is the case of a purely automated method, which is limited by the complex interaction represented by the dotted line 110 , the automated method can only traverse half of the points, as shown in the shaded part 111; the present invention collects user execution traces 101-106, and then recognizes two events in the user execution trace as stop points, as shown in the concentric circles in Figure 3 As shown, the automated testing process starting from these two points covers all the points in the figure, that is, the shaded parts 121 and 122, so the present invention can greatly improve the test code coverage.

As shown in Fig. 4 and Fig. 5, Fig. 4 is a working flowchart of the present invention; Fig. 5 is a schematic diagram of user execution traces of the present invention; this embodiment uses the popular Android system as an example for introduction. The mobile application testing method based on user execution trace replay includes five parts: event recording, stop point recognition, replay engine, test engine and mobile test framework. Each section completes the following four steps:

Step 1: Record user execution traces. As shown in the left part of FIG. 4 , during the user's use of the mobile device, all sensor operation events are written into /dev/input/event*(210). This step reads all events written into /dev/input/event* to form user execution trace 211 . Figure 5 is the recorded user execution trace of the screen during two "press-release" times. Each line represents an event, and from left to right are the time of the event, event source (device) and event parameters. The event parameters include Type, code and value, all events described in Figure 5 are sorted by timestamp.

The user execution trace refers to the input event sequence of all sensors of the mobile device during the process of the user operating the mobile device. When a user uses a mobile device, all interactions are between the user and the sensor. Such interaction events are ultimately funneled to the mobile device system input processing unit. The recording refers to going deep into the input processing unit of the mobile device, and saving all event sequences therein as user execution traces.

Step 2: Identify stopping points. The stop point refers to the stop position of the next playback, and it specifically refers to an event in the user execution trace obtained above. On the basis of user execution traces arranged in chronological order of events, the stopping point is determined according to the following three principles:

After executing an event in the user execution trace, a new GUI interface is generated, and this event may become a stop point. For example, after the second "press-release" action in Figure 5 is completed, a new screen appears on the mobile device. The 14th event is the stopping point.

After executing an event in the user execution trace, an interface with a large number of GUI elements is dynamically generated. This event may become a stopping point. For example, after the first "press-release" action is completed, a large number of dynamic GUI elements appear in the screen, the seventh event is the stop point.

Compare the time interval between two adjacent events in the user's execution trace. The larger the interval, the more likely the previous event of this interval will become the stop point. For example, calculate the event interval between any two consecutive events in Figure 5, the 7th to the 8th and the 14th to 15th events are relatively large, so both the 7th event and the 14th event may be stopping points.

The stopping point identification (212) may be generated by some or all of the principles described above.

Step 3: replay the application execution trace until a certain stop point (213). Specifically, it includes the following steps:

Send user execution trace and all stopping points to mobile device.

Start the mobile device, and then write the received events in the user execution trace into /dev/input/event* (the mobile device system input processing unit 210) in chronological order, so as to achieve the effect that the user re-operates the mobile device. The steps of this step The mobile device needs to obtain ROOT permission. In addition, the interval of writing each event should be consistent with the interval of recording.

When a stop point is reached, the replay of the event is stopped and the stop point is removed from the list of stop points.

After step 4 is completed, the playback process described above continues to be repeatedly executed until the stop point list is empty.

Step 4, starting from the stop point, execute the automated testing method (214). Described automated test method can be existing or the method that tester proposes oneself, and they generally all comprise following three steps: first obtain the GUI dynamic layout information of application at running time, comprise various GUI elements; Then obtain GUI element Based on the information, the next input action is determined according to the automation strategy, which may be random selection or depth-first search; finally, the input action is sent to the mobile device and executed.

A typical automated testing process includes the random algorithm RND and the depth-first search algorithm DFS; the random algorithm randomly selects the next action from all optional actions when making automated decisions, and the depth-first algorithm accesses the model in a depth-first manner in the graph traversal algorithm All vertices and edges in .

Although the present invention has been disclosed above with preferred embodiments, they are not intended to limit the present invention. Any skilled person can make various changes or modifications without departing from the spirit and scope of the present invention. Also within the protection scope of the present invention.

Claims (5)

1. A mobile application testing method based on user execution trace playback, characterized in that the steps are: Step 1. User execution trace recording refers to recording the user's execution trace of the mobile application once, and refers to recording various sensor input event sequences of the mobile device; In step 1, when the user is using the mobile device, all operations on the sensor are events, and all input events form the user execution trace; Step 2. Stop point identification refers to analyzing the execution trace and determining several stop points, which refers to identifying several stop points in the input event sequence according to a given principle; In step 2, the stop point refers to the stop position when replaying in the next step. Specifically, it refers to an event in the user execution trace obtained in step 1. Based on the user execution traces arranged in the order of event occurrence time, stop The principle of point determination: after executing an event in the user execution trace, a new GUI interface is generated, and this event becomes the stop point; after executing an event in the user execution trace, an interface containing a large number of GUI elements is dynamically generated, This event becomes the stop point; compare the time interval between two adjacent events in the user's execution trace, if the interval exceeds a certain threshold, the previous event of this interval is the stop point; the stop point identification can be generated by some or all of the above principles; Step 3. The replay engine refers to replaying the execution trace of the application until a certain stop point, and refers to rewriting the recorded part of the input event sequence to the mobile device; In step three, specifically: send the user's execution trace and all stop points to the mobile device; start the mobile device, and then write the events in the user's execution trace into the mobile device system input processing unit in chronological order, so as to reach the user's The effect of re-operating the mobile device. At this time, the mobile device needs to obtain ROOT permission. The interval of each event writing should be consistent with the recording interval; stop the replay of the event when a certain stop point is reached, and change the stop point from the stop point Delete in the list, enter step 4; After step 4 is completed, the replay process continues to be repeatedly executed until the stop point list is empty; Step 4, the automated testing engine refers to starting from the stop point, executing the automated testing method, and completing the automated testing process of the mobile device; In step 4, the GUI dynamic layout information of the application at runtime is obtained, including various GUI elements; on the basis of the obtained GUI element information, the next input action is determined according to the automation strategy, and the automation strategy can be randomly selected or Depth-first search; the input action is sent to the mobile device and executed, and so on until the automation test task is completed. 2. The mobile application testing method based on user execution trace replay according to claim 1, characterized in that: The user refers to a group of people who have certain experience in using mobile devices; The mobile application refers to a program that runs on a mobile device and can complete various tasks, and uses various sensors of the mobile device to complete complex interactions; The recording refers to going deep into the input processing unit of the mobile device, and saving all event sequences therein as user execution traces; The execution trace refers to the operation input event sequence generated by the user's daily use of the mobile application without any prompt information; The recording execution trace refers to collecting input event sequences generated when the user operates the mobile device; The stop point refers to the stop position when the user executes the track playback; The stop position is an event in the user execution trace; The replay is a partial sequence of events pointing to the mobile device writing the collected user execution trace; The automated testing methods refer to several methods that can be used for fully automated testing of mobile applications. 3. The mobile application testing method based on user execution trace replay according to claim 2, characterized in that: The mobile device refers to a pocket-sized portable computing device equipped with an intelligent operating system and capable of helping users complete various tasks, including smart phones and tablet computers; The input event refers to all sensor input information received by the mobile device; The event includes event time, event source and event parameter information; The event parameters include event type, code and value. 4. The mobile application testing method based on user execution trace replay according to claim 1, characterized in that: the threshold refers to the minimum value of several significantly larger time intervals. 5. The mobile application testing method based on user execution trace replay according to claim 1, characterized in that: according to the event time information, the writing time interval is precisely controlled to complete the accurate replay of the user execution trace.