CN107992405A - application software testing method and device - Google Patents

Abstract

The invention discloses a kind of application software testing method and apparatus, belong to technical field of measurement and test.This method includes：The reasonable interval of the test output valve of tested application software system is obtained, the reasonable interval is analyzed to obtain according to the test output valve for repeatedly carrying out automatic test to being tested application software system；Tested application software system is tested using acquired reasonable interval；Judge whether the tested application software system is abnormal according to test result.Thus, it is possible to quickly judge whether tested application software system is abnormal.

Description

Application software testing method and device

technical field

The invention relates to the technical field of testing, in particular to an application software testing method and device.

Background technique

When one application software is used to track and manage another application software, one of the two application software involved is called the tracking system, and the other is called the tracked system. The tracked system consists of many modules (such as microservices). That is to say, the tracking system is used to track the tracked system to manage the functions of each module of the tracked system.

When testing the application software system composed of the tracking system and the tracked system, since the source code of the internal modules of the tracked system and the mutual calling relationship are unknown, the problem encountered is that one of the tracked systems Module testing, such as testing the number of times a certain module is called within a certain period of time, in addition to the calls initiated by the user itself leading to an increase in the number of calls to the module, if one or several functional modules in the tracked system are also Calling this module will also increase the number of calls. The resulting call count is the sum of the two types of call counts. That is, when a disturbance is input during the test, the result obtained is not caused by this single disturbance, but the superposition of the results of multiple disturbances, and the result caused by a single disturbance cannot be obtained.

For example, assume that the X service of the tracked system in the test application software system, the user inputs a value A ₀ , and expects to get the returned output value R ₀ . However, when the application software is working normally, there are other services, such as Y service and Z service, which are also calling the X service to be tested, wherein the input of the Y service is Y ₀ , and the input of the Z service is Z ₀ , resulting in The obtained test output value R is the result of the joint action of the input values A ₀ , Y ₀ , and Z ₀ , and R ₀ cannot be obtained directly. Since the number and frequency of services simultaneously calling X service in the application software are unknown at different times, the output value R obtained each time is not a fixed value.

Therefore, the existing methods for testing application software systems cannot quickly determine whether the test results are normal because they cannot obtain the test results caused by individual disturbances, let alone quickly find the cause of the abnormalities when the test results are abnormal.

Contents of the invention

In view of this, the object of the present invention is to provide an application software testing method and device to solve at least one of the above technical problems.

In order to achieve the above object, the present invention provides a method for testing application software. The method includes the steps of: obtaining a reasonable interval of the test output value of the application software system under test, and the reasonable interval is based on multiple times of performing automated testing on the application software system under test. The test output value is analyzed and obtained; using the obtained reasonable interval to test the tested application software system; and judging whether the tested application software system is abnormal according to the test result.

Optionally, the process of obtaining the reasonable interval based on the analysis of the test output values of the tested application software system for multiple times of automated testing specifically includes: setting the time interval and times of automated testing; The application software system under test performs automated testing and obtains test output values until the set number of times is reached; the reasonable interval is obtained according to the test output values.

Optionally, the step of obtaining a reasonable interval according to the test output value specifically includes: counting the value range of multiple test output values obtained by the automated test; removing discrete points from the value range; removing The value range after the discrete point is determined as the reasonable interval.

Optionally, the step of testing the tested application software system using the obtained reasonable interval specifically includes: testing the tested application software system to obtain a test output value of the test; judging the test output value Whether the value falls within the scope of the reasonable interval; when the test output value falls within the scope of the reasonable interval, it is determined that the test result of the tested application software system is passed; when the test When the output value is not within the reasonable range, it is determined that the test result of the tested application software system is failed.

Optionally, the method further includes the step of: when it is determined that the tested application software system is abnormal, determining the cause of the abnormality in combination with whether the application software in the tested application software system has a version update.

In order to achieve the above object, the present invention also proposes an application software testing device, which includes: an acquisition unit for obtaining a reasonable range of the test output value of the tested application software system, and the reasonable range is based on multiple tests of the tested application The software system is obtained by analyzing the test output value of the automated test; the test unit is used to use the obtained reasonable interval to test the tested application software system; and the judging unit is used to judge whether the tested application software system according to the test results An exception occurs.

Optionally, the device further includes: a setting unit, used to set the time interval and times of automated testing; the testing unit is also used to perform automated testing on the tested application software system at every set time interval and obtain The output value is tested until the set number of times is reached; the analysis unit is configured to obtain the reasonable interval according to the output value of the test.

Optionally, the process of the analysis unit obtaining a reasonable interval according to the test output value specifically includes: counting the value ranges of multiple test output values obtained by the automated test; removing discrete points from the value ranges; The value range after removing the discrete points is determined as the reasonable interval.

Optionally, the process of the testing unit testing the tested application software system using the obtained reasonable interval specifically includes: testing the tested application software system to obtain a test output value of the test; judging the Whether the test output value falls within the range of the reasonable interval; when the test output value falls within the range of the reasonable interval, it is determined that the test result of the tested application software system is passed; when the When the test output value is not within the reasonable interval, it is determined that the test result of the tested application software system is failed.

Optionally, the judging unit is further configured to, when it is judged that an abnormality occurs in the tested application software system, determine the cause of the abnormality in conjunction with whether a version update of the application software in the tested application software system occurs.

The application software testing method and device proposed by the present invention obtain a reasonable interval of the test output value through multiple tests, and then judge whether the subsequent test of the tested application software system is passed according to the reasonable interval, and then judge whether the tested application software system is The exception occurred and the reason for the exception.

Its beneficial effects are mainly reflected in the following two aspects: For the tracking system itself, it can be used as a way of regression testing. When the version of the tracking system itself is updated, it can be verified by testing whether the output value falls within a reasonable range. Whether the version update is abnormal; if the application software (tracking system) is used to manage another application software (tracked system), for the tracked system, when the tracking system is not updated, if the test output value does not fall within a reasonable range If it is within the range, it can be judged that the tracked system has an update exception or a failure. In addition, it can also be combined with other tracking functions in the tracking system to find specific abnormal locations in the tracked system, and then provide data support for troubleshooting or optimizing the system.

Description of drawings

FIG. 1 is an environmental architecture diagram of an application software testing device for implementing various embodiments of the present invention.

FIG. 2 is a schematic diagram of a calling relationship between microservices in the present invention.

FIG. 3 is a block diagram of an application software testing device proposed in the first embodiment of the present invention.

FIG. 4 is a flowchart of an application software testing method proposed by the second embodiment of the present invention.

FIG. 5 is a flow chart of an application software testing method proposed by the third embodiment of the present invention.

FIG. 6 is a flow chart of an application software testing method proposed by the fourth embodiment of the present invention.

FIG. 7 is a detailed flowchart of step S602 in FIG. 6 .

FIG. 8 is a detailed flowchart of step S604 in FIG. 6 .

FIG. 9 is a detailed flowchart of steps S608 and S610 in FIG. 6 .

The realization of the purpose of the present invention, functional characteristics and advantages will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Detailed ways

In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer and clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

As shown in FIG. 1 , it is an environment architecture diagram of an application software testing device 100 implementing various embodiments of the present invention. The application software testing device 100 is deployed in a server or other suitable hardware architecture.

The application software testing device 100 is used to test the application software system 120 under test. The tested application software system 120 may be one application software, or a public system composed of multiple application software. In this embodiment, the tested application software system 120 includes a tracking system 122 and a tracked system 124 , and the tracked system 124 includes a plurality of microservices 126 .

During the process of testing the application software system 120 under test by the application software testing device 100 , there may also be mutual calls between the multiple microservices 126 inside the tracked system 124 .

Taking the invocation relationship between microservices 126 shown in Figure 2 as an example, in the tracked system 124, microservice A can not only be invoked and executed by the client, but also be invoked by microservice B and microservice C, and microservice A can also be invoked by the client. The database can be called. The client, microservice B and microservice C cannot directly call the database, but need to call the database through microservice A and return the result.

first embodiment

As shown in FIG. 3 , the first embodiment of the present invention proposes an application software testing device 100 .

In this embodiment, the application software testing device 100 includes a setting unit 102 , a testing unit 104 , an analyzing unit 106 , an acquiring unit 108 and a judging unit 110 .

The setting unit 102 is used to set the time interval and the number of times for automatic testing of the application software system 120 under test.

For example, the time interval of the automated test is set to 15 minutes, and the number of times is set to 10,000 times.

The testing unit 104 is used for performing an automatic test on the tested application software system 120 and obtaining a test output value every set time interval until the set number of times is reached.

The analysis unit 106 is configured to obtain the reasonable interval according to the test output value.

Specifically, each time the test unit 104 performs an automated test on the application software system 120 under test, a test output value can be obtained, and the analysis unit 106 obtains a large amount of test output value data from multiple automated tests, and then analyzes the output value data , to obtain a reasonable interval for the test output value data.

The specific process for the analysis unit 106 to obtain the reasonable interval includes: counting the value ranges of multiple test output values obtained by the automated test; removing discrete points from the value range; determined as the reasonable interval. The discrete points are abnormal data in the value range that are obviously different from most of the test output values.

For example, 20,000 test output values are obtained by performing 20,000 automated tests on the tested application software system 120 . The range of the 20,000 test output values is counted, and it is found that 19,998 of them are in the range of [4, 7], and 2 test output values are 14 and 16 respectively. The above two test output values 14 and 16 are discrete points, which need to be removed from the value range. After the discrete points 14 and 16 are removed, the obtained value range [4, 7] is the reasonable interval corresponding to the test output value of the tested application software system 120 .

The obtaining unit 108 is used to obtain a reasonable interval of the test output value of the tested application software system 120 .

Specifically, the reasonable interval obtained by the analysis unit 106 can be saved in a storage area such as a database, and when the application software system under test 120 needs to be tested later, the acquisition unit 108 obtains the corresponding interval of the application software system under test 120 from the database. reasonable range.

The testing unit 104 is also used to test the tested application software system 120 using the acquired reasonable interval.

Specifically, the testing unit 104 tests the tested application software system 120 to obtain a test output value of the test. Then it is judged whether the test output value falls within the range of the reasonable interval. When the test output value falls within the reasonable range, the testing unit 104 determines that the test result of the tested application software system 120 is passed. When the test output value is not within the reasonable interval, the testing unit 104 determines that the test result of the tested application software system 120 is failed.

The judging unit 110 is used for judging whether the tested application software system 120 is abnormal according to the test result.

Specifically, when the result of the test is passed, the judging unit 110 judges that no abnormality occurs in the tested application software system. When the result of the test is not passed, the judging unit 110 judges that the tested application software system is abnormal.

Further, the judging unit 110 is also used for judging the cause of the abnormality in conjunction with whether the application software in the tested application software system 120 has a version update when it is judged that the tested application software system 120 is abnormal.

Specifically, whether the version update of the application software may include whether the tracking system 122 has a version update, and whether the tracked system 124 has a version update. When the version update of the tracking system 122 occurs, the cause of the abnormality may be that the version update of the tracking system 122 is abnormal or malfunctions. When the version update of the tracking system 122 does not occur but the version update of the tracked system 124 occurs, the cause of the abnormality may be that the version update of the tracked system 124 is abnormal or a fault occurs. In addition, other tracking functions in the tracking system 122 can also be combined to find specific abnormal locations in the tracked system 124, thereby providing data support for troubleshooting or optimizing the system.

second embodiment

As shown in FIG. 4 , the second embodiment of the present invention proposes an application software testing method. The method includes the steps of:

S400. Obtain a reasonable interval of the test output value of the tested application software system.

Specifically, the reasonable interval is obtained by analyzing the test output values of the automated tests performed on the tested application software system for multiple times. That is to obtain a large amount of test output value data from multiple automated tests, and then analyze the output value data to obtain a reasonable range of test output value data. Determination of the reasonable interval requires a large number of automated tests to obtain, and the number of executions in this embodiment is at least 10,000 times. When the sampling times (automated testing times) are sufficient, the reasonable interval is closer to the real value. For example, if 20,000 automated tests are performed on the application software system under test, and 19998 of the obtained test output values are in the range of [4, 7], it means that [4, 7] is the test output of the application software system under test A reasonable interval corresponding to the value.

S402. Use the acquired reasonable interval to test the application software system under test.

Specifically, after obtaining the reasonable interval, a corresponding test may be performed on the tested application software system to obtain a test output value of the test, and then determine whether the test output value falls within the range of the reasonable interval. When the test output value falls within the range of the reasonable interval, it is determined that the test result of the tested application software system is passed. When the test output value is not within the reasonable range, it is determined that the test result of the tested application software system is failed.

S404. Determine whether the tested application software system is abnormal according to the test result.

Specifically, when the result of the test is passed, it is determined that no abnormality occurs in the tested application software system. When the result of the test fails, it is determined that the tested application software system is abnormal.

third embodiment

As shown in FIG. 5 , the third embodiment of the present invention proposes an application software testing method. In the method, a reasonable range of test output values is obtained by automatically testing the tested application software system for many times, and then the subsequent tested application software system is tested according to the reasonable range.

S500, through multiple automated tests to obtain a reasonable range of test output values.

Specifically, the time interval and the number of times of the automated test can be set, and the automated test is performed on the application software system under test according to the set time interval until the set number of times is reached. Obtain a large amount of test output value data from multiple automated tests, and then analyze the output value data to obtain a reasonable range of test output value data. Determination of the reasonable interval requires a large number of automated tests to obtain, and the number of executions in this embodiment is at least 10,000 times. When the sampling times (automated testing times) are sufficient, the reasonable interval is closer to the real value.

In this embodiment, the test output value is the number of times a certain microservice under test in the application software system under test is invoked within a predetermined period of time. It should be noted that, in other embodiments, the test output value may also be the output result of other test items.

S502. Test the application software system under test, and judge whether the test passes according to the reasonable interval.

Specifically, at the first time point, the number of invocations of the microservice under test is acquired for the first time, and then a call to the microservice under test is initiated, and at the second time point, the number of invocations of the microservice under test is acquired for the second time, Write the call times obtained twice into the result file. Calculate the difference between the number of calls obtained for the second time and the number of calls obtained for the first time, and the difference is the test output value of this test. When the difference falls within the reasonable range, it is judged that the test result of the microservice under test is passed. When the difference is not within the reasonable range, it is judged that the test result of the microservice under test is failed.

S504. Determine whether the tested application software system is abnormal according to whether the test is passed.

Specifically, when the result of the test is passed, it is determined that no abnormality occurs in the tested application software system. When the result of the test fails, it is determined that the tested application software system is abnormal.

Further, the method also includes the steps of:

S506. When it is determined that an abnormality occurs, determine the cause of the abnormality in combination with whether a version update of the application software in the tested application software system occurs.

In this embodiment, the application software in the tested application software system includes a tracking system and a tracked system. Therefore, whether the version update of the application software may include whether a version update occurs in the tracking system and whether a version update occurs in the tracked system. When the version update of the tracking system occurs, the cause of the abnormality may be that the version update of the tracking system is abnormal or a fault occurs. When the version update of the tracking system does not occur but the version update of the tracked system occurs, the cause of the abnormality may be that the version update of the tracked system is abnormal or a fault occurs. In addition, it can also be combined with other tracking functions in the tracking system to find specific abnormal locations in the tracked system, and then provide data support for troubleshooting or optimizing the system.

Fourth embodiment

As shown in FIG. 6 , the fourth embodiment of the present invention proposes an application software testing method, including the process of obtaining a reasonable range and the process of testing using the reasonable range.

The method comprises the steps of:

S600, setting the time interval and times of automated testing.

For example, the time interval of the automated test is set to 15 minutes, and the number of times is set to 10,000 times.

S602. Perform automated testing on the application software system under test at every set time interval until the set number of times is reached.

For example, refer to FIG. 7 , which is a detailed flow chart of step S602. The process includes steps:

S700, setting the time interval and times of automated testing.

S702, judging whether the set time interval has been reached. If yes, execute step S704. If not, continue to wait.

S704. Perform an automated test on the tested application software system to obtain a test output value.

S706, judging whether the number of automated tests reaches the set number of times. If so, the process ends. If not, return to step S702, and execute the next automated test after waiting for the set time interval.

Returning to FIG. 6 , in step S604 , a reasonable interval is obtained according to the test output values of the set number of automated tests.

Specifically, each time an automated test is performed on the application software system under test, a test output value can be obtained, a large amount of test output value data is obtained from multiple automated tests, and then the output value data is analyzed to obtain the test output value data. Reasonable range.

Take the test output value as an example of the number of calls of a certain microservice under test within a predetermined period of time in the application software system under test. Through the analysis of a large number of test output value data, a reasonable interval [a, b] of the number of calls can be obtained in almost the same time period Δt. Among them, a is the reasonable lower limit of the output value, and b is the reasonable upper limit of the output value, such as [4, 7]. Due to reasons such as network delay, it can only be guaranteed that the time period Δt of each interval is almost the same. According to the test results, the difference in Δt is very small, and this small difference will not cause a large number of calls, so it will not affect the accuracy of the test data. Determination of the reasonable interval requires a large number of automated tests to obtain, and the number of executions in this embodiment is at least 10,000 times. When the sampling times (automated testing times) are sufficient, the reasonable interval is closer to the real value.

Referring to FIG. 8 , it is a detailed flow chart of step S604. The process includes steps:

S800. Count the value ranges of the multiple test output values obtained by the automated test.

For example, by performing 20,000 automated tests on the application software system under test, 20,000 test output values are obtained. The range of the 20,000 test output values is counted, and it is found that 19,998 of them are in the range of [4, 7], and 2 test output values are 14 and 16 respectively.

S802. Remove discrete points from the value range.

Specifically, the discrete points are abnormal data in the value range that are obviously different from most test output values. For example, the above two test output values 14 and 16 are discrete points, which need to be removed from the value range.

S804. Determine the value range after removing the discrete points as the reasonable interval.

Specifically, after the discrete points are removed from the value ranges of the multiple test output values obtained by the automated test, what remains is a reasonable interval corresponding to the test output values of the tested application software system. For example, after the discrete points 14 and 16 are removed from the above value range, the obtained value range [4, 7] is a reasonable interval corresponding to the test output value of the tested application software system.

The above steps S600-S604 are the process of obtaining a reasonable interval. Using the reasonable interval, it is possible to judge whether the test is passed or not according to whether the test output value falls within the range of the reasonable interval during subsequent application software testing. That is, further, the application software testing method also includes a process of using the reasonable interval for testing, and the process specifically includes the steps of:

S608. Execute a test on the application software system under test.

In this embodiment, the test output value is the number of times a certain microservice under test in the application software system under test is invoked within a predetermined period of time. It should be noted that, in other embodiments, the test output value may also be the output result of other test items.

Specifically, at the first time point, the number of invocations of the microservice under test is acquired for the first time, and then a call to the microservice under test is initiated, and at the second time point, the number of invocations of the microservice under test is acquired for the second time, Write the call times obtained twice into the result file. For example, for microservice A in Figure 2, at time _t1 , the statistics of the number of calls of microservice A is obtained for the first time as calls_before, and then a call to microservice A is initiated. When the time is t ₂ (t ₂ >t ₁ ), the number of calls to microservice A is obtained for the second time as calls_after, and both the calls_before and calls_after data are written to the result file.

S610, judging whether the test is passed according to the reasonable interval.

Specifically, the difference between the number of calls obtained for the second time and the number of calls obtained for the first time is calculated, which is the test output value of this test. When the difference falls within the reasonable range, it is judged that the test result of the microservice under test is passed. When the difference is not within the reasonable range, it is judged that the test result of the microservice under test is failed. For example, assuming that the reasonable interval of the test output value obtained from the automated test of microservice A is [Δcalls _min , Δcalls _max ], when the value of Δcalls=calls_after-calls_before falls within the range of the reasonable interval, the test passes, When the value of Δcalls is not within the reasonable range, the test fails.

Referring to FIG. 9 , it is a detailed flow chart of steps S608 and S610. The process includes steps:

S900. Obtain the number of invocations of the microservice under test for the first time at the first time point.

S902. Initiate a call to the microservice under test.

S904. Obtain the number of invocations of the microservice under test for the second time at a second time point.

S906. Write the obtained invoking times twice into the result file.

S908. Calculate the difference between the number of calls obtained for the second time and the number of calls obtained for the first time.

S910, judging whether the difference falls within a reasonable interval. If yes, execute step S912. If not, execute step S914.

S912. Determine that the test result of the tested microservice is passed.

S914. Determine that the test result of the tested microservice is failed.

Returning to FIG. 6 , in step S612 , it is judged whether the tested application software system is abnormal according to whether the test is passed or not.

Specifically, when the result of the test is passed, it is determined that no abnormality occurs in the tested application software system. When the result of the test fails, it is determined that the tested application software system is abnormal.

S614. When it is determined that an abnormality occurs, determine the cause of the abnormality in combination with whether a version update of the application software in the tested application software system occurs.

In this embodiment, the application software in the tested application software system includes a tracking system and a tracked system. Therefore, whether the version update of the application software may include whether a version update occurs in the tracking system and whether a version update occurs in the tracked system. When the version update of the tracking system occurs, the cause of the abnormality may be that the version update of the tracking system is abnormal or a fault occurs. When the version update of the tracking system does not occur but the version update of the tracked system occurs, the cause of the abnormality may be that the version update of the tracked system is abnormal or a fault occurs.

For example, the tracking system is an APM (Application Performance Management, application performance management) system, and the tracked system is an EMS (Element Management System, network element management system) system. The reasonable interval obtained through multiple tests is [4, 7].

In the exception call scenario 1, the tracked system is in normal use and the version is not updated. Assuming that the developer modifies the code of the tracking system, the merged new code has problems. Except for the tested microservice A, other microservices cannot be tracked. Then the number of times the client calls microservice A increases by 1 each time. The test client calls the output value data of microservice A twice, and the output value obtained is 2, which is not within the reasonable range [4, 7], but falls on the left side of the reasonable range. It can be concluded that, for the tracking system itself, the application software testing method proposed by the present invention can be used as a regression testing method, and when the tracking system itself is iteratively updated, the function of the updated version can be verified.

In the second abnormal call scenario, other modules call microservice A continuously in an infinite loop. The test client calls the output value data of microservice A, and the obtained output values are 64 and 69, which do not fall within the reasonable interval [4, 7], but fall on the right side of the reasonable interval. From this, it can be concluded that when the tracking system itself has not changed, if the test output value does not fall within the reasonable range, it can be judged that the tracked system has changed.

For another example, the tracking system is an APM system, and the alarm micro-service system is used as the tracked system. Under normal circumstances, once the historical alarm is queried, the number of times the user calls the micro-service fm-history is increased by 1. That is to say, the reasonable interval of the test output value is a fixed value, namely [1, 1].

In the abnormal call scenario, suppose that the version update of the alarm microservice system introduces problems, such as internal modules (or services) that continuously call the query history alarm microservice. The test client calls to query the output value data of the historical alarm microservice, and the obtained output values are 12 and 14, which do not fall within the reasonable interval [1, 1], but fall on the right side of the reasonable interval.

Therefore, the main value of this method is reflected in the following two aspects: For the tracking system itself, it can be used as a way of regression testing. When the version of the tracking system itself is updated, the output value of the test is within a reasonable range. It can be verified whether the version update is abnormal; if the application software (tracking system) is used to manage another application software (tracked system), for the tracked system, when the tracking system is not updated, if the test output value does not fall Within a reasonable range, it can be judged that the tracked system has an update abnormality or a failure. Combined with other tracking functions of the tracked system, such as the trace recording module of the call stack, the specific abnormal location (faulty module) of the tracked system can be found, and then provide data support for troubleshooting or optimizing the system.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods in the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on such an understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products are stored in a storage medium (such as ROM/RAM, disk, CD) contains several instructions to make a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in various embodiments of the present invention.

The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, and the scope of rights of the present invention is not limited thereto. The serial numbers of the above embodiments of the present invention are for description only, and do not represent the advantages and disadvantages of the embodiments. In addition, although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.

Those skilled in the art can realize the present invention with many variants without departing from the scope and spirit of the present invention, for example, the features of one embodiment can be used in another embodiment to obtain another embodiment. All modifications, equivalent replacements and improvements made within the technical conception of the application of the present invention shall fall within the scope of rights of the present invention.

Claims (10)

1. An application software testing method, the method comprising steps: Obtaining a reasonable range of test output values of the tested application software system, the reasonable range is obtained by analyzing the test output values of the tested application software system for multiple times of automated testing; Use the obtained reasonable interval to test the application software system under test; According to the test result, it is judged whether the tested application software system is abnormal. 2. The application software testing method according to claim 1, wherein the process of obtaining the reasonable interval according to the analysis of the test output values of the automated testing of the application software system under test for multiple times specifically includes: Set the time interval and number of automated tests; Perform automated testing on the tested application software system and obtain test output values at every set time interval until the set number of times is reached; The reasonable interval is obtained according to the test output value. 3. The application software testing method according to claim 2, wherein the step of obtaining a reasonable interval according to the test output value specifically comprises: Count the value ranges of multiple test output values obtained by the automated test; removing discrete points from the range of values; The value range after removing the discrete points is determined as the reasonable interval. 4. The application software testing method according to claim 1, wherein the step of testing the application software system under test using the acquired reasonable interval specifically comprises: Testing the application software system under test to obtain a test output value of the test; judging whether the test output value falls within the range of the reasonable interval; When the test output value falls within the range of the reasonable interval, it is determined that the test result of the tested application software system is passed; When the test output value is not within the reasonable range, it is determined that the test result of the tested application software system is failed. 5. application software testing method according to claim 1, is characterized in that, this method also comprises the step: When it is determined that the tested application software system is abnormal, the cause of the abnormality is determined in combination with whether the application software in the tested application software system has a version update. 6. An application software testing device, characterized in that the device comprises: An acquisition unit, configured to acquire a reasonable range of test output values of the tested application software system, the reasonable range is obtained by analyzing the test output values of the tested application software system for multiple times of automated testing; a test unit for testing the application software system under test using the obtained reasonable interval; and The judging unit is configured to judge whether the tested application software system is abnormal according to the test result. 7. The application software testing device according to claim 6, characterized in that the device further comprises: Setting unit, used to set the time interval and times of automated testing; The test unit is also used to perform automated tests on the tested application software system and obtain test output values at every set time interval until the set number of times is reached; An analysis unit, configured to obtain the reasonable interval according to the test output value. 8. The application software testing device according to claim 7, wherein the process of the analysis unit obtaining a reasonable interval according to the test output value specifically comprises: Count the value ranges of multiple test output values obtained by the automated test; removing discrete points from the range of values; The value range after removing the discrete points is determined as the reasonable interval. 9. The application software testing device according to claim 6, wherein the process of testing the application software system under test by the testing unit using the obtained reasonable interval specifically includes: Testing the application software system under test to obtain a test output value of the test; judging whether the test output value falls within the range of the reasonable interval; When the test output value falls within the range of the reasonable interval, it is determined that the test result of the tested application software system is passed; When the test output value is not within the reasonable range, it is determined that the test result of the tested application software system is failed. 10. The application software testing device according to claim 6, wherein the judging unit is further configured to combine the application software in the tested application software system when it is judged that the tested application software system is abnormal. Whether the software has been updated to determine the cause of the abnormality.