CN111881014B - System test method, device, storage medium and electronic equipment - Google Patents
System test method, device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN111881014B CN111881014B CN202010520816.2A CN202010520816A CN111881014B CN 111881014 B CN111881014 B CN 111881014B CN 202010520816 A CN202010520816 A CN 202010520816A CN 111881014 B CN111881014 B CN 111881014B
- Authority
- CN
- China
- Prior art keywords
- service
- tested
- exception
- server
- testing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Prevention of errors by analysis, debugging or testing of software
- G06F11/3668—Testing of software
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/26—Functional testing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Prevention of errors by analysis, debugging or testing of software
- G06F11/3668—Testing of software
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application relates to the technical field of software testing, and provides a system testing method, a system testing device, a storage medium and electronic equipment. The system testing method comprises the following steps: injecting server abnormity into a server running with a system to be tested; and monitoring the operation index of the service to be tested contained in the system to be tested, and testing the interface function of the service to be tested. The method actively injects server abnormity into a server running with a system to be tested, simulates the real abnormal condition of the server, monitors the running index of the service to be tested, tests the interface function of the service to be tested, and then performs statistical analysis based on the running index and the test result, so that the working condition of the service to be tested and the system to be tested under the condition that the server is abnormal can be determined, the conventional test range of a business system is expanded, and the method is beneficial to exposing as early as possible and solving the problems in the test process.
Description
Technical Field
The invention relates to the technical field of software testing, in particular to a system testing method, a system testing device, a storage medium and electronic equipment.
Background
The service system is tested before being formally on line, but in the prior art, the service system is basically tested under the condition that the server environment is relatively stable, the testing range is relatively limited, and the service system has more faults after being on line.
Disclosure of Invention
An embodiment of the present invention is directed to a system testing method, an apparatus, a storage medium, and an electronic device, so as to solve the above technical problems.
In order to achieve the above purpose, the present application provides the following technical solutions:
in a first aspect, an embodiment of the present application provides a system testing method, including: injecting server abnormity into a server running with a system to be tested; and monitoring the operation index of the service to be tested contained in the system to be tested, and testing the interface function of the service to be tested.
In the method, firstly, the server abnormity is injected into the server running with the system to be tested actively, the real abnormal condition of the server is simulated, then the running index of the service to be tested is monitored, the interface function of the service to be tested is tested, and then the statistical analysis is carried out based on the obtained running index and the test result, so that the working condition (for example, whether the server can work normally, problems exist in the working process and the like) of the system to be tested under the condition that the server is abnormal can be determined. Or the method provides a new scheme for testing the working condition of the service system under the condition that the server is abnormal, expands the conventional testing range of the service system and fills the blank in the prior art.
Furthermore, the method is beneficial to exposing as early as possible, solving the problems in the test process and reducing the probability of the fault of the system to be tested after the system to be tested is actually on line.
In one implementation form of the first aspect, the method further comprises: injecting a service exception into the service to be tested; and monitoring the operation index of the service to be tested, and testing the interface function of the service to be tested.
In the implementation mode, the actual abnormal condition of the service to be tested is simulated by actively injecting the service abnormity into the service to be tested, so that the working condition of the service to be tested under the abnormal condition can be tested.
In an implementation manner of the first aspect, after the injecting a service exception to the service to be tested, the method further includes: determining related services of the service to be tested, wherein the related services comprise upstream services and/or downstream services of the service to be tested; and monitoring the operation index of the related service, and testing the interface function of the related service.
Services in the system to be tested may have a dependency relationship, a service which is depended on by the current service is called an upstream service of the current service, a service which is depended on by the current service is called a downstream service of the current service, and if the current service is abnormal, the downstream service is abnormal, and the upstream service is abnormal. Therefore, after the exception is injected into the service to be tested, the service to be tested can be tested, and the working conditions of the upstream and/or downstream service of the service to be tested can be tested, so that the influence of the exception in the service call chain can be comprehensively evaluated.
In an implementation manner of the first aspect, the system to be tested is deployed on a DevOps platform, and an operation and maintenance monitoring system and an automated testing system are also deployed on the DevOps platform; the step of injecting server abnormity into the server running with the system to be tested comprises the following steps: calling a first exception acquisition interface provided by the DevOps platform to acquire a server exception list, and calling a first exception injection interface provided by the DevOps platform to inject server exceptions in the server exception list into the server; the monitoring the operation index of the service to be tested contained in the system to be tested and testing the interface function of the service to be tested comprise: calling a service instance acquisition interface provided by the DevOps platform to acquire a service list formed by services deployed on the DevOps platform, and determining the service to be tested contained in the system to be tested in the service list; and monitoring the operation index of the service to be tested by using the operation and maintenance monitoring system, and testing the interface function of the service to be tested by using the automatic testing system.
In an implementation manner of the first aspect, the system to be tested is deployed on a DevOps platform, and an operation and maintenance monitoring system and an automated testing system are also deployed on the DevOps platform; the injecting the service exception into the service to be tested includes: calling a service instance acquisition interface provided by the DevOps platform to acquire a service list formed by services deployed on the DevOps platform, and determining the service to be tested contained in the system to be tested in the service list; calling a second exception acquisition interface provided by the DevOps platform to acquire a service exception list, and calling a second exception injection interface provided by the DevOps platform to inject service exceptions in the service exception list into the service to be tested; the monitoring the operation index of the service to be tested and testing the interface function of the service to be tested comprise: and monitoring the operation index of the service to be tested by using the operation and maintenance monitoring system, and testing the interface function of the service to be tested by using the automatic testing system.
The DevOps platform can be understood as a tool for installing and deploying applications, and the DevOps emphasizes how to efficiently organize how to complete the life cycle management of software through automated tool cooperation and communication between teams, so as to deliver more stable software more quickly and frequently. In the two implementation modes, the system to be tested, the operation and maintenance monitoring system and the automatic testing system related in the scheme are all deployed on the DevOps platform for unified management, and meanwhile, the DevOps platform also encapsulates interfaces for external calling, such as service instance acquisition, exception injection and the like, so that operations of exception injection, service testing and the like become simple, and a user does not need to write complex instructions by himself.
In one implementation of the first aspect, the server exception includes at least one of: disk exception, IO exception, memory exception, CPU exception, power off, and network outage.
In one implementation form of the first aspect, the service exception includes at least one of: database access timeouts, array violations, request exceptions, and response exceptions.
In a second aspect, an embodiment of the present application provides a system test apparatus, including: the injection module is used for injecting server abnormity to a server running the system to be tested; and the test module is used for monitoring the operation index of the service to be tested contained in the system to be tested and testing the interface function of the service to be tested.
In a third aspect, an embodiment of the present application provides a computer-readable storage medium, where computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are read and executed by a processor, the computer program instructions perform the method provided by the first aspect or any one of the possible implementation manners of the first aspect.
In a fourth aspect, an embodiment of the present application provides an electronic device, including: a memory in which computer program instructions are stored, and a processor, where the computer program instructions are read and executed by the processor to perform the method provided by the first aspect or any one of the possible implementation manners of the first aspect.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 illustrates a schematic diagram of a system under test according to an embodiment of the present application;
FIG. 2 is a flow chart illustrating a system testing method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a test platform provided in an embodiment of the present application;
FIG. 4 is a functional block diagram of a system test apparatus according to an embodiment of the present disclosure;
fig. 5 shows a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element. The terms "first," "second," "third," and the like are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
The system test method provided by the embodiment of the application tests part or all of services in a certain service system, the tested service system detects a system to be tested, and the service to be tested in the system to be tested is called the service to be tested for short. The service to be tested may be a traditional service or a micro service, which is not limited in the present application.
Fig. 1 shows a schematic diagram of a system under test according to an embodiment of the present application. Referring to fig. 1, the system under test 100 includes several components, namely, a front-end service 110, a business-side micro-service 120, a core-side micro-service 130, and a middleware 140, and the system under test 100 may be deployed on a server or a server cluster. The system under test 10 may be used for different business scenarios, and is described below as being used for a video structured scenario, i.e., the system under test 100 is a video structured system. The video structuring can be realized by intelligently analyzing original video streams and video files, extracting key information from the original video streams and video files, and performing semantic description on texts. One possible workflow for a video structuring system is as follows:
when a user wants to perform video structuring, a camera can be selected from a Web page provided by the front-end service 110 to acquire a video stream or upload a video file, an operation logic of a control on the Web page can be supported by the service-side micro-service 120, the service-side micro-service 120 transmits video data to the core-side micro-service 130 after acquiring the video data, and the core-side micro-service calls a corresponding algorithm to perform structured analysis on the video data and returns an analysis result to the service-side micro-service 120 for storage. The middleware 140 has a tool property and can be implemented as a service having a specific function, for example, Redis (a kind of high performance database), RabbitMQ (a kind of message queue), Nginx (a kind of Web server), and the like. The middleware 140 may be divided into a service-side middleware and a core-side middleware for the service-side micro-service 120 and the core-side micro-service 130, respectively. The deployment sequence of each component in the video structuring system is as follows: core side middleware, core side micro-service 130, business side middleware, business side micro-service 120, front end service 110.
Services under test in a video structured system may include a business side microservice 120, a core side microservice 130, and middleware. The inventor finds the following defects in the video structured system during the research of the test work of the video structured system:
defect one: the test of the service to be tested is established on the premise that indexes such as CPU, IO, network environment and the like of the server are relatively stable, the server is abnormal, and a corresponding test scheme is lacked if the video structured system and the service to be tested contained in the system can work normally.
And defect two: the services related to the video structuring system have certain dependency relationship, and if one service is abnormal, the service depending on the service may also be abnormal. For example, according to the foregoing description, when a video structured system is deployed, the service-side middleware should be deployed before the service-side micro-service is started, and if, in a certain deployment, the service-side middleware Redis executed after the service-side micro-service is started (which may be regarded as an exception of Redis in the deployment), the service-side micro-service that depends on the Redis may not work normally. At present, a scheme for effectively testing an upstream service and/or a downstream service of a certain abnormal service to determine whether the abnormal service can work normally is lacked.
The above defects are discovered after the inventor researches existing test schemes for a long time, and therefore, the discovery process and the solutions proposed by the embodiments of the present application in the following for the above defects should be contributions of the inventor to the present application.
It is understood that fig. 1 is only an example, and the system under test involved in the solution of the present application does not necessarily adopt the architecture shown in fig. 1, so fig. 1 should not be taken as a limitation to the scope of the present application, but the two drawbacks set forth above have generality.
Fig. 2 shows a flowchart of a system testing method according to an embodiment of the present application. The method may be performed by an electronic device, and fig. 5 shows a possible structure of the electronic device, which may be referred to as set forth below with reference to fig. 5. Referring to fig. 2, the method includes:
step S200: and injecting server exception into the server running with the system to be tested.
Step S210: and monitoring the operation index of the service to be tested contained in the system to be tested, and testing the interface function of the service to be tested.
The server exception may include, but is not limited to, a disk exception (e.g., a bad track occurs in a disk, a disk remaining space is too low, etc.) of the server, an IO exception (e.g., an IO device error, etc.), a memory exception (e.g., a memory occupancy is too high), a CPU exception (e.g., a CPU occupancy is too high), a power failure, a network disconnection, etc., and the possible exceptions of the server may be combed in advance, and these exceptions are packaged and stored, and an interface is provided to the outside so that these exceptions may be obtained when exception injection is required. The injection server exception in step S200 refers to an active manufacturing server exception scenario used for simulating a real exception condition of the server, for example, a CPU exception is injected into the server through a certain interface, an internal code of the interface tries to increase an occupancy rate of the CPU, and the CPU of the manufacturing server is in an exception condition, so that the operating condition of the system under test can be tested under such an exception condition.
The operation index of the service to be tested in step S210 may include an index of a single service to be tested, or may include an overall index of all services to be tested (or systems to be tested). These indicators include, but are not limited to, CPU occupancy, IO rate, memory occupancy, virtual memory (e.g., SWAP under Linux) occupancy, disk occupancy, and the like. The indexes are mainly used for describing the running condition of the service to be tested from the level of the external environment, for example, if a certain service to be tested occupies nearly 100% of the CPU for a long time, the running of the service to be tested can be reasonably inferred to be abnormal. The operation index of the service to be tested can be monitored by an operation and maintenance monitoring system, such as a Prometheus system, an infiluxdb system, and the like.
Whether the function of the service to be tested is normal cannot be known only based on the operation index, for example, although a certain service to be tested has a normal operation index, the internal data has an error, and the interface provided by the service to be tested is called to always return the error data to the client, which still belongs to an abnormal condition. Therefore, it is also necessary to perform a functional test on the interface of the service under test. The interface test can adopt an automatic test system, such as interface test tools of Postman, RESTClient and the like.
After step S210 is executed, statistical analysis is performed by combining the operation index obtained through monitoring and/or the test result obtained through the interface test, so as to determine the working condition of the service to be tested, including whether the service to be tested can work normally or not, what kind of abnormality exists if the service to be tested cannot work normally, and further determine a solution corresponding to the abnormality, that is, how to maintain the normal operation of the service to be tested under the condition that the server is abnormal. The statistical analysis process and the process of solving the abnormality may be performed in a manual checking manner, or may be performed by automatically analyzing and determining the solution according to the abnormal phenomenon, which is not limited in the present application. Because the system to be tested is composed of a plurality of services, the working condition of the service to be tested is determined, namely the working condition of the whole system to be tested is determined.
Since the server may have a plurality of abnormal conditions, it may be necessary to inject different kinds of abnormal conditions into the server for testing respectively. In one implementation, to avoid interference, only one type of server exception is injected each time, the completion of injection immediately proceeds to determine a solution corresponding to the exception, and the next type of server exception is injected after the solution is successfully determined.
To sum up, in the method shown in fig. 2, firstly, server anomaly is actively injected into the server so as to simulate a real abnormal condition of the server, then the operation index of the service to be tested is monitored, the interface function of the service to be tested is tested, and then statistical analysis is performed based on the obtained operation index and the test result, so that the working condition of the system to be tested can be determined when the server is abnormal. In other words, the method provides a new scheme for testing the working condition of the service system under the condition that the server is abnormal, remarkably expands the testing range of the service system in the conventional method, and fills the blank in the prior art. On the other hand, the abnormal injection in the method is beneficial to exposing as early as possible, solves the problem that the system to be tested possibly occurs when the server is abnormal, and reduces the probability that the system to be tested actually fails after the system to be tested is on line.
The server exception is mainly an exception in an external environment where the service to be tested is located, and is biased to the equipment level. Besides server exception, another type of exception is service exception, which is mainly an exception generated inside the service to be tested and biased to the code logic or data level of the service to be tested.
In some implementation manners, besides injecting the exception into the server, the service exception can be injected into the service to be tested so as to simulate the real exception condition of the service to be tested, and after the exception is injected, the operation index of the service to be tested is monitored, and the interface function of the service to be tested is tested. And then, performing statistical analysis based on the operation indexes obtained by monitoring and/or the test results obtained by interface testing, so as to determine the working condition of the service to be tested, and further determine a solution corresponding to the service abnormity, namely, a solution for how to eliminate the service abnormity or how to enable the service to be tested to still maintain normal working under the condition that the service is abnormal. Because the system to be tested is composed of a plurality of services, the working condition of the service to be tested is determined, namely the working condition of the whole system to be tested is determined. The abnormal service injection for the service to be tested is also beneficial to exposing as early as possible, solving the problem that the system to be tested possibly occurs when the service is abnormal, and reducing the probability that the system to be tested breaks down after being actually on line.
It should be noted that the test for the service exception and the test for the server exception are performed independently, that is, the two are allowed to select the service for testing respectively, and also allowed to select an independent time period for testing respectively. For example, the same service to be tested may be selected, the server is tested for abnormality first, and then the server is tested for abnormality, and when the service is tested for abnormality, the server may be restored to a normal state.
The service exception may include, but is not limited to, a database access timeout, an array violation, a request exception (e.g., request parameter is illegal), a response exception (e.g., response data error), and the like, and possible exceptions in the service may be combed in advance, and these exceptions may be packaged and saved, and an interface may be provided to the outside so that these exceptions may be obtained when exception injection is needed. The injection of the service exception into the service to be tested means that an active service exception scene is manufactured to simulate a real exception condition of the service, for example, an incorrect request parameter is injected into the service to be tested through a certain interface, so that an exception condition caused by an illegal request sent by a client in practice can be simulated, and thus the running condition of the service to be tested can be tested under the exception condition.
As for the operation index of the service to be tested and the interface function of the service to be tested after the abnormal injection, the method is similar to the step S210, and will not be repeated.
Since there may be many types of service anomalies and there may be many services to be tested, it may be necessary to inject different types of anomalies into different services to be tested for many times, and then test the anomalies respectively. In one implementation mode, in order to avoid interference, only one type of service exception is injected into one service to be tested at a time, the completion of injection immediately starts to determine a solution corresponding to the exception, a next type of service exception is injected into the service to be tested after the solution is successfully determined, if a certain service to be tested is injected and all types of service exceptions are solved, the exception injection process of the next service to be tested can be started until all services to be tested complete exception injection and test is completed. Or, for some services to be tested which have no dependency relationship with each other, parallel testing can be performed.
Further, it has been mentioned before that, services in the system under test may have a dependency relationship therebetween, a service on which the current service depends is referred to as an upstream service of the current service, a service dependent on the current service is referred to as a downstream service of the current service, and the upstream service and/or the downstream service may be referred to as related services of the current service for short. If the current service is abnormal, it may cause the downstream service to be abnormal, or it may cause the upstream service to be abnormal. Therefore, after the exception is injected into the service to be tested, the service to be tested can be tested, the working condition of the related service of the service to be tested can be tested, and the influence of the exception in the service call chain can be comprehensively evaluated. The method comprises the following steps:
after injecting service exception into the service to be tested, firstly determining the related service of the service to be tested according to the dependency relationship among the services (certainly, if the related service of each service is recorded in advance, the related service is directly obtained), then monitoring the operation index of the related service, and testing the interface function of the related service.
An embodiment of the scheme of the present application is described below in conjunction with the DevOps platform. The DevOps (deployment Development and operation Operations combined word) platform can be understood as a tool for installing and deploying application programs, and the DevOps emphasizes how to efficiently organize how teams cooperate and communicate through automated tools to complete the life cycle management of software, so that more stable software can be delivered more quickly and frequently.
In some implementation manners, the system to be tested, the operation and maintenance monitoring system (for monitoring the operation index of the service, see the preamble specifically) and the automatic testing system (for testing the interface function of the service, see the preamble specifically) related in the scheme of the application are all deployed on the DevOps platform for unified management, and meanwhile, the DevOps platform further encapsulates interfaces for external calling, such as service instance acquisition, exception injection and the like, so that operations such as exception injection and testing of the service to be tested become simple, a user does not need to write a complex instruction by himself, and the testing efficiency is improved.
Specifically, when the DevOps platform is adopted, step S200 may be implemented as: and calling a first exception acquisition interface provided by the DevOps platform to acquire the server exception list, and calling a first exception injection interface provided by the DevOps platform to inject the server exceptions in the server exception list into the server. The server exception list includes all server exceptions combed and packaged in advance, and the server exception to be injected may be part or all of the exceptions in the list, depending on the test requirements. If a plurality of server exceptions need to be injected, a sequential injection mode can be adopted to solve one exception and then inject the next exception, as described above.
The first exception injection interface may be a unified interface, and different server exceptions are injected by calling the interface: for example, in the server exception list, each server exception is represented by an exception code, and the first exception injection interface provides a parameter for specifying the exception code during design, so that after the first exception injection interface is called, the internal logic of the interface can determine the type of the current server exception according to the value input by the parameter, and execute the corresponding exception injection behavior. Of course, it is not excluded that in some implementations, for different types of server exceptions, different first exception injection interfaces are provided for exception injection.
Step S210 may then be implemented as: firstly, calling a service instance acquisition interface provided by a DevOps platform to acquire a service list formed by services deployed on the DevOps platform, wherein the service list comprises all the services deployed on the DevOps platform, and the services of a system to be tested are also deployed on the DevOps platform, so that the service to be tested belonging to the system to be tested can be determined from the service list; then, the operation index of the service to be tested can be monitored by using the operation and maintenance monitoring system, and the interface function of the service to be tested can be tested by using the automatic testing system.
Further, when the DevOps platform is adopted, the specific implementation for the case of abnormal test service is as follows:
firstly, a service instance acquisition interface provided by a DevOps platform is called to acquire a service list formed by services deployed on the DevOps platform, and the service to be tested contained in the system to be tested in the service list is determined.
Then, a second exception acquisition interface provided by the DevOps platform is called to acquire a service exception list, the service exception list comprises all service exceptions combed and packaged in advance, the service exception to be injected can be part or all of the exceptions in the list, and the second exception acquisition interface and the first exception acquisition interface can be realized as the same interface or different interfaces depending on test requirements. And then, calling a second exception injection interface provided by the DevOps platform to inject the service exception in the service exception list into the service to be tested, wherein the second exception injection interface and the first exception injection interface can be realized as the same interface or different interfaces. If there are multiple service anomalies to be injected, a sequential injection mode can be adopted to solve one anomaly and then inject the next anomaly, as described above. The second abnormal injection interface can be a uniform interface, different service abnormalities are injected by calling the interface, different second abnormal injection interfaces can be provided for abnormal injection aiming at different types of service abnormalities, and the implementation of the second abnormal injection interface can refer to the implementation of the first abnormal injection interface, and is not repeated.
And finally, monitoring the operation index of the service to be tested by using the operation and maintenance monitoring system, testing the interface function of the service to be tested by using the automatic testing system, monitoring the operation index of the related service of the service to be tested by using the operation and maintenance monitoring system if necessary, and testing the interface function of the related service of the service to be tested by using the automatic testing system.
An embodiment of the present application further provides a test platform, where the test platform is used to test a system to be tested by using the system test method provided in the embodiment of the present application, and fig. 3 shows a schematic diagram of the test platform. Referring to fig. 3, from bottom to top, the lowest layer of the platform is a packaged exception, the left side is a server exception, and the right side is a service exception. The second layer is an integrated DevOps platform interface, and includes the above-mentioned first exception acquiring interface, second exception acquiring interface, first exception injecting interface, second exception injecting interface, service instance acquiring interface, and the like. The third layer is the specific function realization of the platform, the left side is an automatic test system for interface test, the right side is an operation and maintenance monitoring system for operation index monitoring, the middle is three main steps in the method, and the method is realized by calling the interface provided by the DevOps platform in the second layer. The left side of the uppermost layer is a Web page used for a user to directly execute test operation, and the right side of the uppermost layer is an API used for butting a third-party application program so as to integrate a test function into the third-party application program.
Fig. 4 is a functional block diagram of a system testing apparatus 300 according to an embodiment of the present disclosure. Referring to fig. 4, the system test apparatus 300 includes:
an injection module 310, configured to inject a server exception to a server running a system under test;
the test module 320 is configured to monitor an operation index of a service to be tested included in the system to be tested, and test an interface function of the service to be tested.
In one implementation of the system testing apparatus 300, the injection module 310 is further configured to: injecting a service exception into the service to be tested; the test module 320 is further configured to: and monitoring the operation index of the service to be tested, and testing the interface function of the service to be tested.
In one implementation of the system testing apparatus 300, the testing module 320 is further configured to: after injecting service abnormity into the service to be tested, determining related services of the service to be tested, wherein the related services comprise upstream services and/or downstream services of the service to be tested; and monitoring the operation index of the related service, and testing the interface function of the related service.
In one implementation of the system testing apparatus 300, the system to be tested is deployed on a DevOps platform, and an operation and maintenance monitoring system and an automated testing system are also deployed on the DevOps platform; the injection module 310 injects server exceptions to the server running the system under test, including: calling a first exception acquisition interface provided by the DevOps platform to acquire a server exception list, and calling a first exception injection interface provided by the DevOps platform to inject server exceptions in the server exception list into the server; the test module 320 monitors the operation index of the service to be tested included in the system to be tested, and tests the interface function of the service to be tested, including: calling a service instance acquisition interface provided by the DevOps platform to acquire a service list formed by services deployed on the DevOps platform, and determining the service to be tested contained in the system to be tested in the service list; and monitoring the operation index of the service to be tested by using the operation and maintenance monitoring system, and testing the interface function of the service to be tested by using the automatic testing system.
In one implementation of the system testing apparatus 300, the server exception includes at least one of: disk exception, IO exception, memory exception, CPU exception, power off, and network outage.
In one implementation of the system testing apparatus 300, the system to be tested is deployed on a DevOps platform, and an operation and maintenance monitoring system and an automated testing system are also deployed on the DevOps platform; the injection module 310 injects a service exception to the service under test, including: calling a service instance acquisition interface provided by the DevOps platform to acquire a service list formed by services deployed on the DevOps platform, and determining the service to be tested contained in the system to be tested in the service list; calling a second exception acquisition interface provided by the DevOps platform to acquire a service exception list, and calling a second exception injection interface provided by the DevOps platform to inject service exceptions in the service exception list into the service to be tested; the test module 320 monitors the operation index of the service to be tested, and tests the interface function of the service to be tested, including: and monitoring the operation index of the service to be tested by using the operation and maintenance monitoring system, and testing the interface function of the service to be tested by using the automatic testing system.
In one implementation of the system test apparatus 300, the service exception includes at least one of: database access timeouts, array violations, request exceptions, and response exceptions.
Fig. 5 shows a possible structure of an electronic device 400 provided in an embodiment of the present application. Referring to fig. 5, the electronic device 400 includes: a processor 410, a memory 420, and a communication interface 430, which are interconnected and in communication with each other via a communication bus 440 and/or other form of connection mechanism (not shown).
The Memory 420 includes one or more (Only one is shown in the figure), which may be, but not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), and the like. The processor 410, as well as possibly other components, may access, read, and/or write data to the memory 420.
The processor 410 includes one or more (only one shown) which may be an integrated circuit chip having signal processing capabilities. The processor 410 may be a general-purpose processor, and includes a Central Processing Unit (CPU), a Micro Control Unit (MCU), a Network Processor (NP), or other conventional processors; or a special-purpose Processor, including a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, and a discrete hardware component.
One or more computer program instructions may be stored in memory 420 and read and executed by processor 410 to implement the system testing methods provided by the embodiments of the present application, as well as other desired functions.
It will be appreciated that the configuration shown in fig. 5 is merely illustrative and that electronic device 400 may include more or fewer components than shown in fig. 5 or may have a different configuration than shown in fig. 5. The components shown in fig. 5 may be implemented in hardware, software, or a combination thereof. The electronic device 400 may be a physical device, such as a PC, a laptop, a tablet, a server, an embedded device, etc., or may be a virtual device, such as a virtual machine, a virtualized container, etc. The electronic device 400 is not limited to a single device, and may be a combination of a plurality of devices or one or more clusters of a large number of devices.
The embodiment of the present application further provides a computer-readable storage medium, where computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are read and executed by a processor of a computer, the computer-readable storage medium executes the system testing method provided by the embodiment of the present application. The computer-readable storage medium may be implemented as, for example, memory 420 in electronic device 400 in fig. 5.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
In addition, units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
Furthermore, the functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (9)
1. A method for system testing, comprising:
injecting server abnormity into a server running with a system to be tested;
monitoring the operation index of the service to be tested contained in the system to be tested, and testing the interface function of the service to be tested;
the system to be tested is deployed on a DevOps platform, and an operation and maintenance monitoring system and an automatic test system are also deployed on the DevOps platform;
the step of injecting server abnormity into the server running with the system to be tested comprises the following steps:
calling a first exception acquisition interface provided by the DevOps platform to acquire a server exception list, and calling a first exception injection interface provided by the DevOps platform to inject server exceptions in the server exception list into the server;
the monitoring the operation index of the service to be tested contained in the system to be tested and testing the interface function of the service to be tested comprise:
calling a service instance acquisition interface provided by the DevOps platform to acquire a service list formed by services deployed on the DevOps platform, and determining the service to be tested contained in the system to be tested in the service list;
and monitoring the operation index of the service to be tested by using the operation and maintenance monitoring system, and testing the interface function of the service to be tested by using the automatic testing system.
2. The system testing method of claim 1, further comprising:
injecting a service exception into the service to be tested;
and monitoring the operation index of the service to be tested, and testing the interface function of the service to be tested.
3. The method of system testing according to claim 2, wherein after said injecting a service exception to said service under test, said method further comprises:
determining related services of the service to be tested, wherein the related services comprise upstream services and/or downstream services of the service to be tested;
and monitoring the operation index of the related service, and testing the interface function of the related service.
4. The system testing method of any of claims 1-3, wherein the server anomaly comprises at least one of: disk exception, IO exception, memory exception, CPU exception, power off, and network outage.
5. The method for system testing according to claim 2, wherein said injecting a service exception into the service under test comprises:
calling a service instance acquisition interface provided by the DevOps platform to acquire a service list formed by services deployed on the DevOps platform, and determining the service to be tested contained in the system to be tested in the service list;
calling a second exception acquisition interface provided by the DevOps platform to acquire a service exception list, and calling a second exception injection interface provided by the DevOps platform to inject service exceptions in the service exception list into the service to be tested;
the monitoring the operation index of the service to be tested and testing the interface function of the service to be tested comprise:
and monitoring the operation index of the service to be tested by using the operation and maintenance monitoring system, and testing the interface function of the service to be tested by using the automatic testing system.
6. The system testing method according to any one of claims 2, 3 and 5, wherein the service anomaly comprises at least one of: database access timeouts, array violations, request exceptions, and response exceptions.
7. A system test apparatus, comprising:
the injection module is used for injecting server abnormity to a server running the system to be tested;
the test module is used for monitoring the operation index of the service to be tested contained in the system to be tested and testing the interface function of the service to be tested;
the system to be tested is deployed on a DevOps platform, and an operation and maintenance monitoring system and an automatic test system are also deployed on the DevOps platform;
the injection module injects server abnormity to a server running with a system to be tested, and comprises: calling a first exception acquisition interface provided by the DevOps platform to acquire a server exception list, and calling a first exception injection interface provided by the DevOps platform to inject server exceptions in the server exception list into the server;
the test module monitors the operation index of the service to be tested contained in the system to be tested and tests the interface function of the service to be tested, and the test module comprises the following steps: calling a service instance acquisition interface provided by the DevOps platform to acquire a service list formed by services deployed on the DevOps platform, and determining the service to be tested contained in the system to be tested in the service list; and monitoring the operation index of the service to be tested by using the operation and maintenance monitoring system, and testing the interface function of the service to be tested by using the automatic testing system.
8. A computer-readable storage medium having computer program instructions stored thereon, which when read and executed by a processor, perform the method of any one of claims 1-6.
9. An electronic device, comprising: a memory having stored therein computer program instructions which, when read and executed by the processor, perform the method of any of claims 1-6.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010520816.2A CN111881014B (en) | 2020-06-09 | 2020-06-09 | System test method, device, storage medium and electronic equipment |
| PCT/CN2020/119616 WO2021248754A1 (en) | 2020-06-09 | 2020-09-30 | System testing method and apparatus, and storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010520816.2A CN111881014B (en) | 2020-06-09 | 2020-06-09 | System test method, device, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111881014A CN111881014A (en) | 2020-11-03 |
| CN111881014B true CN111881014B (en) | 2022-03-29 |
Family
ID=73156842
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010520816.2A Active CN111881014B (en) | 2020-06-09 | 2020-06-09 | System test method, device, storage medium and electronic equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111881014B (en) |
| WO (1) | WO2021248754A1 (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114328210A (en) * | 2021-12-24 | 2022-04-12 | 中国联合网络通信集团有限公司 | Test method, test device and computer readable storage medium |
| CN114780378A (en) * | 2022-03-15 | 2022-07-22 | 中国人寿保险股份有限公司 | System stability detection traceability method and related equipment based on business interface |
| CN114924974A (en) * | 2022-05-27 | 2022-08-19 | 中国银行股份有限公司 | Test method and device based on abnormal scene |
| CN114706733B (en) * | 2022-05-30 | 2022-09-20 | 支付宝(杭州)信息技术有限公司 | Section program abnormity monitoring method and device |
| CN115062966A (en) * | 2022-06-14 | 2022-09-16 | 度小满科技(北京)有限公司 | Method, device, equipment and storage medium for determining deployment check items |
| CN114924990B (en) * | 2022-06-21 | 2025-09-23 | 中国农业银行股份有限公司 | Abnormal scene testing method and electronic equipment |
| CN115080436B (en) * | 2022-06-28 | 2023-09-22 | 中电金信软件有限公司 | Test index determining method and device, electronic equipment and storage medium |
| CN115037653B (en) * | 2022-06-28 | 2023-10-13 | 北京奇艺世纪科技有限公司 | Service flow monitoring method, device, electronic equipment and storage medium |
| CN116541312B (en) * | 2023-07-06 | 2023-09-22 | 广汽埃安新能源汽车股份有限公司 | Continuous integration test method and system for automobile software |
| CN118277280B (en) * | 2024-05-30 | 2024-09-06 | 江苏曼荼罗软件股份有限公司 | Service interface self-test method, system, readable storage medium and equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105808420A (en) * | 2014-12-31 | 2016-07-27 | 阿里巴巴集团控股有限公司 | Implementation method and device of robustness testing process |
| CN107710683A (en) * | 2015-06-22 | 2018-02-16 | 微软技术许可有限责任公司 | Elasticity services |
| CN110262972A (en) * | 2019-06-17 | 2019-09-20 | 中国科学院软件研究所 | A kind of failure testing tool and method towards micro services application |
| CN110673993A (en) * | 2019-09-19 | 2020-01-10 | 聚好看科技股份有限公司 | Fault injection method, platform and system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9652365B2 (en) * | 2010-08-24 | 2017-05-16 | Red Hat, Inc. | Fault configuration using a registered list of controllers |
| GB201605360D0 (en) * | 2016-03-30 | 2016-05-11 | Microsoft Technology Licensing Llc | Local chat service simulator for bot development |
| CN106708700B (en) * | 2016-12-13 | 2019-03-19 | 广东金赋科技股份有限公司 | A kind of O&M monitoring method and device applied to server-side |
| CN108268365B (en) * | 2016-12-30 | 2020-12-18 | 腾讯科技（深圳）有限公司 | Abnormal task injection method, device and system |
| CN109766370A (en) * | 2018-12-27 | 2019-05-17 | 口碑(上海)信息技术有限公司 | Data processing method, data service system and equipment |
| CN111104336B (en) * | 2019-12-30 | 2023-05-23 | 武汉烽火信息集成技术有限公司 | Online service interface testing method and device based on container and VNC |
-
2020
- 2020-06-09 CN CN202010520816.2A patent/CN111881014B/en active Active
- 2020-09-30 WO PCT/CN2020/119616 patent/WO2021248754A1/en not_active Ceased
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105808420A (en) * | 2014-12-31 | 2016-07-27 | 阿里巴巴集团控股有限公司 | Implementation method and device of robustness testing process |
| CN107710683A (en) * | 2015-06-22 | 2018-02-16 | 微软技术许可有限责任公司 | Elasticity services |
| CN110262972A (en) * | 2019-06-17 | 2019-09-20 | 中国科学院软件研究所 | A kind of failure testing tool and method towards micro services application |
| CN110673993A (en) * | 2019-09-19 | 2020-01-10 | 聚好看科技股份有限公司 | Fault injection method, platform and system |
Non-Patent Citations (1)
| Title |
|---|
| "从代码的视角深入浅出理解DevOps";大码哥;《https://www.cnblogs.com/xichji/p/11864667.html》;20191115;第1-9页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021248754A1 (en) | 2021-12-16 |
| CN111881014A (en) | 2020-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111881014B (en) | System test method, device, storage medium and electronic equipment | |
| CN113946499B (en) | A microservice link tracking and performance analysis method, system, device and application | |
| CN112527484A (en) | Workflow breakpoint continuous running method and device, computer equipment and readable storage medium | |
| CN109947585A (en) | The processing method and processing device of PCIE device failure | |
| CN110990289B (en) | Method and device for automatically submitting bug, electronic equipment and storage medium | |
| CN113238930A (en) | Software system testing method and device, terminal equipment and storage medium | |
| CN117056081B (en) | Verification method and device for system load balancing capability based on chaotic engineering | |
| CN112596750B (en) | Application testing method and device, electronic equipment and computer readable storage medium | |
| CN111767218B (en) | Automatic test method, equipment and storage medium for continuous integration | |
| CN112363897A (en) | Full link log tracking method, device and storage medium | |
| CN111124724A (en) | A node fault testing method and device for a distributed block storage system | |
| CN119814529B (en) | Fault alarm method, device, computer equipment and storage medium | |
| CN111966599B (en) | Method, system, terminal and storage medium for reliability testing of virtualization platform | |
| CN114201393A (en) | Processing methods, apparatus, equipment, media and program products for software testing | |
| CN116931965B (en) | Integrated stream processing method, device, electronic equipment and storage medium | |
| Chen et al. | Big data system testing method based on chaos engineering | |
| CN114090382B (en) | Health inspection method and device for super-converged cluster | |
| CN119578687A (en) | Emergency plan drill platform and method, device, equipment and storage medium | |
| CN116820988A (en) | High availability test method, device, equipment and medium for distributed middleware system | |
| CN116230067A (en) | Automatic testing method, system, equipment and medium for solid state disk | |
| CN115759518A (en) | Usability treatment system based on chaos engineering | |
| CN115391110A (en) | Test method of storage device, terminal device and computer readable storage medium | |
| CN116089283A (en) | Monitoring and testing method, system, device and readable medium for simulating quasi-production environment | |
| CN120686769A (en) | Fault scenario construction method and device | |
| CN118193363A (en) | Quick fault positioning method and system based on sonic system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |