CN116185810A - Automated testing with improved scalability and compatibility - Google Patents

Abstract

The present disclosure presents methods for performing automated testing. A test request for performing an automated test with a specified set of test equipment may be received by a registry. A test task corresponding to the test request may be generated by the registry. The test tasks may be scheduled by the registry to test agents associated with the specified test equipment set. The automated test may be performed by the test agent using the specified set of test equipment. The present disclosure also proposes an automated testing system. The automated test system may include: a registry, at least one test agent, and at least one set of test devices.

Description

Automated testing with improved scalability and compatibility

Background technique

In the development process of software applications, testing, as an extremely important part of ensuring application quality, plays a very critical role. Herein, the software application against which testing is performed may be referred to as a target application. Usually, when testing the target application, after the test case is determined, the tester can execute the test step by step according to the procedure described in the test case, and compare the actual test result with the expected test result to verify the target Whether the functions of the application are correct. During this process, in order to save manpower, time or hardware resources and improve test efficiency, automated testing is introduced. Automated testing can be the process of turning human-driven testing into machine-executed testing. In automated testing, specific software or programs can be used to control the execution of tests and the comparison between actual test results and expected test results. Through automated testing, some repetitive but necessary testing tasks that exist in the testing process can be automated, or some testing tasks that are difficult to perform manually can be performed.

Contents of the invention

This Summary is provided to introduce a set of concepts that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

Embodiments of the present disclosure propose methods for performing automated testing. A test request for performing automated testing with a specified set of test equipment may be received through the registry. A test task corresponding to the test request may be generated through the registration center. The test task may be dispatched to a test agent associated with the specified set of test equipment through the registration center. The automated testing may be performed by the testing agent using the specified set of testing equipment.

Embodiments of the present disclosure also propose an automated testing system. The automated test system may include: a registration center configured to receive a test request for performing an automated test for a target application using a specified set of test equipment, generate a test task corresponding to the test request, and upload the Test tasks are dispatched to test agents associated with the specified set of test equipment; at least one test agent, each test agent configured to receive test tasks and perform automation using the specified set of test equipment in the received test tasks testing; and at least one set of test equipment, each set of test equipment associated with one of the at least one test agent and configured to run the target application.

It should be noted that one or more of the above aspects include the features specified in the following detailed description as well as in the claims. Certain illustrative features of the one or more aspects are set forth in detail in the following description and accompanying drawings. These features are merely indicative of the various ways in which the principles of various aspects can be implemented and this disclosure is intended to include all such aspects and their equivalents.

Description of drawings

The disclosed aspects will be described below with reference to the accompanying drawings, which are provided to illustrate but not limit the disclosed aspects.

FIG. 1 illustrates an exemplary process for performing automated testing according to an embodiment of the present disclosure.

FIG. 2 illustrates an exemplary process for creating a test agent according to an embodiment of the disclosure.

FIG. 3 illustrates an exemplary process for obtaining an authorization code for a set of test devices according to an embodiment of the present disclosure.

FIG. 4 illustrates an exemplary video navigation interface according to an embodiment of the disclosure.

FIG. 5 shows an exemplary architecture of an automated testing system according to an embodiment of the present disclosure.

FIG. 6 illustrates an exemplary set of test equipment according to an embodiment of the disclosure.

FIG. 7 is a flowchart of an exemplary method for performing automated testing according to an embodiment of the disclosure.

FIG. 8 illustrates an exemplary automated testing system according to an embodiment of the disclosure.

FIG. 9 shows an exemplary apparatus for performing automated testing according to an embodiment of the present disclosure.

Detailed ways

The present disclosure will now be discussed with reference to several exemplary embodiments. It should be understood that the discussion of these embodiments is only for enabling those skilled in the art to better understand and thus implement the embodiments of the present disclosure, rather than teaching any limitation to the scope of the present disclosure.

Currently, a single machine is typically utilized to execute automated tests and present test results. Taking automated testing for mobile applications as an example, the automated testing usually runs on a personal computer connected to the mobile device, and the personal computer executes the automated testing and presents the test results. Personal computers may go down due to malfunctions. Once down, automated testing will be interrupted and existing test results will be lost. Therefore, it may be difficult to provide reliable automated testing services in this way. Furthermore, since mobile devices are usually wired to personal computers, the number of mobile devices that can be connected to personal computers is limited. This results in a limited number of automated tests that can be executed simultaneously.

Embodiments of the present disclosure propose improved automated testing services. Automated testing services can be decoupled to registries and testing agents. The registry can manage test agents, schedule test tasks, visualize test results, and more. The test agent can perform test tasks, send test results to the registration center, and so on. Registry and test agent can be deployed in different places. Multiple test agents can be deployed at the same time. Even if one or some of the test agents fail, automated tests can be executed by other test agents and the test results can be preserved. Registries can also be deployed on multiple nodes to avoid single node failures. In this way, the reliability of automated testing services can be significantly improved.

In one aspect, embodiments of the present disclosure propose utilizing a terminal device to create a test agent. An end device may be any computing device such as a desktop computer, laptop computer, tablet computer, cellular phone, wearable device, etc. located in any geographic location. An end device can be configured as a test agent by running a test agent creation program on the end device and registering with the registry. A desired set of test devices on which a target application is installed can be connected to a test agent for automated testing of the target application. Herein, a set of test devices may refer to a collection of test devices connected to a test agent as a whole. The form of the test equipment set may include, for example, a single test equipment, a test equipment pair composed of two test equipment, a test equipment group composed of more than two test equipment, and so on. Herein, a test device may refer to a computing device, such as a desktop computer, laptop computer, tablet computer, cell phone, wearable device, etc., on which a target application is run and utilized to perform automated testing for the target application. Since test agents can be created by terminal devices located in any geographic location, and automated testing can be performed by connecting the desired test device set to the test agent, the number of supported test device sets can be greatly expanded, thereby improving Scalability of automated testing services. In addition, this approach makes it easy to utilize test equipment sets located in any geographic location for automated testing. For example, a user may be located at site A, while the desired set of test equipment is located at site B. A test agent can be created at site B and the desired set of test devices connected to the test agent. A user at location A can request to use the test equipment set at location B to perform automated testing. Furthermore, in cases where the number of test fixture sets currently available is insufficient, the test fixture sets can be supplemented by creating test agents and connecting suitable test fixture sets to the test agents, or by requesting the use of test fixture sets on other test agents. Complementary set of test equipment. In this way, the time for scheduling test tasks and waiting for test execution can be reduced.

In another aspect, the embodiments of the present disclosure propose to enable the automated testing service to be compatible with various Types of automated tests. Types of compatible automated tests may include, for example, tests for single operating system applications, tests for cross-operating system applications, tests for web applications, and so on. Herein, a single operating system application may refer to an application involving only a single operating system, such as an Android application, an iOS application, a Windows application, a macOS application, and the like. A cross-operating system application may refer to an application involving the interaction between multiple operating systems, such as an application involving the interaction between the Android operating system and the Windows operating system, an application involving the interaction between the iOS operating system and the macOS operating system, etc. . A web application may refer to a web-based application, such as a website accessed through a browser, a plug-in run through a browser, and the like. For example, the test for cross-operating system applications can be implemented by enabling the test agent to support a test framework such as Appium, and connecting a pair of test devices installed with different operating systems to the test agent. In this way, the application scenarios of automated testing can be made wider, thereby significantly improving the compatibility of automated testing services.

In another aspect, the embodiments of the present disclosure propose unified management of test resources so as to be shared among various test agents. Test resources may include, for example, an application package of a target application, a test suite including test cases for automated tests to be performed, a set of test equipment, and the like. Application packages and test suites can be stored in data storage devices deployed in the cloud and managed by the registry. Sets of test devices can also be managed through a registry. In this way, different users or teams, such as users or teams located in different geographical locations, can share test resources, thereby improving the reusability of test resources and saving resource costs.

In another aspect, the embodiment of the present disclosure proposes to manage the use authority of the test equipment set through the authorization code. Herein, the authorization code may refer to an encrypted code associated with a specific test equipment set and used to authorize the sender of the test request to use the specific test equipment set to perform automated testing. Unlike the identifiers of test sets that are publicly visible, authorization codes for test sets are restricted access. The authorization code of the test equipment set can be provided to the test agent by the registration center when creating the test agent associated with the test equipment set, and further provided to the designated user of the test agent, such as the creator and manager of the test agent. Additionally, the authorization code for the set of test equipment may be provided by the registration center to the general user upon authorization by the designated user of the test agent associated with the set of test equipment. When the test agent receives from the registration center a test task for using a specified test equipment set to perform automated testing, the test agent can determine whether the test task it receives includes the authorization code of the specified test equipment set, and when the test task includes Execute automated tests without authorization codes. In this way, arbitrary use of the test equipment set can be avoided, thereby improving the security of the automated testing service.

In another aspect, the embodiments of the present disclosure propose to visualize the test results of the automated test by presenting a video navigation interface. The video navigation interface may include a navigation area for displaying a plurality of test cases included in the automated test. Each test case can be selected. The video navigation interface may also include a video area for displaying video clips corresponding to the selected test cases. In this way, it is possible for the user to conveniently view the status of the test equipment during the execution of each use case of the automated test. Furthermore, video clips of failed test cases can be combined with corresponding test logs, device logs, etc., so as to more quickly and accurately locate the loopholes in the target application.

FIG. 1 illustrates an exemplary process 100 for performing automated testing according to an embodiment of the disclosure. In process 100, automated testing services can be decoupled to registries and testing agents. The registry can manage test agents, schedule test tasks, visualize test results, and more. The test agent can perform test tasks, send test results to the registration center, and so on.

At 102, a test request for performing automated testing with a specified set of test equipment can be received through a registry. A test request may include, for example, an application package of a target application, a test suite including a set of test cases for an automated test to be performed, an identifier specifying a set of test equipment, recipients of test reports, and the like. For example, when a user initiates a test request, the test equipment set on which the automated test is to be executed may be selected as the designated test equipment set. In addition, in the case that the user knows the authorization code specifying the test equipment set, the test request may also include the authorization code. The specified test equipment set can have various forms, such as a single test equipment, a test equipment pair consisting of two test equipment, a test equipment group consisting of more than two test equipment, and so on. An exemplary form of the test equipment set will be described later with reference to FIG. 6 . The test device may be any type of computing device, such as a desktop computer, laptop computer, tablet computer, cellular phone, wearable device, and the like. Preferably, after the registration center receives the test request, it can present the information of the specified test equipment set in the test request to the user via the front end connected to it, such as the identifier of the specified test equipment set, the supported test type, The supported test framework specifies the current status, device type, and device model of the test equipment included in the test equipment set.

At 104, a test task corresponding to the test request may be generated through the registration center. For example, the registry can identify an application package, test suite, identifier for a specified test set, authorization code for a specified test set, etc. from a test request, and based on the identified application package, test suite, specified test set The identifier of the test device, the authorization code specifying the set of test equipment, etc. to generate test tasks.

At 106, the test task can be dispatched to the test agent associated with the specified set of test equipment through the registration center. For example, a test task may include an identifier specifying a set of test equipment. The registry can identify test agents associated with the specified set of test equipment, and schedule test tasks to the identified test agents.

Automated testing can be performed by a test agent using a specified set of test equipment. Preferably, the test agent may first determine whether the test task it has received includes an authorization code for the set of test equipment specified in the test task, and execute the automated test in response to determining that the test task includes the authorization code. For example, at 108, it may be determined, by the testing agent, whether the testing task includes an authorization code specifying a set of testing equipment. Authorization codes can be from test requests.

If it is determined at 108 that the test task does not include an authorization code specifying a set of test devices, process 100 may proceed to step 110 . At 110, automated testing is performed without the specified set of test equipment.

If it is determined at 108 that the test task includes an authorization code specifying a set of test devices, process 100 may proceed to step 112 . At 112, automated testing can be performed by a testing agent using the specified set of testing equipment.

According to the embodiments of the present disclosure, during the execution of the automated test, the state of each test device in the test device set can be monitored in real time, so as to obtain performance data of the tested target application. For example, the central processing unit (Central Processing Unit, CPU) utilization rate, memory utilization rate, network connection status, etc. of the testing device may be monitored in real time. The battery usage of the test device can also be monitored in real time, so that the user can know the power consumption of the tested target application. In addition, the delay of application clicking can also be monitored in real time, so that the user can know the running speed of the target application. In addition, a video corresponding to the automated test may be recorded by capturing the screen and/or sound of the testing device during execution of the automated test in real time. The recorded video may include a plurality of video clips corresponding to the plurality of test cases included in the automated test.

At step 114, the test result of the automated test may be obtained through the test agent, and the test result may be sent to the registration center. The test results of the automated test may include, for example, pass/fail information corresponding to each test case, running time, test logs, device logs, custom log screenshots, performance data, videos, and the like. The above test results can fully reflect the situation of the tested target application. A detailed test report for the automated test can be generated based on the above-mentioned comprehensive test results. The detailed test report can help the developer of the target application quickly and accurately locate the vulnerabilities in the target application.

At 116, the test results can be visualized through the registry. For example, test results can be presented through a dashboard. Dashboards may be displayed, for example, via a front end connected to a registry. Preferably, when visualizing test results, a video navigation interface can be presented. The video navigation interface may include a navigation area for displaying a plurality of test cases included in the automated test. Each test case can be selected. The video navigation interface may also include a video area for displaying video clips corresponding to the selected test cases. An exemplary video navigation interface will be described later with reference to FIG. 4 .

It should be understood that the process for performing automated testing described above in conjunction with FIG. 1 is only exemplary. According to actual application requirements, the steps in the process for executing the automated test may be replaced or modified in any manner, and the process may include more or fewer steps. For example, after receiving the test task, the test agent can directly use the specified test equipment set in the test task to execute the automated test without determining whether the test task includes the authorization code of the specified test equipment set, for example, when the test agent knows that the test task is from its When specifying a user. Additionally, at 116, in addition to visualizing the test results, the status of the test devices in the test device set can also be presented. Furthermore, the specific order or hierarchy of steps in process 100 is exemplary only, and the process for performing automated testing may be performed in an order different than that described.

According to an embodiment of the present disclosure, a test agent may be created by a terminal device. An end device may be any computing device such as a desktop computer, laptop computer, tablet computer, cellular phone, wearable device, etc. located in any geographic location. For example, a user can create a test agent using a computing device that they have access to. FIG. 2 illustrates an exemplary process 200 for creating a test agent according to an embodiment of the disclosure.

At 202, a test agent creation program can be run on the terminal device. The test agent creation program may be a predetermined computer program that can be used to create test agents.

At 204, a registration process may be initiated to the registration center at the terminal device. For example, a registration request may be sent to the registration center through the terminal device. The registration center may register the terminal device in response to receiving the registration request.

At 206, the end device may be configured as a test agent in response to completion of the registration process.

After the test agent is registered, the authorization code of the test equipment set associated with the test agent can be further obtained.

At 208, an authorization code generation request for generating an authorization code for a set of test devices associated with the test agent can be received, via the registration center. For example, a test agent may send an authorization code generation request to the registry for generating authorization codes for a set of test devices associated therewith. The authorization code generation request may include the information of the test equipment set, such as the identifier of the test equipment set, the supported test type, the supported test framework, the current state of the test equipment included in the test equipment set, the equipment type, and the equipment model wait.

At 210, an authorization code may be generated through the registration center. For example, the registration center may generate the authorization codes for the set of test devices based on the information of the set of test devices included in the authorization code generation request received from the test agent.

At 212, the generated authorization code can be sent to the testing agent through the registration center.

Since test agents can be created by terminal devices located in any geographic location, and automated testing can be performed by connecting the desired test device set to the test agent, the number of supported test device sets can be greatly expanded, thereby improving Scalability of automated testing services. Alternatively, the user may be located at site A, while the desired set of test equipment is located at site B. A test agent can be created at site B and the desired set of test devices connected to the test agent. A user at location A can request to use the test equipment set at location B to perform automated testing. This approach makes it easy to use test equipment sets located in any geographic location for automated testing. Furthermore, in cases where the currently available test set is insufficient, the test set can be supplemented by creating a test agent and connecting a suitable test set to the test agent. Additionally or alternatively, test equipment may be supplemented by requesting use of test equipment sets on other test agents. In this way, the time for scheduling test tasks and waiting for test execution can be reduced.

It should be understood that the process for creating a test agent described above in conjunction with FIG. 2 is exemplary only. According to actual application requirements, the steps in the process for creating the test agent can be replaced or modified in any manner, and the process can include more or fewer steps. Furthermore, the specific order or hierarchy of steps in process 200 is exemplary only, and the process for creating test agents may be performed in an order different than that described.

As described above, when initiating a test request to the registration center for using the specified test equipment set to perform automated testing, the authorization code of the specified test equipment set can be provided. When the test agent receives the test task corresponding to the test request from the registration center, the test agent can determine whether the test task it receives includes the authorization code of the specified test equipment set, and if the test task includes the authorization code Execute automated tests. For a designated user of a testing agent associated with a designated set of testing equipment, such as a creator, manager, etc. of the testing agent, the authorization code may be obtained from the testing agent. The authorization code at the testing agent may be obtained from the registry when the testing agent is created, eg, through process 200 of FIG. 2 . For ordinary users, for example, for designated users who are not test agents, they can request the authorization code of the designated test device set from the test agent associated with the designated test device set via the registration center. The registration center may provide the authorization code to the general user with the permission of the designated user of the testing agent associated with the set of testing devices. In this way, arbitrary use of the test equipment set can be avoided, thereby improving the security of the automated testing service. FIG. 3 illustrates an exemplary process 300 for obtaining an authorization code for a set of test devices according to an embodiment of the disclosure.

At 302, an authorization code acquisition request for acquiring an authorization code of a specified test equipment set may be received through the registration center. The authorization code retrieval request may be received, for example, from a computing device associated with a user who wants to perform automated testing with a specified set of testing devices.

At 304, the authorization code acquisition request can be forwarded to the test agent associated with the specified set of test equipment through the registration center.

Subsequently, whether to provide an authorization code can be determined by the registry based on the response received from the test agent.

At 306, a response to the authorization code retrieval request can be received from the testing agent via the registry. For example, after receiving the authorization code acquisition request forwarded by the registration center, the test agent can determine whether to provide the authorization code to the sender of the authorization code acquisition request, and include the determination result in the authorization code acquisition request sent to the registration center. in response to the request.

At 308, it may be determined, by the registry, whether the response received from the testing agent indicates that the testing agent grants the authorization code. If at 308 it is determined that the response received from the testing agent indicates that the testing agent grants the authorization code, then process 300 may proceed to step 310 . At 310, the authorization code can be provided by the registration center. If at 308 it is determined that the response received from the testing agent does not indicate that the testing agent grants the authorization code, then process 300 may proceed to step 312 . At 312, no authorization code is provided.

It should be understood that the process for obtaining the authorization code of the test device set described above in conjunction with FIG. 3 is only exemplary. According to actual application requirements, the steps in the process of obtaining the authorization code of the test equipment set may be replaced or modified in any manner, and the process may include more or fewer steps. For example, the test agent may consider whether the sender of the authorization code acquisition request has ever created other test agents when determining whether to grant the authorization code to the sender of the authorization code acquisition request. The sender who has created other test agents can preferentially use the test equipment set of this test agent. Furthermore, the specific order or hierarchy of steps in process 300 is exemplary only, and the process for obtaining an authorization code for a set of test devices may be performed in an order different from that described.

FIG. 4 illustrates an exemplary video navigation interface 400 according to an embodiment of the disclosure.

The video navigation interface 400 may include a navigation area 410 for displaying a plurality of test cases included in the automated test. In the navigation area 410, the names and corresponding times of the respective test cases are shown. Optionally, in the navigation area 410, initialization events, test run start events and corresponding times are also shown. Preferably, failed test cases can be flagged. Failed test cases can be marked in various ways. For example, failed test cases can be displayed with a different color than successful test cases, failed test cases can be underlined, failed test cases can be highlighted, and so on. For example, the test case "5.NotesCardTest.createTextNote" is highlighted in the navigation area 410, which indicates that the test case failed.

Each test case can be selected by clicking on it. The video navigation interface 400 may also include a video area 420 for displaying video clips corresponding to the selected test cases. After selecting a test case from the navigation area 410, the video corresponding to the automated test can jump to a point in time corresponding to the selected test case, so that the video corresponding to the selected test case can be displayed in the video area 420. video clip. In this way, it is possible for the user to conveniently view the status of the test equipment during the execution of each use case of the automated test. Furthermore, video clips of failed test cases can be combined with corresponding test logs, device logs, etc., so as to more quickly and accurately locate the loopholes in the target application.

Optionally, the video navigation interface 400 may also include a button 430 for downloading the video and a set of buttons 440 for setting the playback speed.

It should be understood that the video navigation interface 400 shown in FIG. 4 is just an example of a video navigation interface. According to actual application requirements, the video navigation interface may have any other structure, and may include more or less elements. For example, in the video navigation interface 400, in addition to displaying buttons for downloading videos and setting playback speed, buttons such as setting video resolution may also be displayed.

FIG. 5 shows an exemplary architecture 500 of an automated testing system according to an embodiment of the disclosure. The architecture 500 can provide automated testing services for target applications, for example, by performing the processes described above in conjunction with FIGS. 1 to 4 . The architecture 500 may include a registration center 510 and at least one test agent, such as a test agent 520-1 to a test agent 520-K (K≧1). In addition, the architecture 500 may also include at least one test equipment set. Each set of test equipment may be associated with one of the at least one test agents. For example, test equipment set 540-1 through test equipment set 540-M (M≥1) may be associated with test agent 520-1, and test equipment set 542-1 through test equipment set 542-N (N≥1) may Associated with Testing Agent 520-K.

The registry 510 can manage test agents, schedule test tasks, visualize test results, and so on. The registration center 510 can be deployed in the cloud. It should be understood that although only one registry 510 is shown in architecture 500, in some embodiments, the registry can also be extended. For example, an automated testing system may include more than one registry. These registries can be managed as clusters of registries with a unified endpoint. Each registry in the registry cluster can be deployed on multiple nodes in a distributed manner, thereby avoiding single-node failures.

Registration center 510 may be connected to front end 550 . Front end 550 may interface with a user and present a user interface associated with registry 510 to the user. In addition, the registration center 510 may be connected to a data storage device 560 . The data storage device 560 can be deployed in the cloud. Data storage device 560 may store test resources, such as application package 562, test suite 564, and the like. The application package 562 may include application programs for installing and running target applications. Test suite 564 may include a set of test cases for automated testing to be performed. The registration center 510 can manage the test resources stored in the data storage device 560 .

The registration center 510 may include a rights management unit 512 . The authority management unit 512 can manage the authority of the test agent. For example, the rights management unit 512 may determine whether to register the terminal device to configure the terminal device as a test agent when receiving a registration request from the terminal device. The registration request may be triggered by a test agent creation program running on the terminal device. In addition, the authority management unit 512 can manage the user's authority to determine the test equipment set that the user can use.

The registration center 510 may include a proxy and device set management unit 514 . The agent and device set management unit 514 can be used to manage the test agents registered to the registration center 510 and the test device sets associated with the test agents. The status of the test agents and/or test devices in the set of test devices may be presented to the user via the front end 550 .

The registry 510 may include a test task scheduling unit 516 . The test task scheduling unit 516 can generate a test task corresponding to the test request, and schedule the test task to a corresponding test agent. For example, a test request might specify a set of test equipment with which to execute automated tests. The test task scheduling unit 516 can schedule the test task corresponding to the test request to the test agent associated with the test equipment set specified in the test request.

Registry 510 may include a test result visualization unit 518 . The test result visualization unit 518 can visualize the test results of the automated test. The test results of the automated test may include, for example, pass/fail information corresponding to each test case, running time, test logs, device logs, custom log screenshots, performance data, and the like. Preferably, when visualizing the test results, a video navigation interface may be presented, such as the video navigation interface 400 described above in conjunction with FIG. 4 .

Each of the test agents 520 - k (1≤k≤K) among the test agents 520 - 1 to 520 -K can register with the registration center 510 . The registration center 510 and the test agent 520-k can access each other through, for example, Remote Procedure Call (Remote Procedure Call). The test agent 520-k can be created by any computing device located in any geographic location. For example, a test agent may be created through the process 200 described above in connection with FIG. 2 . The test agent 520-k can perform test tasks, send test results to the registration center 510, and so on.

The testing agent 520 - k may include a registration unit 522 - k for initiating a registration process with the registration center 510 .

The testing agent 520-k may include a security unit 524-k for determining whether to execute the testing task scheduled by the registration center 510. For example, security unit 524-k may analyze test tasks received from registry 510, determine whether the received test tasks include an authorization code for a corresponding specified set of test devices, and if determined that the received test tasks include an authorization code, The testing agent 520-k, such as the test execution unit 530-k in the testing agent 520-k, is notified to execute the automated test.

A test agent 520-k may include a device set management unit 526-k for locally managing one or more test device sets associated therewith.

A test agent 520-k may include a set control tool 528-k for controlling and debugging one or more test sets associated therewith. A testing agent 520-k is typically associated with one type of set of testing equipment. A device set control tool 528-k may, for example, correspond to a type of test device set associated with a test agent 520-k. As an example, when the test device set associated with the test agent 520-k is an Android device, the device set control tool 528-k may be a software development kit (SDK) for Android devices. As another example, when the test device set associated with the test agent 520-k is an iOS device, the device set control tool 528-k may be an SDK for iOS devices.

The test agent 520-k may include a test execution unit 530-k, configured to execute automated tests using the set of test equipment, test suites, etc. specified in the test task.

The test agent 520-k may include a test result processing unit 532-k, configured to obtain the test result of the automated test, and send the test result to the registration center 510.

The test agent 520-k can support various types of test frameworks, such as Appium, Espresso, Junit and so on. Various forms of test equipment sets can be associated with the test agent 520-k to make the automated test service compatible with various types of automated tests. Types of compatible automated tests may include, for example, tests for single operating system applications, tests for cross-operating system applications, tests for web applications, and so on. A test agent 520-k may be associated with one or more sets of test devices. For example, test agent 520-1 may be associated with test equipment set 540-1 through test equipment set 540-M, and test agent 520-K may be associated with test equipment set 542-1 through test equipment set 542-N. Each set of test devices can be configured to run a target application.

The test equipment set can have various forms, such as a single test equipment, a test equipment pair composed of two test equipment, a test equipment group composed of more than two test equipment, and so on. FIG. 6 illustrates exemplary sets of test equipment 600, 620, and 640 according to an embodiment of the disclosure. Each of test equipment sets 600, 620, and 640 may correspond to test equipment sets 540-1 through 540-M and test equipment sets 542-1 through 542-N in FIG. any set of test equipment.

Set of test equipment 600 may include a single test equipment 610 . This single test device 610 can be used independently to perform automated testing. Target application 612 can be run on test device 610 . Target application 612 may be, for example, a single operating system application, a web application, or the like.

Test device set 620 may include two test devices, such as test device 630-1 and test device 630-2. The test device 630-1 and the test device 630-2 may be bound to each other to form a test device pair. The test equipment 630-1 and the test equipment 630-2 can be used to cooperate to execute automated testing. Target application 632 may run on test device 630-1 and test device 630-2. Target application 632 may be, for example, a single operating system application, a web application, a cross-operating system application, and the like. In the case that the target application 632 running on the test device 630-1 and the test device 630-2 is a cross-operating system application, the target application 632 and the corresponding The corresponding version of the operating system. As an example, target application 632 may be an application that involves interaction between the Android operating system and the Windows operating system. In this case, the testing device 630-1 may be an Android mobile phone installed with an Android operating system, and the testing device 630-2 may be a Windows computer installed with a Windows operating system. An Android version of target application 632 may be run on test device 630-1 and a Windows version of target application 632 may be run on test device 630-2. The test agents associated with the set of test devices 620 may be test agents that support a testing framework such as Appium. When performing automated testing, operations may be performed on both the testing device 630-1 and the testing device 630-2, respectively. Interoperation between the test equipment 630-1 and the test equipment 630-2 is also possible.

The test equipment set 640 may include more than two test equipment, for example, the test equipment 650-1, the test equipment 650-2, . . . , the test equipment 650-T (T>2). The test equipment 650-1, the test equipment 650-2, . . . , the test equipment 650-T may be bound to each other to form a test equipment group or a test equipment cluster. These test devices can be used to coordinate the execution of automated tests. Target application 652 may run on test device 650-1, test device 650-2, ..., test device 650-T. Target application 652 may be, for example, a single-operating system application, a web application, a cross-operating system application, and the like. As an example, target application 652 may be, for example, an application that requires more than two test devices to participate in a server distribution operation.

By enabling the test agent to support various types of test frameworks and associating various forms of test equipment sets with the test agent, the application scenarios of automated testing can be extended, thereby significantly improving the compatibility of automated testing services.

Referring back to FIG. 5 , according to an embodiment of the present disclosure, a test agent may have various implementations with respect to a set of test equipment.

In one embodiment, the test agent may be implemented independently of the set of test equipment. For example, test agent 520-1 may be implemented independently of test equipment set 540-1 through test equipment set 540-M, and test agent 520-K may be implemented independently of test equipment set 542-1 through test equipment set 542-N. And implemented, and so on. Figure 5 illustrates the manner in which the test agent is implemented independently of the set of test devices. In such an embodiment, a set of test devices may be wired to its associated test agents. The number of sets of test devices connected to a test agent may depend on the number of ports the test agent can provide.

In another implementation, a test agent may be implemented in a set of test devices. For example, a test agent can be implemented as a test device in a set of test devices. The test agent needs to obtain the resources of other applications when performing automated tests, so cross-application interaction is required. If a test device can interact across applications, then the test device itself can become a test agent. In such an embodiment, the test agent and the test device may be the same device when the set of test devices only includes a single test device. When the set of test equipment includes more than one test equipment, the test agent may be one of the more than one test equipment.

According to the embodiment of the present disclosure, in the architecture 500, the test resources can be managed uniformly so as to be shared among various test agents. Test resources may include, for example, application package 562, test suite 564, test set 540-1 through test set 540-M, test set 542-1 through test set 542-N, and the like. The application package 562 and the test suite 564 can be stored in the data storage device 560 deployed in the cloud and managed by the registration center 510 . The test equipment set 540 - 1 to the test equipment set 540 -M and the test equipment set 542 - 1 to the test equipment set 542 -N can also be managed by the registration center 510 . In this way, different users or teams, such as users or teams located in different geographical locations, can share test resources, thereby improving the reusability of test resources and saving resource costs.

It should be understood that the architecture 500 shown in FIG. 5 is only an example of the architecture of the automated testing system. According to actual application requirements, the automated test system may have any other structure, and may include more or less components. For example, a new test agent can be added to the existing automated test system through the process 200 in FIG. 2 to enhance the scalability of the automated test system. In addition, the registration center 510 can serve as a test agent and then register with other registration centers, thereby further enhancing the scalability of the automated testing system.

FIG. 7 is a flowchart of an exemplary method 700 for performing automated testing according to an embodiment of the disclosure.

At 710, a test request to perform automated testing with a specified set of test equipment can be received via a registry.

At 720, a test task corresponding to the test request may be generated through the registration center.

At 730, the test task can be dispatched to a test agent associated with the specified set of test equipment through the registration center.

At 740, the automated testing can be performed by the testing agent using the specified set of testing equipment.

In an implementation manner, the automated testing may include at least one of the following: testing for a single operating system application, testing for a cross-operating system application, and testing for a Web application.

In one embodiment, the form of the specified test equipment set may be at least one of the following: a single test equipment, a test equipment pair composed of two test equipment, and a test equipment group composed of more than two test equipment .

In an implementation manner, the specified test equipment set may include more than one test equipment. The executing the automated test may include: using the test agent to coordinately execute the automated test with the one or more test devices.

In one embodiment, the test agent may be implemented independently of the specified set of test equipment, or implemented in the specified set of test equipment.

In an implementation manner, the method 700 may further include: by the test agent, determining whether the test task includes an authorization code of the specified test equipment set, the authorization code is from the test request. The automated testing may be performed in response to determining that the testing task includes the authorization code.

In one embodiment, the method 700 may also include creating the test agent through the following operations: running a test agent creation program on the terminal device; at the terminal device, initiating a registration process to the registration center; and responding Upon completion of the registration process, the terminal device is configured as the test proxy.

The method 700 may further include: receiving, through the registration center, an authorization code generation request for generating an authorization code for a set of test devices associated with the test agent; generating the authorization code through the registration center; and The registration center sends the generated authorization code to the test agent.

In one embodiment, the method 700 may further include: through the registration center, receiving an authorization code acquisition request for obtaining the authorization code of the specified test equipment set; through the registration center, obtaining the authorization code forwarding the request to the testing proxy; and determining, by the registry, whether to provide the authorization code based on a response received from the testing proxy.

In an implementation manner, the method 700 may further include visualizing test results of the automated test. The test results may include a video navigation interface. The video navigation interface may include: a navigation area for displaying a plurality of test cases included in the automated test, each test case can be selected, and a video for displaying a video segment corresponding to the selected test case area.

It should be understood that the method 700 may also include any steps/processes for performing automated testing according to the above-mentioned embodiments of the present disclosure.

FIG. 8 illustrates an exemplary automated testing system 800 according to an embodiment of the disclosure.

The automated testing system 800 may include a registration center 810 configured to receive a test request for using a specified test equipment set to perform automated testing for a target application, generate a test task corresponding to the test request, and store the test task scheduling to test agents associated with the specified set of test equipment; at least one test agent 820, each test agent configured to receive a test task and perform automation using the specified set of test equipment in the received test task testing; and at least one set of test equipment 830, each set of test equipment associated with one of the at least one test agent and configured to run the target application.

In an implementation manner, the target application may include at least one of the following: a single operating system application, a cross-operating system application, and a Web application.

In one embodiment, the form of each test equipment set in the at least one test equipment set may be at least one of the following: a single test equipment, a test equipment pair consisting of two test equipment, and a test equipment set consisting of more than two Test equipment constitutes a test equipment group.

In one embodiment, each test agent of the at least one test agent may be implemented independently of or within the set of test equipment associated with the test agent.

In one embodiment, each test agent of the at least one test agent may be further configured to: determine whether the received test task includes an authorization code corresponding to a specified set of test equipment; A task includes the authorization code to execute the automated test.

In one embodiment, the registration center may also be configured to: receive a registration request from a terminal device, the registration request is triggered by a test agent creation program running on the terminal device; and respond to receiving The registration request registers the terminal device as a test agent.

The registration center may also be configured to: receive an authorization code generation request for generating an authorization code for a set of test devices associated with the registered test agent; generate the authorization code; and send the generated authorization code to The registered testing agent.

In an implementation manner, the registration center may also be configured to: receive an authorization code acquisition request for acquiring the authorization code of the specified test equipment set; forward the authorization code acquisition request to the test agent; and determining whether to provide the authorization code based on a response received from the testing agent.

In an implementation manner, the registration center may also be configured to visualize test results of the automated test. The test results may include a video navigation interface. The video navigation interface may include: a navigation area for displaying a plurality of test cases included in the automated test, each test case can be selected, and a video for displaying a video segment corresponding to the selected test case area.

It should be understood that the automated testing system 800 may also include any other components configured to perform automated testing according to the above-mentioned embodiments of the present disclosure.

FIG. 9 shows an exemplary apparatus 900 for performing automated testing according to an embodiment of the present disclosure.

Apparatus 900 may include: at least one processor 910; and memory 920 storing computer-executable instructions. The computer-executable instructions, when executed, may cause the at least one processor 910 to: receive, through a registry, a test request for performing an automated test using a specified set of test equipment; Test a test task corresponding to the request; through the registration center, schedule the test task to a test agent associated with the specified test equipment set; and through the test agent, use the specified test equipment set to execute The automated tests.

It should be understood that the processor 910 may also perform any other steps/processes of the method for performing automated testing according to the above-mentioned embodiments of the present disclosure.

Embodiments of the present disclosure propose a computer program product for performing automated testing, comprising a computer program executed by at least one processor for: receiving, through a registration center, a program for performing automated testing with a specified set of test equipment through the registration center, generating a test task corresponding to the test request; through the registration center, dispatching the test task to a test agent associated with the specified test equipment set; and through The test agent executes the automated test by using the specified set of test equipment. In addition, the computer program can also be executed to implement any other steps/processes of the method for performing automated testing according to the above-mentioned embodiments of the present disclosure.

Embodiments of the present disclosure can be embodied on a non-transitory computer readable medium. The non-transitory computer-readable medium may include instructions that, when executed, cause one or more processors to perform any of the operations of the method for performing automated testing according to an embodiment of the present disclosure as described above. .

It should be understood that all operations in the method described above are exemplary only, and the present disclosure is not limited to any operation in the method or the order of these operations, but should cover all other equivalent transformations under the same or similar concept . In addition, the articles "a (a)" and "an (an)" as used in this specification and the appended claims should generally be construed unless otherwise specified or clear from the context to refer to a singular form. means "one" or "one or more".

It should also be understood that all modules in the apparatus described above may be implemented in various ways. These modules may be implemented as hardware, software, or a combination thereof. Furthermore, any of these modules may be functionally further divided into sub-modules or grouped together.

Processors have been described in connection with various apparatus and methods. These processors may be implemented using electronic hardware, computer software, or any combination thereof. Whether such processors are implemented as hardware or software will depend upon the particular application and overall design constraints imposed on the system. As examples, a processor, any portion of a processor, or any combination of processors presented in this disclosure may utilize a microprocessor, microcontroller, digital signal processor (DSP), field programmable gate array (FPGA) , programmable logic devices (PLDs), state machines, gating logic units, discrete hardware circuits, and other suitable processing components configured to perform the various functions described in this disclosure. The functions of a processor, any part of a processor, or any combination of processors given in this disclosure may be implemented by software executed by a microprocessor, microcontroller, DSP, or other suitable platform.

Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, threads of execution, procedures, functions, etc. The software may reside on a computer readable medium. The computer readable medium may include, for example, memory, which may be, for example, a magnetic storage device (e.g., hard disk, floppy disk, magnetic strip), an optical disk, a smart card, a flash memory device, random access memory (RAM), read only memory (ROM), Programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), register or removable disk. Although memory is shown as being separate from the processor in the various aspects presented in this disclosure, memory may also be located internal to the processor, such as a cache or register.

The above description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects. Accordingly, the claims are not intended to be limited to the aspects shown herein. All structural and functional equivalents to the elements of the described aspects of this disclosure that are known or come to be known to those of ordinary skill in the art are expressly incorporated herein and are covered by the claims.

Claims (20)

1. A method for performing automated testing comprising: receiving, through the registry, a test request for performing automated testing using a specified set of test equipment; Generate a test task corresponding to the test request through the registration center; dispatching, via the registry, the test tasks to test agents associated with the specified set of test devices; and The automated testing is performed by the testing agent using the specified set of testing equipment. 2. The method according to claim 1, wherein the automated testing comprises at least one of the following: testing for single operating system applications, testing for cross-operating system applications, and testing for Web applications. 3. The method according to claim 1, wherein the form of the designated test equipment set is at least one of the following: single test equipment, a test equipment pair consisting of two test equipment, and A test equipment group consisting of two or more test equipment. 4. The method of claim 1, wherein the specified set of test equipment includes more than one test equipment, and the performing the automated test comprises: The automated testing is performed cooperatively by the testing agent using the one or more testing devices. 5. The method of claim 1, wherein the test agent is implemented independently of or within the specified set of test equipment. 6. The method of claim 1, further comprising: determining, by the test agent, whether the test task includes an authorization code for the specified set of test equipment, the authorization code from the test request, and Wherein, the automated testing is performed in response to determining that the testing task includes the authorization code. 7. The method of claim 1, further comprising creating the test proxy by: Run the test agent creation program on the terminal device; at the terminal device, initiate a registration process with the registration center; and In response to completion of the registration process, the terminal device is configured as the testing agent. 8. The method of claim 7, further comprising: receiving, via the registration center, an authorization code generation request for generating an authorization code for a set of test devices associated with the test agent; generating the authorization code via the registry; and The generated authorization code is sent to the testing agent through the registration center. 9. The method of claim 1, further comprising: receiving an authorization code acquisition request for obtaining an authorization code of the designated test equipment set through the registration center; forwarding the authorization code acquisition request to the testing agent through the registration center; and Whether to provide the authorization code is determined by the registry based on a response received from the testing agent. 10. The method of claim 1, further comprising visualizing test results of the automated test, the test results comprising a video navigation interface, and the video navigation interface comprising: a navigation area for displaying a plurality of test cases included in the automated test, each test case can be selected, and The video area used to display the video clip corresponding to the selected test case. 11. An automated testing system comprising: The registration center is configured to receive a test request for performing an automated test for a target application by using a specified set of test equipment, generate a test task corresponding to the test request, and schedule the test task to the specified the test agent associated with the set of test devices; at least one test agent, each test agent configured to receive test tasks and perform automated testing using a set of test equipment specified in the received test tasks; and At least one set of test equipment, each set of test equipment associated with one of the at least one test agent and configured to run the target application. 12. The automated testing system according to claim 11, wherein the target application comprises at least one of the following: a single operating system application, a cross-operating system application and a Web application. 13. The automated testing system according to claim 11, wherein each test equipment set in the at least one test equipment set is in at least one of the following forms: single test equipment, a test equipment pair consisting of two test equipment, and A test equipment group consisting of two or more test equipment. 14. The automated testing system of claim 11 , wherein each of the at least one test agent is implemented independently of a set of test equipment associated with the test agent, or is implemented in the test Equipment concentration. 15. The automated testing system of claim 11 , wherein each of the at least one testing agent is further configured to: determining whether the received test task includes an authorization code for the corresponding specified set of test equipment; and The automated testing is performed in response to determining that the received testing task includes the authorization code. 16. The automated testing system according to claim 11, the registration center is further configured to: receiving a registration request from an end device, the registration request being triggered by a test agent creation program running on the end device; and In response to receiving the registration request, the terminal device is registered as a test agent. 17. The automated testing system according to claim 16, wherein the registration center is further configured to: receiving an authorization code generation request for generating an authorization code for a set of test devices associated with the registered test agent; generate said authorization code; and Send the generated authorization code to the registered testing agent. 18. The automated testing system according to claim 11, wherein the registration center is further configured to: receiving an authorization code obtaining request for obtaining the authorization code of the specified test equipment set; forwarding the authorization code acquisition request to the testing agent; and Whether to provide the authorization code is determined based on a response received from the testing agent. 19. The automated testing system according to claim 11 , wherein the registration center is further configured to visualize test results of the automated tests, the test results include a video navigation interface, and the video navigation interface includes : a navigation area for displaying a plurality of test cases included in the automated test, each test case can be selected, and The video area used to display the video clip corresponding to the selected test case. 20. A computer program product for performing automated testing, comprising a computer program executed by at least one processor for: receiving, through the registry, a test request for performing automated testing using a specified set of test equipment; Generate a test task corresponding to the test request through the registration center; dispatching, via the registry, the test tasks to test agents associated with the specified set of test devices; and The automated testing is performed by the testing agent using the specified set of testing equipment.

Images