CN114546740A - Touch screen testing method, device and system and storage medium - Google Patents

Abstract

The invention provides a touch screen testing method, device, system and storage medium. The method includes: acquiring first image data displayed on a touch screen of a device to be tested in real time; acquiring feature data of a control to be operated according to configuration information of a test task , and determine whether there is a control to be operated in the first image data according to the feature data; determine that there is a control to be operated in the first image data, then obtain the actual coordinates of the corresponding control according to the image coordinates of the operation control; control the execution end of the robotic arm to move To the actual coordinates of the control, perform the first operation on the touch screen of the device under test; acquire second image data displayed on the touch screen of the device under test, and determine the test result of the test task according to the second image data. The present invention does not depend on the attributes of UI controls, and accurately locates the controls of the touch screen interface in real time through machine vision, which improves test efficiency while simulating manual operations.

Description

Touch screen testing method, device, system and storage medium

technical field

The invention relates to the technical field of product testing, in particular to a touch screen testing method, device, system and storage medium.

Background technique

At present, there are a wide variety of touch-screen electronic products on the market with rich functions, and their system versions are iteratively updated quickly. Automated testing of such touch screen products is usually performed by obtaining properties of UI controls or manipulating UI controls through test instructions. However, this method has the following defects: sometimes the control properties cannot be obtained, resulting in the inability to operate the UI interface. For example, third-party customized applications do not support users to obtain control properties or some controls cannot obtain properties after the Android system is upgraded.

SUMMARY OF THE INVENTION

The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, embodiments of the present invention provide a touch screen testing method, device, system, and storage medium.

On the one hand, an embodiment of the present invention provides a touch screen testing method, including: acquiring first image data displayed on a touch screen of a device under test in real time; acquiring feature data of a control to be operated according to configuration information of a test task, and The feature data determines whether the to-be-operated control exists in the first image data; if it is determined that the to-be-operated control exists, acquire the image coordinates of the to-be-operated control in the first image data, and according to the The image coordinates are used to obtain the corresponding actual coordinates of the control, and the actual coordinates of the control are used to represent the position of the control to be operated on the touch screen side of the device under test; control the execution end of the robotic arm to move to the actual coordinates of the control. The touch screen of the device under test performs a first operation, wherein the first operation is determined by the configuration of the test task; acquires second image data displayed on the touch screen of the device under test, according to the second image The data determines the test result of the test task; the second image data is used to represent the display image of the touch screen of the device under test after the first operation is performed.

The touch screen testing method according to the embodiment of the present invention has at least the following beneficial effects: by performing real-time photography on the touch screen of the device to be tested, the corresponding control to be operated in the test task is identified, and the corresponding control to be operated is obtained through the image coordinates of the control to be operated. The actual coordinates of the control, move the execution end of the robotic arm to the actual coordinates of the control, perform corresponding operations on the control to be operated on the touch screen, and obtain the display image of the response of the device to be tested, and generate the corresponding test results; independent of UI control properties , and through machine vision, the controls of the touch screen interface are accurately positioned in real time, and the manipulator is controlled to operate the UI interface controls on the touch screen products, which simulates the manual operation and improves the test efficiency, avoiding the use of commands or serial ports bypassing the touch screen. , the test error caused by the inconsistency with the actual operation of the user.

According to some embodiments of the present invention, the feature data of the to-be-operated control includes third image data of the to-be-operated control; the determining whether the to-be-operated control exists in the first image data according to the feature data , including: obtaining a matching result through image recognition, where the matching result is used to represent whether there is an area matching the third image data in the first image data; if the matching result is that there is an area matching the third image data; If the image data matches the area, it is determined that the control to be operated exists in the first image data.

According to some embodiments of the present invention, the feature data of the to-be-operated control includes a first character, and the first character is used to represent text information displayed by the to-be-operated control; the first image is determined according to the feature data Whether the to-be-operated control exists in the data includes: acquiring the text recognition result of the first image data; if the text recognition result includes the first text, determining that the to-be-operated control exists in the first image data Action controls.

According to some embodiments of the present invention, the configuration information of the test task includes: a first position of the control to be operated, where the first position is used to represent when the control to be operated is displayed on the touch screen of the device under test The partition information where it is located; the determining whether the control to be operated exists in the first image data according to the feature data includes: obtaining a corresponding recognition area image from the first image data according to the first position ; Determine whether the control to be operated exists in the recognition area image according to the feature data.

According to some embodiments of the present invention, the obtaining the corresponding actual coordinates of the control according to the image coordinates includes: obtaining corresponding calibration data according to the type of the device under test; the calibration data is used to represent The touch screen of the device under test, the mapping relationship between the actual movement distance of the execution end and the image display movement distance; according to the calibration data, the image coordinates are converted into the actual coordinates of the control.

According to some embodiments of the present invention, the method for acquiring the calibration data includes: displaying a real-time captured image of the touch screen of the device under test through an interactive interface; in the real-time captured image, the touch screen of the device under test is The area is determined as a first area, and a first calibration point and a second calibration point are determined in the first area; the execution end is controlled to move from the origin position, so that the execution end is in the real-time captured image with the The first calibration point is aligned to obtain the first actual coordinates of the execution end; the movement of the execution end is controlled so that the execution end is aligned with the second calibration point in the real-time captured image, and the execution end is obtained. obtaining the calibration data according to the screen coordinates of the first calibration point, the screen coordinates of the first calibration point, the first actual coordinates and the second actual coordinates.

According to some embodiments of the present invention, the method further includes configuring the axial movement speed and touch delay of the horizontal plane of the manipulator; the controlling the execution end of the manipulator to move to the actual coordinates of the control, Performing the first operation on the touch screen of the device under test includes: adjusting the height of the execution end according to the thickness of the device under test; controlling the execution of the robot arm according to the axial movement speed of the horizontal plane of the robot arm The terminal moves to the actual coordinates of the control; and after the touch delay is delayed, the first operation is performed on the touch screen of the device under test.

On the other hand, an embodiment of the present invention provides a touch screen testing device, including: a first module for acquiring first image data displayed on a touch screen of a device under test in real time; a second module for configuring according to a test task information, obtain the characteristic data of the control to be operated, and determine whether the control to be operated exists in the first image data according to the characteristic data; the third module is used for obtaining the control to be operated if it is determined that the control to be operated exists. The image coordinates of the control to be operated in the first image data, and the corresponding actual coordinates of the control are obtained according to the image coordinates, and the actual coordinates of the control are used to represent the control to be operated on the touch screen of the device under test. The fourth module is used to control the execution end of the robotic arm to move to the actual coordinates of the control, and perform a first operation on the touch screen of the device under test, wherein the first operation is determined by the configuration of the test task determine; the fifth module is used to acquire second image data displayed on the touch screen of the device under test, and determine the test result of the test task according to the second image data; the second image data is used to represent A display image of the touch screen of the device under test after the first operation is performed.

The touch screen testing device according to the embodiment of the present invention has at least the following beneficial effects: by performing real-time photography on the touch screen of the device to be tested, the corresponding control to be operated in the test task is identified, and the corresponding control to be operated is obtained through the image coordinates of the control to be operated. The actual coordinates of the control, move the execution end of the robotic arm to the actual coordinates of the control, perform corresponding operations on the control to be operated on the touch screen, and obtain the display image of the response of the device to be tested, and generate the corresponding test results; independent of UI control properties , and through machine vision, the controls of the touch screen interface are accurately positioned in real time, and the manipulator is controlled to operate the UI interface controls on the touch screen products, which simulates the manual operation and improves the test efficiency, avoiding the use of commands or serial ports bypassing the touch screen. , the test error caused by the inconsistency with the actual operation of the user.

On the other hand, an embodiment of the present invention provides a touch screen testing system, including: a robotic arm, connected to a computer device through a serial port, for receiving an instruction from the computer device to control the execution of terminal movement, and to perform a touch operation on the touch screen of the device to be tested; a camera; The device is erected above the device to be tested, and is used for real-time shooting of images displayed on the touch screen of the device to be tested; computer equipment, including at least one processor, and at least one for storing at least one program The memory, when the at least one program is executed by the at least one processor, enables the at least one processor to implement the above-described touch screen testing method.

On the other hand, a computer-readable storage medium of the present invention stores program instructions in the storage medium, and when the program instructions are executed by a processor, the above-mentioned touch screen testing method is implemented.

Additional aspects and advantages of the present invention will be set forth, in part, from the following description, and in part will be apparent from the following description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

FIG. 1 is a flow chart of steps of a touch screen testing method provided by an embodiment of the present invention.

FIG. 2 is a flowchart of steps of a method for acquiring calibration data in an embodiment of the present invention.

3 is a schematic diagram of converting image coordinates into actual coordinates for different types of testing equipment in an embodiment of the present invention.

FIG. 4 is a schematic block diagram of a touch screen testing device provided by an embodiment of the present invention.

FIG. 5 is a schematic diagram of the connection of each device in the touch screen test system provided by the embodiment of the present invention.

Detailed ways

The following describes in detail the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. In the ensuing description, suffixes such as 'module', 'component' or 'unit' used to represent elements are used only to facilitate the description of the present invention, and have no specific meaning per se. Thus, "module", "component" or "unit" may be used interchangeably. "First", "second", etc. are only used for the purpose of distinguishing technical features, and should not be understood as indicating or implying relative importance, or implicitly indicating the number of indicated technical features or implicitly indicating the indicated technical features. successive relationship. In the following description, the consecutive numbers on the method steps are for the convenience of review and understanding. Combined with the overall technical solution of the present invention and the logical relationship between the various steps, adjusting the execution order of the steps will not affect the technology of the present invention. The technical effect achieved by the program. The embodiments described below with reference to the accompanying drawings are exemplary, only used to explain the present invention, and should not be construed as a limitation of the present invention.

At present, the method for automated testing of touch screen products is mainly to operate the control by obtaining the properties of the UI control or to operate the control through a test command or the like. However, this test method has certain defects. For example, using automated test frameworks such as Uiautomator, Appium, and Airtest to obtain the properties of UI interface controls, and perform automated tests on controls. There are two flaws. On the one hand, sometimes the control attributes cannot be obtained, resulting in the inability to operate the corresponding UI interface controls. For example, third-party customized applications do not support users to obtain control attributes, or some controls cannot obtain attributes after the Android system is upgraded. On the other hand, the method of testing through the test command is out of the actual usage scenario of the user, and there may be a certain test deviation due to this.

To this end, the present invention provides a touch screen testing method, which can accurately locate the controls of the touch screen interface through machine vision, and control the manipulator to operate the UI interface controls on the touch screen products, so as to simulate the actual use scene of the user. Since the present invention does not depend on the acquisition of UI interface control attributes, it can also avoid programming dependent on the operating system of the product, reduce technical requirements for testers and improve test efficiency.

Referring to FIG. 1 , this embodiment discloses a touch screen testing method, which can test a device under test with a touch screen, including but not limited to the following steps S100 to S500 .

Step S100, acquiring the first image data displayed on the touch screen of the device under test in real time.

Specifically, the camera device is erected above the device to be tested, and the area including the touch screen of the device to be tested is photographed, so as to obtain the display screen on the touch screen of the device to be tested in real time as the first image data. The camera device can send the captured first image data through a USB cable.

Step S200: Acquire characteristic data of the control to be operated according to the configuration information of the test task, and determine whether the control to be operated exists in the first image data according to the characteristic data.

In this embodiment, the test task may exist in the form of a test script. The test task may include at least one test case, and each test case may operate at least one control to be operated. The configuration information of the test task includes relevant information of at least one control to be operated, and the relevant information of each control to be operated includes, for example, characteristic data of the control to be operated, the first operation, the expected result, and the like. The feature data of the control to be operated is used to find and locate the position of the control to be operated. The first operation is used to perform a corresponding touch operation on the control to be operated, such as clicking, dragging, and the like. The expected result may be a response picture obtained by performing the first operation, or may be an image feature or a text feature of the response picture.

Exemplarily, the characteristic data of the control to be operated may include at least one of the following: third image data, or, first text. The third image data includes the image to be operated and controlled. The first text is used to represent text information displayed by the control to be operated, for example: all or part of the text displayed on the control to be operated.

When the characteristic data of the to-be-operated control is the third image data, an area matching the third image data is acquired from the first image data through image recognition. If there is an area matching the third image data, it is determined that the to-be-operated control exists in the first image data. When performing image recognition, a recognition threshold can be configured. When the matching degree between the image area in the first image data and the third image data is greater than the recognition threshold, it is determined that the to-be-operated control exists in the first image data, and the to-be-operated control is obtained. Controls the image coordinates in the first image data. In this embodiment, it is also possible to set whether the image recognition is grayscale recognition or color recognition. When it is grayscale recognition, the first image data and the third image data are converted into grayscale images for matching; when it is color recognition, the color of the matching area and the color of the third image data are compared during matching. is within the margin of error. Before the implementation of the test, the third image data can take a screenshot of the control to be operated, and save it in a corresponding address; configure the address in the test task.

When the characteristic data of the control to be operated is the first character, the character recognition result of the first image data is obtained through character recognition, and the first character is searched in the character recognition result. If the first text is found, it is determined that there is a corresponding control to be operated in the first image data, and the image coordinates of the control to be operated in the first image data are acquired.

In this embodiment, the touch screen of the device under test can also be divided into multiple partitions, and the first position of the control to be operated is configured in the test task, and the first position is used to indicate which partition the control to be operated is displayed in middle. The positioning process of the to-be-operated control is as follows: according to the first position, the image data of the corresponding partition is obtained from the first image data as the identification area image; according to the characteristic data of the to-be-operated control, it is determined whether the to-be-operated image exists in the identification area image controls. If the to-be-operated control exists, the image coordinates of the to-be-operated control in the first image data are acquired. Screen partition recognition can prevent misjudgment of control positioning and improve recognition accuracy.

In this embodiment, when there is no corresponding control to be operated in the first image data, the information of the matching failure is recorded, such as the operation type of the device to be tested, the time of the matching failure, and the like. And, when the matching fails, the test of the to-be-operated control is suspended, or the test of the to-be-operated control is suspended, and the test of the to-be-operated control is suspended after waiting for a period of time to retry, and when the retry exceeds a certain number of times, the test of the to-be-operated control is suspended. After the test of the to-be-operated control is terminated, the next to-be-operated control may be tested until all the to-be-operated controls in the test task are executed.

Step S300, if it is determined that there is a to-be-operated control, obtain the image coordinates of the to-be-operated control in the first image data, and obtain the corresponding actual coordinates of the control according to the image coordinates.

The actual coordinates of the control are used to represent the position of the control to be operated on the touch screen side of the device under test.

Specifically, according to the type of the device to be tested, the corresponding calibration data is obtained; according to the calibration data, the image coordinates are converted into the actual coordinates of the control. The calibration data is used to characterize the mapping relationship between the actual movement distance of the execution end of the robotic arm and the image display movement distance for the touch screen of the device under test. By converting the image coordinates into the actual coordinates of the controls, it is convenient to control the execution end of the robotic arm to move to the controls to be operated on the touch screen of the device under test to perform corresponding operations.

Referring to FIG. 2 , the method for acquiring calibration data includes but is not limited to steps S710 to S750.

Step S710, displaying a real-time captured image of the touch screen of the device under test through an interactive interface.

For example, the touch screen of the device under test is photographed in real time by the camera device, and the acquired image is displayed on the display interface of the terminal connected to the camera device.

In step S720, in the real-time captured image, the touch screen area of the device under test is determined as the first area, and the first calibration point and the second calibration point are determined in the first area.

The touch screen area of the device to be tested can be determined as the first area through the interactive interface, or the first area can be demarcated by a software program by using the different brightness when the touch screen is displayed. The first calibration point and the second calibration point may be any two different points on the touch screen of the device under test. In this embodiment, as shown in FIG. 3 , B' is the first area of the device to be tested, the point (x1, y1) in the upper left corner is the first calibration point, and the point (x2, y2) in the lower right corner is the first calibration point. is the second calibration point.

In step S730, the execution end is controlled to move from the origin position, so that the execution end is aligned with the first calibration point in the real-time captured image, and the first actual coordinates of the execution end are obtained.

Specifically, by sending a movement command to the robot arm, the robot arm can be controlled to move in the X axis and the Y axis in the first region, and the sent movement command is stored. The movement command can be sent to the robot arm in the form of pulses, for example, sending 1000 pulses makes the robot arm advance 5mm along the X axis. According to the movement instruction sent during the process of moving the execution end of the manipulator from the origin position to being aligned with the first calibration point, the first movement data of the execution end from the origin can be obtained. The first movement data includes movement data in the X-axis and the Y-axis. Furthermore, the first actual coordinates of the execution end can be obtained according to the first movement data. For example, if the coordinates of the origin position are (0, 0) and the first movement data is (87mm, 52mm), obviously, the first actual coordinates are (87, 52) (unit: mm).

In step S740, the execution end is controlled to move, so that the execution end is aligned with the second calibration point in the real-time captured image, and the second actual coordinates of the execution end are obtained.

Specifically, the execution end can be controlled to move from the origin position to be aligned with the second calibration point, and the execution end can also be controlled to move from the first calibration point to the second calibration point.

Step S750: Acquire calibration data according to the screen coordinates of the first calibration point, the screen coordinates of the first calibration point, the first actual coordinates and the second actual coordinates.

Referring to FIG. 3 , the coordinate unit of the robotic arm is millimeters (mm), its origin position is Machine_0, and the X-axis and Y-axis of the coordinate system of the robotic arm are represented as Machine_X and Machine_Y, respectively.

In this embodiment, the display interface is a PC display interface. Therefore, in FIG. 3, the coordinate origin corresponding to the display interface is marked as PC_0, the X axis and the Y axis of the coordinate system of the display interface are respectively represented as PC_X and PC_Y, and the coordinate unit is Pixel (pixel), the unit of the framed first area (B' area) is also pixel (pixel).

In FIG. 3 , (x1, y1) and (x2, y2) are the screen coordinates of the first calibration point and the second calibration point, respectively. The actual coordinates of the two calibration points in the first area (B' area) corresponding to the execution terminal of the robotic arm are (Xadj1, Yadj1) and (Xadj2, Yadj2) respectively. Among them, (Xadj1, Yadj1) corresponds to (x1, y1), and (Xadj2, Yadj2) corresponds to (x2, y2).

Through FIG. 3, calibration data can be obtained, and the calibration data includes: the mapping relationship X_scale between the actual moving distance of the X-axis of the execution end and the displayed moving distance, and the mapping relationship between the actual moving distance of the X-axis of the execution end and the displayed moving distance Y_scale. In FIG. 3, the calibration data can be understood as the proportional relationship of the number of pixels on the display interface corresponding to each unit (millimeter) movement of the robotic arm in the X-axis direction and the Y-axis direction. That is:

X_scale=(x2-x1)/(Xadj2-Xadj1)

Y_scale=(y2-y1)/(Yadj2-Yadj1)

If the image coordinate point corresponding to the control to be operated is (x, y), the actual coordinates (Xm, Ym) of the control corresponding to the control to be operated can be calculated in the following ways. The calculation method is as follows:

Xm=Xadj1+(x-x1)/X_scale

Ym=Yadj1+(y-y1)/Y_scale

Since the camera is generally installed above the equipment to be operated, for a camera device with a certain height, when the same object in the field of view is at different distances from the camera, the size displayed in the camera preview or in the imaging will be different. That is to say, the thickness of the device to be tested will cause the images obtained by the camera to be different in size, which will lead to errors in the control of the robotic arm. Compared with thinner mobile devices such as tablet computers and smart phones, when testing the control operation of thicker devices such as attendance machines, the difference in imaging will lead to great errors, which makes the automation of the controls possible. Testing is almost impossible.

In this embodiment, for different types of equipment to be tested, two calibration points are selected in the touch screen area of the equipment to be tested, calibration is performed, calibration data is obtained, and the coordinates of the image to be operated and controlled are converted into the coordinates of the execution end of the robotic arm according to the calibration data. The actual sitting side can improve the control accuracy of the robotic arm for the equipment to be operated with different thicknesses, so as to achieve the purpose of precisely touching the controls on the screen of the equipment to be operated.

Step S400, controlling the execution end of the robotic arm to move to the actual coordinates of the control, and performing the first operation on the touch screen of the device under test.

The first operation is determined by the configuration of the test task.

In this embodiment, the corresponding parameters of the manipulator are also configured. Taking a three-axis manipulator as an example, it includes the movement speed of the two axes of the horizontal plane (that is, the X-axis and the Y-axis), and the movement in the Z-axis direction perpendicular to the horizontal plane. Speed, touch delay, and whether the robotic arm returns to the origin after each operation is completed.

Specifically, according to the thickness of the device to be tested, the height of the execution end of the manipulator is adjusted; then, according to the axial movement speed of the horizontal plane of the manipulator, the execution end of the manipulator moves to the actual coordinates of the control; The touch screen of the device performs the first operation.

Step S500, acquiring second image data displayed on the touch screen of the device under test, and determining a test result of the test task according to the second image data.

The second image data is used to represent the display image of the touch screen of the device under test after the first operation is performed.

In this embodiment, a corresponding test report can also be generated according to the test result of the test task, and the report includes, for example, the type of the device to be tested, the execution start and end time of the test task (including the start time and the end time), and the total number of test cases. Quantity, total number of action controls, number of pass/fail items, and the running result of each test case.

This embodiment does not depend on the acquisition of UI interface control attributes, and can accurately locate the touch screen interface controls through machine vision, and control the manipulator to perform UI interface control operations on touch screen products, so as to simulate the actual use scenario of the user. Avoid programming dependent on the product's operating system, reduce technical requirements for testers, and improve test efficiency.

4 , the present embodiment discloses a touch screen testing device 400 , including but not limited to a first module 410 , a second module 420 , a third module 430 , a fourth module 440 and a fifth module 450 .

The first module 410 is configured to acquire the first image data displayed on the touch screen of the device under test in real time. The second module 420 is configured to acquire characteristic data of the control to be operated according to the configuration information of the test task, and determine whether the control to be operated exists in the first image data according to the characteristic data. The third module 430 is configured to acquire the image coordinates of the to-be-operated control in the first image data if it is determined that there is a to-be-operated control, and to acquire the corresponding actual coordinates of the control according to the image coordinates. The actual coordinates of the control are used to represent the position of the control to be operated on the touch screen side of the device under test. The fourth module 440 is used to control the execution end of the robotic arm to move to the actual coordinates of the control, and perform a first operation on the touch screen of the device under test, where the first operation is determined by the configuration of the test task. The fifth module 450 is configured to acquire the second image data displayed on the touch screen of the device under test, and determine the test result of the test task according to the second image data. The second image data is used to represent the display image of the touch screen of the device under test after the first operation is performed.

The touch screen test device shown in FIG. 4 and the touch screen test method shown in FIG. 1 are based on the same inventive concept. Therefore, under the synergistic effect of the above-mentioned modules, the emotion recognition device at least has the same touch screen as the touch screen test method shown in FIG. 1 . The same beneficial effect of the screen test method.

Referring to FIG. 5 , this embodiment provides a touch screen testing system, including: a robotic arm 510 , a camera device 520 and a computer device 530 . As shown in FIG. 5 , the robotic arm 510 is connected to the computer device 530 through a serial port, receives an instruction from the computer device to control the movement of the execution end, and performs touch operations on the touch screen of the device under test. The camera device 520 is erected above the device under test, and is used for real-time shooting of the image displayed on the touch screen of the device under test, and sends the acquired image to the computer device 530 . The computer device 530 includes at least one processor, and at least one memory for storing at least one program, when the at least one program is executed by the at least one processor, the at least one processor implements the touch screen testing method as described above. The processor may also be referred to as a CPU (Central Processing Unit, central processing unit). The processor may be an integrated circuit chip with signal processing capability. The processor may also be a general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. A general-purpose processor may be a microprocessor, or, a general-purpose processor may be any conventional processor, or the like. The memory may include various components (eg, machine-readable media) including, but not limited to, random access memory components, read-only components, and any combination thereof. It should be understood that, in order to avoid redundant description, for the content not involved in this embodiment, reference may be made to the content not involved in this embodiment, and reference may be made to the above-mentioned touch screen testing method.

This embodiment provides a computer-readable storage medium, where program instructions are stored in the storage medium, and when the program instructions are executed by the processor, the above-mentioned touch screen testing method can be implemented. It is hoped to be understood that, in order to avoid redundant description, reference may be made to the above-mentioned touch screen testing method for the content not involved in this embodiment.

Those of ordinary skill in the art can understand that all or some steps in the methods disclosed above, and functional modules/units in the device can be implemented as software, firmware, hardware, and appropriate combinations thereof.

In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be composed of several physical components Components execute cooperatively. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit . Such software may be distributed on computer-readable media (storage media for short), which may include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those of ordinary skill in the art, the term computer-readable medium includes both volatile and non-transitory media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules or other data. volatile, removable and non-removable media. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, magnetic tape, magnetic disk storage or other magnetic storage devices, or may Any other medium used to store desired information and which can be accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery media, as is well known to those of ordinary skill in the art .

The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, but are not intended to limit the scope of the rights of the present invention. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and spirit of the present invention shall fall within the right scope of the present invention.

Claims (10)

1. a touch screen testing method, is characterized in that, comprises: Acquire the first image data displayed on the touch screen of the device under test in real time; According to the configuration information of the test task, obtain characteristic data of the control to be operated, and determine whether the control to be operated exists in the first image data according to the characteristic data; If it is determined that the to-be-operated control exists, the image coordinates of the to-be-operated control in the first image data are obtained, and the corresponding actual coordinates of the control are obtained according to the image coordinates, and the actual coordinates of the control are used to represent all the controls. the position of the control to be operated on the touch screen side of the device to be tested; Controlling the execution end of the robotic arm to move to the actual coordinates of the control, and performing a first operation on the touch screen of the device under test, wherein the first operation is determined by the configuration of the test task; Acquire second image data displayed on the touch screen of the device under test, and determine the test result of the test task according to the second image data; the second image data is used to represent the result of performing the first operation. The display image of the touch screen of the device under test. 2 . The touch screen testing method according to claim 1 , wherein the characteristic data of the control to be operated comprises third image data of the control to be operated; and the first image is determined according to the characteristic data. 3 . Whether the control to be operated exists in the data, including: Obtain a matching result through image recognition, where the matching result is used to represent whether there is an area matching the third image data in the first image data; If the matching result is that there is an area matching the third image data, it is determined that the to-be-operated control exists in the first image data. 3. The touch screen testing method according to claim 1 or 2, wherein the characteristic data of the control to be operated comprises a first character, and the first character is used to represent the text information displayed by the control to be operated; the Determining whether the control to be operated exists in the first image data according to the feature data includes: obtaining the character recognition result of the first image data; If the character recognition result includes the first character, it is determined that the control to be operated exists in the first image data. 4 . The touch screen testing method according to claim 1 , wherein the configuration information of the test task comprises: a first position of the to-be-operated control, and the first position is used to represent the display of the to-be-operated control. 5 . The partition information where the touch screen of the device under test is located; the determining whether the control to be operated exists in the first image data according to the feature data includes: obtaining a corresponding recognition area image from the first image data according to the first position; Whether the control to be operated exists in the image of the recognition area is determined according to the feature data. 5. The touch screen testing method according to claim 1, wherein the obtaining the corresponding actual coordinates of the control according to the image coordinates comprises: Acquire corresponding calibration data according to the type of the device under test; the calibration data is used to represent the mapping relationship between the actual movement distance of the execution terminal and the image display movement distance for the touch screen of the device under test; According to the calibration data, the image coordinates are converted into the actual coordinates of the control. 6. The touch screen testing method according to claim 5, wherein the method for acquiring the calibration data comprises: Display the real-time captured image of the touch screen of the device under test through the interactive interface; In the real-time captured image, the touch screen area of the device under test is determined as a first area, and a first calibration point and a second calibration point are determined in the first area; controlling the execution end to move from the origin position, so that the execution end is aligned with the first calibration point in the real-time captured image, to obtain the first actual coordinates of the execution end; controlling the movement of the execution end so that the execution end is aligned with the second calibration point in the real-time captured image to obtain the second actual coordinates of the execution end; The calibration data is acquired according to the screen coordinates of the first calibration point, the screen coordinates of the first calibration point, the first actual coordinates and the second actual coordinates. 7 . The touch screen testing method according to claim 1 , wherein the method further comprises configuring the axial movement speed and touch delay of the horizontal plane of the robotic arm; the control of the execution end of the robotic arm. 8 . Moving to the actual coordinates of the control, performing the first operation on the touch screen of the device under test includes: Adjust the height of the execution end according to the thickness of the device to be tested; Controlling the execution end of the robotic arm to move to the actual coordinates of the control according to the axial movement speed of the horizontal plane of the robotic arm; After waiting for the touch delay, perform the first operation on the touch screen of the device under test. 8. A touch screen testing device, comprising: The first module is used to acquire the first image data displayed in the touch screen of the device under test in real time; a second module, configured to acquire characteristic data of the control to be operated according to the configuration information of the test task, and determine whether the control to be operated exists in the first image data according to the characteristic data; The third module is configured to acquire the image coordinates of the to-be-operated control in the first image data if it is determined that the to-be-operated control exists, and obtain the corresponding actual coordinates of the control according to the image coordinates, and the control The actual coordinates are used to represent the position of the control to be operated on the touch screen side of the device under test; a fourth module, configured to control the execution end of the robotic arm to move to the actual coordinates of the control, and perform a first operation on the touch screen of the device under test, wherein the first operation is determined by the configuration of the test task; The fifth module is used for acquiring the second image data displayed on the touch screen of the device under test, and determining the test result of the test task according to the second image data; the second image data is used to represent the execution The display image of the touch screen of the device under test after the first operation. 9. A touch screen test system, comprising: The robotic arm is connected to the computer equipment through the serial port, and is used to receive instructions from the computer equipment to control the movement of the terminal, and to perform touch operations on the touch screen of the device to be tested; a camera device, erected above the device to be tested, and used for real-time shooting of images displayed on the touch screen of the device to be tested; Computer device, comprising at least one processor, and, at least one memory for storing at least one program that, when executed by the at least one processor, causes the at least one processor to implement the at least one processor as claimed in claims 1 to The touch screen test method described in any one of 7. 10 . A computer-readable storage medium, wherein program instructions are stored in the storage medium, and when the program instructions are executed by a processor, the touch screen testing method according to any one of claims 1 to 7 is implemented.

Images