CN102830958B - A kind of method and system for obtaining interface control information - Google Patents

Abstract

A method and system for obtaining interface control information, comprising: selecting an area in the interface according to a preset strategy, and intercepting an image of the area, wherein the selected area is consistent with the previously obtained area containing the control to be detected; according to According to the pixel combination of the image of the selected area, the pixel features of the image of the selected area are obtained according to the preset strategy; according to the pixel feature of the image of the selected area and the pixel feature of the image of the area containing the control to be detected obtained in advance according to the preset strategy Compare the results to determine the information of the interface controls in the selected area. By applying the present invention, by analyzing the pixel characteristics of the image of the non-standard window control and comparing it with the pixel characteristics of the region image containing the control to be detected obtained in advance according to the preset strategy, it can be judged whether there is a The required controls can also obtain the status information of the controls in the selected area, realizing the purpose of automatic testing of non-standard window controls through a unified method.

Description

A method and system for acquiring interface control information

technical field

The invention relates to the field of computer technology, in particular to a method and system for acquiring interface control information.

Background technique

With the rapid development of computer technology and the wide application of Windows system, there are more and more various application programs running on Windows system. In the automated testing of Windows software, it is necessary to use programs instead of people to operate the software under test, so the program It is necessary to get the position of a certain control in the window, and then perform mouse or keyboard operations on it.

Windows provides methods for obtaining standard control information, such as FindWindow, GetWindow, GetWindowRect and other system APIs (application programming interfaces) that can obtain specific information of a control. Windows windows or standard controls have properties such as class name and text.

The FindWindow function can get the handle of the window through the text and class name of the window. The GetWindow function can get the handle of the control through the handle of the parent window and the text of the control. The GetWindowRect function can get the coordinate information of the control relative to the screen through the handle of the control. After obtaining the screen coordinate information, the automated test program can perform clicks, focus switching, and key operations on it.

However, the controls in the Windows graphical interface are not necessarily created in a standard way. Such non-standard controls cannot or do not process standard Windows messages, and may not have properties such as control text, so functions such as FindWindow cannot meet the requirements. Non-standard controls have their own unique way of reality, you can go deep into their internal implementation, or get the support of the control developer, so that the control receives specific messages and returns corresponding information. However, this method is not universal enough. Every time a different non-standard control is processed, a specific method must be used, which reduces the development efficiency of system automation testing.

The graphical interface shown in Figure 1 is composed of a group of non-standard controls, and the positions of check boxes and buttons cannot be determined by conventional methods.

Since the implementation methods of non-standard controls are different, there is currently a need for a technical solution for selecting non-standard window controls in the Windows graphical interface, so as to achieve the purpose of automatic testing under the Windows system.

Contents of the invention

The technical problem to be solved by the present invention is to provide a method and system for obtaining interface control information, which solves the problem of how to select the required non-standard window control in the Windows graphical interface, and achieves the purpose of automatic testing under the Windows system.

In order to solve the above problems, the present invention provides a method for obtaining interface control information, including:

Selecting an area in the interface according to a preset strategy, and intercepting an image of the area, wherein the selected area is consistent with the previously obtained area containing the control to be detected;

According to the pixel combination of the image in the selected area, obtain the pixel features of the image in the selected area according to a preset strategy;

The information of the interface control in the selected area is determined according to the comparison result of the pixel feature of the image of the selected area with the pixel feature of the image of the area containing the control to be detected obtained in advance according to a preset strategy.

Further, the above method may also include, the step of obtaining the pixel features of the image of the region according to the preset strategy, including:

Determine the target pixel value according to the color of the control to be detected;

calculating the proportion of the target pixel value in the pixel combination of the selected region image.

Further, the above method may further include that the step of calculating the proportion of the target pixel value in the pixel combination of the selected region image includes:

The value of each pixel in the image of the selected area is stored in an array, the array is traversed, and the ratio of the target pixel value in the array to the total number of the array is calculated.

Further, the above method may further include, the step of determining the information of the interface controls in the selected area according to the comparison result, including:

Determine the proportion threshold according to the proportion of the target pixel value in the region image containing the control to be detected obtained in advance;

When the proportion of the target pixel value in the pixel combination of the selected area image reaches the threshold, it is determined that the control to be detected appears in the selected area.

Further, the above method may further include calculating distribution characteristics of the target pixel value in the selected region image;

According to the distribution feature, the position of the control to be detected in the selected area is determined.

Further, the above method may also include, the step of obtaining the pixel features of the image of the region according to the preset strategy, including:

A unique operation is performed on the data formed by combining the pixels of the image in the selected area to obtain a unique result.

Further, the above method may further include, the step of performing unique calculation on the data formed by combining the pixels of the selected area image, including:

The value of each pixel in the image of the selected area is stored in an array, and the data in the array is used as a binary data string for unique operation.

Further, the above method may further include, the step of determining the information of the interface controls in the selected area according to the comparison result, including:

According to the preset strategy, the data formed by the combination of pixels in the image of the area containing the control to be detected is obtained in advance, and the unique operation is performed, and the unique result is obtained;

When the uniqueness result corresponding to the image of the selected area is consistent with the uniqueness result corresponding to the image of the area containing the control to be detected, it is determined that the control to be detected appears in the selected area.

Further, the above method may further include, according to the position of the control obtained in advance in the area image containing the control to be detected, determining the position of the control in the image of the selected area.

Further, the above method may further include, if it is determined that no control to be detected appears in the currently selected area, shifting the currently selected area by one pixel row by row or column by row as a new currently selected area, and obtaining the selected control again. Pixel features of the region image.

Further, the above method may further include, the step of selecting an area in the interface according to the preset strategy, and intercepting the image of the area, including: judging that if the frequency of the control to be detected in a certain area reaches the preset threshold, select an area in the interface according to this area, and intercept the image of this area.

Further, the above method may further include that the step of determining the information of the interface controls in the selected area further includes: judging the acquired status information of the interface controls in the selected area.

Further, the above method may further include that the judging acquires status information of the interface controls in the selected area, including information on whether the check box is checked or whether the control is available.

The present invention also provides a system for acquiring interface control information, including:

The selection module is used to select an area in the interface according to a preset strategy, and intercept the image in the area, wherein the selected area is consistent with the area containing the control to be detected obtained in advance;

A feature acquisition module, configured to acquire pixel features of the image in the selected area according to a preset strategy according to the pixel combination of the image in the selected area;

The comparison module is used to determine the information of the interface controls in the selected area according to the comparison result of the pixel features of the image of the selected area and the pixel features of the image of the area containing the control to be detected obtained in advance according to a preset strategy.

Further, the above system may further include, the feature acquiring module, determining the target pixel value according to the color of the control to be detected, and calculating the ratio of the target pixel value in the pixel combination of the selected region image as the pixel features.

Further, the above-mentioned system may further include, the comparison module, configured to determine the difference between the ratio of the target pixel value in the region image containing the control to be detected and the ratio of the target pixel value in the selected region image. When it is within the preset range, it is determined that the control to be detected appears in the selected area.

Further, the above system may further include the feature acquisition module, configured to perform a unique operation on the data formed by combining pixels of the selected area image, and obtain a unique result as a pixel feature.

Further, the above-mentioned system may further include the comparison module, which is used to compare the unique operation result of the combination of pixels in the region image containing the control to be detected obtained in advance with the uniqueness of the combination of pixels in the image of the selected region. The consistency of the results of the linear operation, if consistent, it is determined that the control to be detected appears in the selected area.

Compared with the prior art, applying the present invention, by analyzing the pixel characteristics of the image displayed on the screen of the non-standard window control, and comparing it with the pixel characteristics of the area image containing the control to be detected obtained in advance according to the preset strategy, and then It can judge whether the required control appears in the selected area of the window, and can also obtain the status information of the control in the selected area, realizing the purpose of automatic testing of non-standard window controls through a unified method, reducing human work Intensity, has practical significance.

Description of drawings

Fig. 1 is a schematic diagram of a current graphical interface of a group of non-standard controls;

Fig. 2 is a schematic flow chart of the method for obtaining interface control information of the present invention;

Fig. 3 is a schematic flow chart of a "pixel density scanning method" according to an embodiment of the present invention;

Fig. 4 is a schematic diagram of a graphical interface characterized by a special color of the non-standard control of the present invention;

Fig. 5 is a schematic diagram of a graphical interface of a non-standard control with the same body color in the same area of the present invention;

Fig. 6 is a schematic diagram of the process of traversing to find the same area as the digital summary of the desired control in a larger area including the selected window area of the present invention;

Fig. 7 is a schematic diagram of a system for acquiring interface control information of the present invention.

detailed description

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

The method of this application is based on the screen coordinate system and the window coordinate system.

The following concepts are involved in this application:

The screen coordinate system is a two-dimensional coordinate system. The upper left corner of the screen is the origin of the coordinate system. The coordinates of the origin are marked as (0, 0). The abscissa axis coincides with the upper frame of the screen. The coordinate value increases from left to right. The vertical axis coincides with the right border of the screen, the coordinate value increases from top to bottom, and the coordinate unit is pixel.

The window coordinate system is similar to the screen coordinate system. The horizontal and vertical coordinates are parallel to the screen frame. The upper left corner of the window is the origin of the coordinate system. The coordinates of the origin are marked as (0, 0). The abscissa axis coincides with the upper border of the window, from left to right The coordinate value increases, the coordinate unit is pixel, the vertical axis coincides with the right border of the window, the coordinate value increases from top to bottom, and the coordinate unit is pixel. Obviously window coordinates can be converted to screen coordinates.

All the areas mentioned in this application are rectangular areas, and the areas mentioned below all refer to rectangular areas. A rectangular area can be described by the coordinates of two points in the window coordinate system, which are the upper left corner and the lower right corner of the area. Each area can be expressed as a quadruple of the shape (x1, y1, x2, y2), where (x1, y1) is the coordinates of the upper left corner, (x2, y2) is the coordinates of the lower right corner, and each area is set as The width in pixels is X, and the height is Y. These settings will be used in the pseudo-code below.

The selected area to be analyzed by the test system is recorded in the configuration file of the system. Because the position of the window on the screen is uncertain, the position of the control in the window coordinate system is recorded in the configuration file. When the test system is running, it is converted into actual screen coordinates.

MD5: The English full name is Message Digest Algorithm 5, which is defined as the fifth edition of the message digest algorithm in Chinese. The MD5 algorithm converts a piece of digital information through its irreversible string transformation algorithm to generate the unique MD5 information digest of this piece of information. Digests calculated by MD5 for different information cannot be the same. For details, please refer to the RFC 1321 protocol.

Control: A component of the Windows graphical interface that can accept input from input devices such as a keyboard or mouse to complete a specific function. Common controls in Windows systems include buttons, option buttons, edit boxes, list boxes, scroll bars, and static controls etc.

Pixel: The smallest component unit of an image in a computer. A pixel can be understood as the value of the color of a certain point in the image.

Since the implementation methods of non-standard window controls are different, it is impossible to start from the internal operation mechanism of the control. This application finds a new way to obtain the information in the control by analyzing the image displayed on the screen of the non-standard window control.

This application can obtain the information in the non-standard window control of the Windows graphical interface, and then can determine whether the required control appears in the selected rectangular area in the window; or determine and obtain the state information of the control in the selected area, including such as multi-selection Whether the box is checked or whether the control is available and so on.

As shown in Figure 2, a method for obtaining interface control information of the present invention includes the following steps:

Step 210, according to the preset strategy, select an area in the interface, and intercept the image of the area;

The selected area to be analyzed is recorded in the configuration file of the system. Because the position of the window on the screen is uncertain, what is recorded in the configuration file is the position of the control in the window coordinate system.

It specifically includes: selecting an area in the interface, converting the position information of the desired control in the window coordinate system recorded in the configuration file into the actual screen coordinate position information, and storing it.

The step of selecting an area in the interface according to the preset strategy and intercepting the image of the area specifically includes: judging that if the frequency of the control to be detected in a certain area reaches the preset threshold, then according to the area in the interface An area is selected, and an image of the area is intercepted, wherein the preset threshold is set as: the frequency of the control to be detected is 50 times or 100 times, etc., depending on the actual situation, and the present invention is not limited.

It is worth noting that the range of the selected area is consistent with the range of the previously obtained area containing the control to be detected, that is, the two areas may have the same area.

Step 220, according to the pixel combination of the image of the selected area, obtain the pixel features of the image of the selected area according to the preset strategy;

Wherein, the pixel feature of the selected area image may be the pixel feature of a certain position in the selected area image, or the pixel feature of the entire selected area image.

Specifically, the following steps may be included:

Determine the target pixel value according to the color of the control to be detected;

calculating the proportion of the target pixel value in the pixel combination of the selected region image.

Further includes:

calculating the distribution characteristics of the target pixel values in the selected region image;

According to the distribution feature, the position of the control to be detected in the selected area is determined.

When detecting whether a control appears, the specific position can also be detected, and the control position in the selected area image can be determined according to the control position in the area image containing the control to be detected obtained in advance.

Wherein, the step of calculating the ratio of the target pixel value to the pixel combination of the selected region image includes:

The value of each pixel in the image of the selected area is stored in an array, the array is traversed, and the ratio of the target pixel value in the array to the total number of the array is calculated.

Alternatively, the acquisition of the pixel features of the region image according to a preset strategy is to obtain a unique result by performing a unique operation on data formed by combining pixels of the selected region image.

Wherein, the step of performing a unique operation on the data formed by combining the pixels of the selected area image includes:

The value of each pixel in the image of the selected area is stored in an array, and the data in the array is used as a binary data string for unique operation.

Step 230: Determine the information of the interface controls in the selected area according to the comparison result of the pixel features of the image of the selected area and the pixel features of the image of the area containing the control to be detected obtained in advance according to a preset strategy.

The step of determining the information of the interface controls in the selected area according to the comparison result includes:

Determine the proportion threshold according to the proportion of the target pixel value in the region image containing the control to be detected obtained in advance;

When the proportion of the target pixel value in the pixel combination of the selected area image reaches the threshold, it is determined that the control to be detected appears in the selected area.

Or the step of determining the information of the interface controls in the selected area according to the comparison result includes:

According to the preset strategy, the data formed by the combination of pixels in the image of the area containing the control to be detected is obtained in advance, and the unique operation is performed, and the unique result is obtained;

When the uniqueness result corresponding to the image of the selected area is consistent with the uniqueness result corresponding to the image of the area containing the control to be detected, it is determined that the control to be detected appears in the selected area.

Can further include:

If it is determined that the control to be detected does not appear in the currently selected area, the currently selected area is shifted by one pixel row by row or column by row as a new currently selected area, and the pixel features of the image of the selected area are obtained again.

An embodiment of the method for obtaining interface control information of the present invention, as shown in Figure 3, includes the following steps:

Step 310, testing the selected window area of the system;

The selected area to be analyzed by the test system is recorded in the configuration file of the system. Because the position of the window on the screen is uncertain, the position of the control in the window coordinate system is recorded in the configuration file.

It specifically includes: the test system selects a window area, converts the position information of the required control in the window coordinate system recorded in the configuration file of the test system into the actual screen coordinate position information, and stores it.

Step 320, extracting the image of the selected window area, and analyzing the color value of each control pixel in the image;

The image in the area selected by the test system in the window is extracted by means of an image processing class library similar to the Python Image Library.

The extracted image is composed of pixels. Each pixel corresponds to a point in the image. The value of the pixel represents the color of the point. For example, yellow corresponds to one value, and red corresponds to another value. For details, refer to the RFC 797 protocol. A 32-bit An integer is used to record the value of the pixel, and the present invention saves these extracted pixels in an array for subsequent analysis.

Step 330 , judging whether a certain color value in a control pixel of the image reaches a preset ratio value, and if so, selecting the control as the required control.

The specific steps are as follows: the test system saves the color value of each pixel in the image of the selected window area in an array, traverses the array, and when the ratio of a certain color value in the array to the total number of the array reaches the preset ratio value , the control is determined to be the selected control.

In the actual test, if the test system needs to perform certain operations on a certain window control, it needs to determine whether the control exists in the area where the control should appear in the window of interest. If the feature of this control is a special color , as shown in Figure 4, the yellow color is the color of the "Agree and Install" button that is different from other controls. The test system should first capture the image of the area where it should appear, and save the color value of each pixel in the image in an array. The test system traverses the array, and when the yellow color value in the array accounts for a certain percentage of the total number of the array, it is considered that the "agree and install" control appears in this area, otherwise the button does not appear.

The process works as follows:

This method is called "pixel density scanning method" in the present invention, and its parameter rect is the area selected by the test system, featureColor refers to the main body color of the control that the test system pays attention to, and threshold is when the number of main color pixels accounts for the number of all pixels in the area What is the ratio of can determine the threshold of control existence.

Furthermore, the "pixel density scanning method" is limited in that it cannot distinguish controls that can appear in the same area and have the same body color. In order to solve this problem, the present invention proposes an implementation of the "MD5 summary judgment method".

As shown in Figure 5: the "Open" button and the "Close" button appear at the same position, and the black color of the text is also used as the identification color, so the "pixel density scanning method" at the same position cannot judge the current appearance. Is the close button or the open button.

The steps of "MD5 Summary Judgment Method" are as follows: If there are multiple controls that appear in the same area and have the same main body color, it is judged that if a certain color value in the multiple control pixels of the image reaches the preset ratio value, then it can be Identify the controls according to the digital summary of different controls, and select the required controls;

In view of the above situation, the control is identified according to the digital summary of different controls, and the required control is selected, specifically through the following steps:

After the test system obtains the pixel array of a certain control area in the selected window, the data in the array is used as a binary data string, and by calculating the obtained data string, the digital abstract of the data string is obtained, and different controls are identified.

Among them, the MD5 abstract of the image of the area where the control is located is mainly used as the identification of the control. After the test system obtains the pixel array of a certain area in the window, the data in the array is used as a binary data string, and the data string is processed by the MD5 abstract method. Calculation can get the digital summary of the data string. Since the images of different controls are different, the digital summary is also different, so they can be distinguished, the required control can be selected, and the corresponding operation control can be taken.

The digital summary of the image of the control in the selected area can be calculated in advance, and stored in the database of the test system as the identification summary of the control.

The test system compares the digital digest of the image of the selected area with the identification digest of the control, and if they match, the control is present in the selected area.

It is worth noting that the control controlled by the same operation may be displayed in multiple different appearances, so the identification summary of the control is not a single one, but can be a group of summaries.

Furthermore, the limitation of the above-mentioned "MD5 summary judgment method" is that when the position of the control changes in the selected area, even if the control does appear in the selected area, the digital summary of the image in this area cannot be compared with the digital content of the control. Digest matches. For this situation, the present invention provides a solution, that is, to find the same sub-area as the digital summary of the control in a larger area containing the selected area, as shown in Figure 6:

The method includes: the testing system determining that if the digital digest of the control image in the image of the selected window area does not match the digital digest of the desired control, then in a larger area containing the selected window area Traversing to find an area identical to the numeric summary of the desired control, if found, determining that the control in this area is the desired control.

Specific method: the test system expands the original selected window area to a certain extent, and the expanded area range can include the originally selected window area, starting from the upper left corner of the expanded area, sampling the same length and width as the originally selected window area Calculate the digital summary of the area, and judge if the digital summary of the sampling area is the same as the identification summary of the required control, then determine that the required control exists; otherwise, shift the sampling area to the left by one pixel, and determine if the right boundary of the sampling area has been aligned with the extended area coincide, the sampling area will be shifted down by one pixel until the bottom right corner of the sampling area coincides with the bottom right corner of the expansion area.

If the digital summary of the sampling area is the same as that of the control during this process, it is determined that the control exists, otherwise this process will continue until the bottom right corner of the sampling area coincides with the bottom right corner of the expansion area.

The process works as follows:

Among them, rect is the original selected area, and ctrlMd5 is the logo summary of the control.

"Pixel density scanning method" and "MD5 summary judgment method" can also judge the state of the control.

When it is determined that the control appears in the selected area, the state of the control is judged according to the MD5 summary of the area. For example, the current value of the drop-down list, whether the check box is selected, etc.

The images of a radio button before being selected are different from those after being selected. For example, the image before being selected is ○ZIP, and its MD5 summary is e1fb6b1310405f94cc92ce5ea2a8347d.

It should be noted that in the present invention, after analyzing the image pixels, various methods can be used to calculate the feature value of the image. "Pixel density scanning method" and "MD5 summary judgment method" are just two of the implementation methods; of course, other implementation methods can also be used to obtain interface control information, such as: calculation array HASH, edge pixel matching or pixel inversion and other implementation methods, The present invention is not limited thereto. The above methods are all through analyzing and comparing the features of the interface images to finally obtain the information of the interface controls.

As shown in Figure 7, the present invention also provides a system for acquiring interface control information, including: a selection module 701, a feature acquisition module 702 and a comparison module 703, wherein,

The selection module 701 is configured to select an area in the interface according to a preset strategy, and intercept an image in the area, wherein the selected area is consistent with the area containing the control to be detected obtained in advance;

A feature acquisition module 702, configured to acquire the pixel features of the image of the selected area according to the pixel combination of the image of the selected area according to a preset strategy;

The comparison module 703 is configured to determine the information of the interface controls in the selected area according to the comparison result of the pixel features of the selected area image and the pixel features of the area images containing the controls to be detected obtained in advance according to a preset strategy.

Wherein, the feature acquisition module 702 determines the target pixel value according to the color of the control to be detected, and calculates the proportion of the target pixel value in the pixel combination of the selected region image as the pixel feature.

The comparison module 703 is configured to determine when the difference between the proportion of the target pixel value in the area image containing the control to be detected and the proportion in the selected area image is within a preset range A control to be inspected appears within the selected area.

In this embodiment, the present invention describes the comparison process from another perspective.

The feature acquisition module 702 is configured to perform a unique operation on the data formed by combining pixels of the selected area image, and obtain a unique result as a pixel feature.

The comparison module 703 is used to compare the consistency between the unique operation result of the pixel combination in the previously obtained region image containing the control to be detected and the uniqueness operation result of the pixel combination in the selected region image, if If it is consistent, it is determined that the control to be detected appears in the selected area.

The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person familiar with the technology can easily think of changes or replacements within the technical scope disclosed in the present invention. , should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (21)

1. A method for obtaining interface control information, comprising: According to a first preset strategy, an area is selected in the interface, and an image of the area is intercepted, wherein the area of the selected area is the same as the area of the area including the control to be detected obtained in advance; According to the pixel combination of the image in the selected area, obtain the pixel features of the image in the selected area according to a second preset strategy; According to the comparison result of the pixel feature of the selected area image and the pixel feature of the area image containing the control to be detected obtained in advance according to the third preset strategy, determine the information of the interface control in the selected area; Wherein, the step of obtaining the pixel features of the selected region image according to the second preset strategy includes: Determine the target pixel value according to the color of the control to be detected; calculating the proportion of the target pixel value in the pixel combination of the selected region image as the pixel feature. 2. The method according to claim 1, wherein the step of calculating the ratio of the target pixel value to the pixel combination of the selected region image comprises: The value of each pixel in the image of the selected area is stored in an array, the array is traversed, and the ratio of the target pixel value in the array to the total number of the array is calculated. 3. The method according to claim 1, wherein the step of determining the information of the interface controls in the selected area according to the comparison result comprises: Determine the proportion threshold according to the proportion of the target pixel value in the region image containing the control to be detected obtained in advance; When the proportion of the target pixel value in the pixel combination of the selected area image reaches the threshold, it is determined that the control to be detected appears in the selected area. 4. The method of claim 1, further comprising: calculating the distribution characteristics of the target pixel values in the selected region image; According to the distribution feature, the position of the control to be detected in the selected area is determined. 5. The method of claim 3, further comprising: The position of the control in the image of the selected area is determined according to the position of the control obtained in advance in the image of the area containing the control to be detected. 6. The method of claim 3, further comprising: If it is determined that the control to be detected does not appear in the currently selected area, the currently selected area is shifted by one pixel row by row or column by row as a new currently selected area, and the pixel features of the image of the selected area are obtained again. 7. The method according to claim 1, wherein the step of selecting an area in the interface according to the first preset strategy and intercepting the image of the area comprises: judging that if the control to be detected is in the When the occurrence frequency of a certain area reaches the preset threshold, an area is selected in the interface according to the area, and an image of the area is intercepted. 8. The method according to claim 1, wherein the step of determining the information of the interface controls in the selected area further comprises: judging the acquired state information of the interface controls in the selected area. 9 . The method according to claim 8 , wherein the judging the acquired status information of the interface controls in the selected area comprises: information about whether the check box is checked or whether the control is available. 10. A method for obtaining interface control information, comprising: According to a first preset strategy, an area is selected in the interface, and an image of the area is intercepted, wherein the area of the selected area is the same as the area of the area including the control to be detected obtained in advance; According to the pixel combination of the image in the selected area, obtain the pixel features of the image in the selected area according to a second preset strategy; According to the comparison result of the pixel feature of the selected area image and the pixel feature of the area image containing the control to be detected obtained in advance according to the third preset strategy, determine the information of the interface control in the selected area; Wherein, the step of obtaining the pixel features of the selected region image according to the second preset strategy includes: A unique operation is performed on the data formed by combining the pixels of the image in the selected area to obtain a unique result. 11. The method according to claim 10, wherein the step of performing a unique operation on the data formed by combining the pixels of the selected area image comprises: The value of each pixel in the image of the selected area is stored in an array, and the data in the array is used as a binary data string for unique operation. 12. The method according to claim 10, wherein the step of determining the information of the interface controls in the selected area according to the comparison result comprises: According to the third preset strategy, a unique operation is performed on the data formed by the combination of pixels in the area image containing the control to be detected in advance, and a unique result is obtained; When the uniqueness result corresponding to the image of the selected area is consistent with the uniqueness result corresponding to the image of the area containing the control to be detected, it is determined that the control to be detected appears in the selected area. 13. The method of claim 12, further comprising: The position of the control in the image of the selected area is determined according to the position of the control obtained in advance in the image of the area containing the control to be detected. 14. The method of claim 12, further comprising: If it is determined that the control to be detected does not appear in the currently selected area, the currently selected area is shifted by one pixel row by row or column by row as a new currently selected area, and the pixel features of the image of the selected area are obtained again. 15. The method according to claim 10, wherein the step of selecting an area in the interface according to the first preset strategy and intercepting the image of the area comprises: judging that if the control to be detected is in the When the occurrence frequency of a certain area reaches the preset threshold, an area is selected in the interface according to the area, and an image of the area is intercepted. 16. The method according to claim 10, wherein the step of determining the information of the interface controls in the selected area further comprises: judging the acquired state information of the interface controls in the selected area. 17. The method according to claim 16, wherein the judging the acquired status information of the interface controls in the selected area comprises: information about whether the check box is checked or whether the control is available. 18. A system for obtaining interface control information, comprising: The selection module is configured to select an area in the interface according to a first preset strategy, and intercept an image in the area, wherein the area of the selected area is the same as the previously obtained area containing the control to be detected ; A feature acquisition module, configured to acquire the pixel features of the image of the selected area according to the second preset strategy according to the combination of pixels of the image of the selected area; A comparison module, configured to determine the information of the interface controls in the selected area according to the comparison result of the pixel features of the selected area image and the pixel features of the area image containing the control to be detected obtained in advance according to the third preset strategy ; Wherein, the feature acquisition module determines the target pixel value according to the color of the control to be detected; and calculates the proportion of the target pixel value in the pixel combination of the selected area image as the pixel feature. 19. The system according to claim 18, wherein the comparison module is configured to compare the ratio of the target pixel value to the image of the region containing the control to be detected to the ratio of the target pixel value to the image of the selected region When the difference of the ratio of is within the preset range, it is determined that the control to be detected appears in the selected area. 20. A system for acquiring interface control information, comprising: The selection module is configured to select an area in the interface according to a first preset strategy, and intercept an image in the area, wherein the area of the selected area is the same as the previously obtained area containing the control to be detected ; A feature acquisition module, configured to acquire the pixel features of the image of the selected area according to the second preset strategy according to the combination of pixels of the image of the selected area; A comparison module, configured to determine the information of the interface controls in the selected area according to the comparison result of the pixel features of the selected area image and the pixel features of the area image containing the control to be detected obtained in advance according to the third preset strategy ; Wherein, the feature acquisition module is configured to perform a unique operation on data formed by combining pixels of the selected region image, and obtain a unique result as a pixel feature. 21. The system according to claim 20, wherein the comparison module is used to compare the unique operation result of the combination of pixels in the region image containing the control to be detected obtained in advance with the selected region The consistency of the operation result of the uniqueness of the pixel combination of the image, if it is consistent, it is determined that the control to be detected appears in the selected area.