CN102164167A - Method and system for remote-control classified transmission - Google Patents

Abstract

The invention discloses a method and a system for remote-control classified transmission. The method comprises the following steps: acquiring an image of a window from a local side and dividing the image into a plurality of regions; identifying the elements contained in each region; and compressing the region in the corresponding compressing manner of the element with maximal area in each region and transmitting the compressed region to a network side. By using the method and system provided by the invention, the data volume of network transmission is efficiently reduced, the transmission rate is increased, and the data processing efficiency of the network side is increased.

Description

Classified transmission method and system for remote control

technical field

The invention belongs to the field of remote control, and in particular relates to a classified transmission method and system for remote control.

Background technique

The so-called remote control means that the management staff connects to the controlled computer in a different place through the Internet, displays the desktop environment of the controlled computer on their own computer, and configures, installs software, Modify and so on.

How remote control works: Remote control must be done through the network. The local computer is the sending end of the manipulation command, called the master control end, and the non-local controlled computer is called the controlled end. The master control end and the controlled end can be located in the same local area network, or two or more computers connected to the Internet at any location. Usually, the server-side program is installed in the computer of the master control end, and the client-side program is installed in the computer of the controlled end; when in use, the server-side program in the computer of the master control end sends a signal to the client program in the computer of the controlled end to establish a special Then through this remote service, use various remote control functions to send remote control commands to control the operation of various applications in the computer at the controlled end.

VNC (Virtual Network Computing virtual network computer) is a remote control technology, which is essentially a cross-platform display system, which transmits the complete window interface of the computer at the controlled end to the computer screen at the main control end through the network for display. When the operator uses the host computer to control the controlled computer, as if sitting in front of the screen of the controlled computer, the operator can start the application program of the controlled computer, use or steal the files and materials of the controlled computer, It is even possible to use the external printing device (printer) and communication device (modem or dedicated line, etc.) of the controlled computer to print and access the external network and intranet.

Because VNC is an abstract tool, it has no direct connection with the graphics system of the client machine. VNC technology captures the entire screen of the controlled end, and directly captures the full screen as a bitmap on the local side and transmits it to the master. As for the bitmap file captured on the local side, even if it is compressed, the file size will inevitably be large. When it is necessary to continuously capture and transmit the entire screen for multiple times, the amount of data to be transmitted will inevitably be very large. Large, the network transmission efficiency is low; and reappearing the screen on the network side host will consume more resources to process these large-sized bitmap files, and the processing efficiency is very low. This will directly cause the update speed of the screen displayed by the master control terminal to be very slow, the screen is severely lagged, and the refresh effect is not obvious, thus affecting the remote control of the controlled terminal by the master control terminal, and giving the user of the controlled terminal the feeling of remote control The response of the master control end is very insensitive, and there is a very large delay. When some applications with fast-changing graphics on the controlled end are opened, VNC is more difficult to cope with, and the delay will be more obvious.

Contents of the invention

In view of this, the technical problem to be solved by the present invention is to provide a classified transmission method and system for remote control, which can effectively reduce the amount of data transmitted by the network, increase the transmission rate, and also facilitate the improvement of data processing on the network side efficiency.

In order to solve the above-mentioned technical problems, the present invention discloses a classification transmission method for remote control, which includes: obtaining an image of a window on the local side and dividing it into several areas; identifying the elements contained in each area, using each A corresponding compression of the largest-area element in a region compresses said region and transmits it to the network side.

Further, for each area, the step of identifying the elements contained therein, using the corresponding compression method of the element with the largest area in each area to compress the area and transmitting it to the network side includes: for each area, identifying the elements contained in it If there is an element with the largest area and its proportion in the area exceeds a threshold, use a compression method corresponding to the type of the element to compress the area and transmit it to the network side.

Further, for each area, identify the elements contained therein, and if the proportion of the element in the area exceeds a threshold, use the compression method corresponding to the element category to compress the area and transmit it to the network side , including: if the proportion of the element in the area of the area does not exceed the threshold, divide the area into several second-level areas, and perform this step again for each second-level area.

Further, it also includes: transmitting the location information of each area to the network side; receiving and decompressing the images of each area on the network side, reorganizing each area according to the location information of each area, and converting the window reduction.

Further, the method includes: acquiring the image of the window at the local side according to the graphics device interface inside the system.

Further, the step of identifying elements included in each area includes: identifying elements in the area at the local side according to a graphics device interface inside the system, the elements including the background.

Further, the step of identifying the elements contained in each area includes: obtaining the color types contained in the area at the local side, identifying the elements in the area according to the number of color types, and the elements Include at least text and/or images.

Further, the step of identifying elements in the area according to the number of color types includes: setting a threshold, and when the number of color types is less than the threshold, identifying the element as a text; when the When the number of color types is greater than the threshold, the element is identified as a picture.

In order to solve the above technical problems, the present invention discloses a classification transmission system for remote control, including: a classification transmission device on the local side and a classification receiving device on the network side, wherein the classification transmission device is used to obtain An image of a form and divided into several regions, identifying the elements contained in it for each region, compressing the region using the corresponding compression method of the element with the largest area in each region and transmitting it together with the location information of the region To the classification receiving device; the classification receiving device is used to receive and decompress the images of each area, reorganize each area according to the location information of each area, and restore the window.

Further, the classifying transmission device is further used to identify the elements contained in each area, and if there is an element with the largest area and the proportion of the area of the area exceeds a threshold, use the element corresponding to the category of the element The compression method for compresses the region.

Further, the classifying and transmitting device is further configured to further divide the area into several areas when the proportion of the element in the area of the area does not exceed the threshold.

Further, the classifying transmission means is also used to obtain the window according to the graphics device interface inside the system.

Further, the classifying transmission means is further configured to identify elements in the area according to the graphics device interface inside the system, and the elements include the background.

Further, the classifying transmission device is further used to identify the color types included in the area, and identify elements in the area according to the number of color types, and the elements include at least text and/or pictures.

Further, the classification transmission device is also used to set a threshold, when the number of the color types is less than the threshold, identify the element as a text; when the number of the color types is greater than the threshold, Identify the element as an image.

Compared with existing solutions, the technical effects obtained by the present invention are:

1) After the form is divided into areas, the form is basically divided into blocks to form areas, and the core operation is to identify the main constituent elements of each area; for areas mainly composed of text, often carry data that needs to be recognized by the naked eye , so it is necessary to ensure the quality of data restoration, and the size of the text itself is also small, you can use a lossless compression method; for pictures that are mainly not rich in color, because their size is not large, you can use a lossless encoding compression method; for It is mainly composed of colorful pictures, which are often not key data for remote control, and a certain amount of distortion is allowed. Therefore, lossy encoding and compression methods, such as JPG, can be used to obtain a higher compression ratio and greatly reduce the size of the picture. Obtain a higher rate during transmission;

2) The present invention can reduce the size of the form by 30%-50% after such classification and compression, greatly reducing the amount of data, greatly improving the transmission rate, and also helping to improve the data processing efficiency on the network side.

Description of drawings

Fig. 1 is the method flowchart of the embodiment of the present invention;

Fig. 2 is the first schematic diagram of the application example of the present invention;

Fig. 3 is the second schematic diagram of the application example of the present invention

Fig. 4 is the 3rd schematic diagram of the application example of the present invention;

Fig. 5 is a schematic diagram of the system of the embodiment of the present invention.

Detailed ways

The implementation of the present invention will be described in detail below in conjunction with the drawings and examples, so that the realization process of how to use technical means to solve technical problems and achieve technical effects in the present invention can be fully understood and implemented accordingly.

The core idea of the present invention is: obtain an image of a window on the local side and divide it into several areas; Compress and transmit to the network side.

In order to obtain a better effect, it is possible to further identify the elements contained in each area, and if there is an element with the largest area and the proportion of the area of the area exceeds a threshold, use the compression method corresponding to the category of the element The area is compressed and transmitted to the network side.

The classification transmission method for remote control of the present invention will be described below with an embodiment. The user accepts the server's request and takes over the user's terminal for remote control. In the process of remote control, the server needs to capture the desktop of the user terminal in real time or periodically, and save it as a picture and send it to the server for display, so that it can be displayed on the server. restore the desktop of the user terminal on the server, so that maintenance personnel can control the user terminal on the server.

The classified transmission method for remote control of the present invention is applied to the network side taking over the local side during remote control, grabbing the terminal desktop of the local side or a part of the hotspot area of the terminal desktop required at a certain moment of remote control and For transmission to the network side, as shown in Figure 1, perform the following operations:

Step S1, obtain a form on the local side;

Acquire the form according to the GDI (Graphics Device Interface) inside the system at the local side, and what is acquired is the image of the form;

Step S2, dividing the form into several areas;

Divide the form into several areas in any manner, for example, the form can be divided into nine areas according to the nine grids, or it can be divided into sixteen grids, etc., the present invention is not limited thereto;

Step S3, identifying elements contained therein for each region;

The elements include at least: background, text, and pictures;

Identify the background in the elements in the area according to the graphics device interface GDI inside the system; for the background, it can be a picture with rich colors, or a picture with a single color;

Identify the types of colors contained in the area, and identify the text and/or pictures in the elements in the area according to the number of color types; generally speaking, the types of colors contained in pictures will be richer, and Compared with pictures, the color types contained in text are relatively single. Therefore, a threshold can be set for color types. When the number of color types is less than the threshold, the element is recognized as text; the number of color types is greater than During the threshold, identifying the element as a picture;

Of course, the picture can also be divided more carefully, and the first threshold and the second threshold can be set for the color type, the first threshold is greater than the second threshold; when the number of color types is smaller than the second threshold, the identified The element is text; when the number of the color types is greater than the first threshold, identify the element as a picture with rich colors; when the color type is between the first threshold and the second threshold, identify the element as not rich in color picture of;

Step S4, for each area, find the element with the largest area and determine the proportion of the element in the area of the area, if the proportion of the element in the area of the area exceeds a threshold, perform step S6, If the proportion of the element in the area of the region does not exceed a threshold, perform step S5;

Generally speaking, if the proportion of the element with the largest area in the area exceeds a preset threshold, it can be considered that the area is mainly composed of this element; if the proportion of the element in the area does not exceed the predetermined threshold If the threshold is higher than the above threshold, it means that the elements in the area are of various types and filled on average, and the area is not mainly composed of certain elements;

Step S5, divide the area into several second-level areas, and execute step S3 again for each second-level area to identify the elements contained therein;

Because the area is not mainly composed of certain elements, and different types of elements are suitable for different compression methods, so it is impossible to choose a suitable compression method for the area at this time; the area can be divided into several levels Areas can be divided into two-level areas in any way. For example, the form can be divided into nine areas according to nine grids, or sixteen grids, etc. The present invention is not limited thereto. If the main constituent elements cannot be determined in some secondary regions at this time, the secondary regions can also be divided again and repeated iterations are performed, the present invention is not limited thereto;

Step S6, using a compression method corresponding to the element category to compress the area and transmit it to the network side, and at the same time transmit the location information of the area to the network side;

Since the area is mainly composed of this element, it is most effective and suitable to use the corresponding compression method of this element for this area;

For the selection of compression methods, the following principles can be followed:

For an area mainly composed of text, text is the main object, and text often carries data that needs to be recognized by the naked eye, so it is necessary to ensure the quality of data restoration, and the size of the text itself is also small, and lossless compression can be used;

For the area mainly composed of pictures with less color, the picture with less color is the main object. Since the size of the picture itself is not large, lossless encoding and compression can be used;

For areas that are mainly composed of colorful pictures, the colorful pictures are the main objects. For remote control, such pictures are often not key data, and certain distortion is allowed. Therefore, lossy encoding compression methods, such as JPG, can be used. A higher compression ratio can be obtained, the image size can be greatly reduced, and a higher rate can be obtained during transmission.

Compared with directly grabbing the form for transmission, after such classification compression, the size of the form can be reduced by 30%-50%, which greatly reduces the amount of data and greatly improves the transmission rate;

Step S7, receiving and decompressing the images of each area on the network side, reorganizing each area on the network side according to the location information of each area, and finally restoring and displaying the window on the local side at this moment;

The location information may be the horizontal and vertical coordinates of each area on the original window image.

The method of the present invention will be described below with an application example.

User A accepts the request from server B and takes over the user's computer for remote control. In the process of remote control, server B needs to capture user A's computer desktop in real time or periodically and save it as a picture and send it to server B for further processing. display, so as to restore user A's computer desktop on server B, which is convenient for maintenance personnel to control user A's computer on server B.

At a certain moment of remote control, when server B grabs the desktop of user A’s computer and sends it to server B for display, it first obtains user A’s computer desktop according to the GDI (Graphics Device Interface) inside the system. The computer desktop is obtained in the form of a window; at this time, a window is opened on the user's desktop, and the content of a text file is displayed in the window.

As shown in Figure 2, it is the desktop of the computer used by user A. At this time, the picture of the computer desktop is divided into nine areas according to the nine-square grid, and the elements contained therein are identified for each area. Generally speaking, the elements include: background, text, picture. Taking area 1 as an example, it includes the desktop background and the display icons of shortcuts (the display icon can be identified as the background), and the elements in area 1 are identified as the background according to the method described in step S3 above. For area 1, the analysis shows that the picture with less color has the largest area and accounts for 100% of the entire area, exceeding the proportion threshold. At this time, it is considered that area 1 is mainly composed of pictures with less color, because these The color of the icon is not rich, and its size is not large. Area 1 can be compressed using a lossless encoding method and transmitted to server B. At the same time, the location coordinates of area 1 are also transmitted to server B. Taking area 4 again as an example, including the desktop background, the elements in area 4 are identified as the background according to the method described in step S3 above. For area 4, the analysis shows that the background area is the largest and accounts for 100% of the entire area, which also exceeds the proportion threshold. At this time, it is considered that area 4 is mainly composed of background, rich in color but not key data, and certain distortion is allowed , and it is likely to involve user privacy and does not need to be displayed, so the area 4 can be compressed using a lossy coding compression method and transmitted to server B, and the location coordinates of area 4 are also transmitted to server B. Taking area 7 again as an example, which includes desktop background and text, the elements in area 7 are identified as background and text according to the method described in step S3 above. For area 7, the analysis shows that the text part has the largest area and accounts for 70% of the entire area, which exceeds the proportion threshold. At this time, it is considered that area 7 is mainly composed of text, and the quality of data restoration needs to be guaranteed, so lossless can be used. The compression method compresses the area 7 and transmits it to the server B, and at the same time transmits the location coordinates of the area 7 to the server B. Other areas are also classified and compressed in a similar manner and then transmitted to server B, and the location coordinates are also transmitted to server B.

After server B decompresses the received images of each area, it reorganizes according to the position coordinates of each area, and finally displays the computer desktop of user A at this moment on the desktop of server B, so that it is convenient for maintenance personnel to In the displayed situation, the computer of user A is remotely controlled on server B.

The method of the present invention will be described below with an application example.

On the basis of the application example shown in Figure 2 above, as shown in Figure 3, it is found that pictures are also displayed in the window displaying text file content on the desktop, that is to say, area 7 contains desktop background, pictures and text, according to the above The method described in step S3 identifies the elements in area 7 as background, picture and text. For area 7, the analysis shows that the proportion of text, picture and background in the entire area does not exceed the proportion threshold. At this time, it is impossible to determine the main elements of area 7. Then, area 7 is divided into nine areas according to the nine-square grid. As shown in FIG. 4 , taking area 71 as an example, which includes desktop background, pictures and text, the elements in area 71 are recognized as background, picture and text according to the method described in step S3 above. For area 71, the analysis shows that the picture area is the largest and the proportion of the entire area exceeds the proportion threshold. At this time, it is considered that area 71 is mainly composed of pictures, rich in color but not key data, and certain distortion is allowed, so it can be used The lossy coding compression method compresses the area 71 and transmits it to the server B, and transmits the position coordinates of the area 71 to the server B at the same time. In the same manner, other areas 72-79 are also classified and compressed in a similar manner and then transmitted to server B, and the location coordinates are also transmitted to server B. In addition, areas 1-6 and areas 8 and 9 are classified and compressed as in the previous application example and then transmitted to server B, and the location coordinates are also transmitted to server B.

After server B decompresses the received data of areas 71-79, it reorganizes according to the location coordinates of each area to obtain area 7. After server B also decompresses the received images of areas 1-6 and areas 8 and 9, Finally, according to the position coordinates of areas 1-9, the computer desktop of user A at this moment is reorganized on the desktop of server B, so that it is convenient for maintenance personnel to perform maintenance on user A’s computer on server B according to the situation displayed on server B. remote control.

The classified transmission system for remote control of the present invention will be described below with an embodiment.

The classification transmission system for remote control is applied to the situation where the network side takes over the local side during remote control, captures the terminal desktop on the local side and transmits it to the network side, as shown in Figure 5, specifically including: The local side device A is the classification transmission device for remote control, and the network side device B is the classification reception device for remote control;

The local side device A is used to obtain an image of a window and divide it into several regions, identify the elements contained in each region, and use the element category when the proportion of the element with the largest area in the region exceeds a threshold Compressing the area in a corresponding compression manner and transmitting it to the network side device B together with the location information of the area;

The network side device B is used to receive and decompress the images of each area, reorganize each area according to the location information of each area, and restore the window.

Specifically, the local device A is further configured to further divide the area into several areas when the proportion of the element in the area does not exceed the threshold.

The local side device A is further configured to obtain the window according to the graphics device interface inside the system.

The local side device A is further configured to identify the elements in the area according to the graphics device interface inside the system, and the elements include the background.

The local device A is further configured to identify the color types contained in the area, and identify elements in the area according to the number of color types, and the elements include at least text and/or pictures.

The local side device A is also used to set a threshold. When the number of the color types is less than the threshold, the element is identified as text; when the number of the color types is greater than the threshold, the element is identified. for the picture.

The above-mentioned system and device correspond to the features of the above-mentioned classification transmission method for remote control, and for the shortcomings, please refer to the above-mentioned classification transmission method for remote control.

The above description shows and describes several preferred embodiments of the present invention, but as mentioned above, it should be understood that the present invention is not limited to the forms disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various Various other combinations, modifications, and environments can be made within the scope of the inventive concept described herein, by the above teachings or by skill or knowledge in the relevant field. However, changes and changes made by those skilled in the art do not depart from the spirit and scope of the present invention, and should all be within the protection scope of the appended claims of the present invention.

Claims (15)

1. a classification transmission method that is used for Long-distance Control is characterized in that, comprising:

Obtain the image of forms and be divided into several regions in local side;

At the element that each zone identification wherein comprises, use the corresponding compress mode of the element of area maximum in each zone that described zone is compressed and transfer to network side.

2. classification transmission method as claimed in claim 1, it is characterized in that, at the element that each zone identification wherein comprises, use the corresponding compress mode of the element of area maximum in each zone that the step of network side is compressed and transferred in described zone, further comprise:

At each zone, the element that wherein comprises of identification if exist the long-pending maximum and proportion that account for the area in described zone of monobasic vegetarian noodles to surpass a thresholding, uses and with the corresponding compress mode of classification of described element described zone is compressed and transferred to network side.

3. classification transmission method as claimed in claim 2, it is characterized in that, discern the element that wherein comprises at each zone, if accounting for the proportion in described zone, described element surpasses a thresholding, use with the corresponding compress mode of described element classification the step of network side compressed and transferred in described zone, further comprise:

Do not surpass described thresholding if described element accounts for the proportion of the area in described zone, described zone is divided into some level-2 areas again, carry out this step once more at each level-2 area.

4. classification transmission method as claimed in claim 1 is characterized in that, also comprises:

The positional information that each is regional transfers to network side;

Receive and each regional image that decompresses at described network side, recombinated in each zone, with described forms reduction according to described each regional positional information.

5. classification transmission method as claimed in claim 1 is characterized in that, obtains the step of the image of forms in local side, further comprises:

Obtain the image of described forms according to the GDI of internal system at described local lateral root.

6. classification transmission method as claimed in claim 1 is characterized in that, at each regional step of discerning the element that wherein comprises, further comprises:

Discern element in the described zone at described local lateral root according to the GDI of internal system, described element comprises background.

7. classification transmission method as claimed in claim 1 is characterized in that, at each regional step of discerning the element that wherein comprises, further comprises:

Obtain the color category that described zone comprises in described local side, discern element in the described zone according to the quantity of described color category, described element comprises literal and/or picture at least.

8. classification transmission method as claimed in claim 7 is characterized in that, discerns the step of the element in the described zone according to the quantity of described color category, further comprises:

Set a thresholding, when the quantity of described color category during less than described thresholding, discerning described element is literal; When the quantity of described color category during greater than described thresholding, discerning described element is picture.

9. a classification transmission system that is used for Long-distance Control is characterized in that, comprising: be positioned at the classification transmitting device and the classification receiving system that is positioned at network side of local side, wherein,

Described classification transmitting device, be used to obtain the image of forms and be divided into several regions, at the element that each zone identification wherein comprises, use the corresponding compress mode of the element of area maximum in each zone that described zone is compressed and transfer to described classification receiving system with the positional information in described zone;

Described classification receiving system is used to receive and each regional image that decompresses, and according to each regional positional information is recombinated in each zone, with described forms reduction.

10. classification transmission system as claimed in claim 9 is characterized in that,

Described classification transmitting device, be further used at each zone, the element that wherein comprises of identification if exist the long-pending maximum and proportion that account for the area in described zone of monobasic vegetarian noodles to surpass a thresholding, uses with the corresponding compress mode of classification of described element described zone is compressed.

11. classification transmission system as claimed in claim 10 is characterized in that,

Described classification transmitting device also is used for when the proportion that described element accounts for the area in described zone surpasses described thresholding described zone being divided into several regions again.

12. classification transmission system as claimed in claim 9 is characterized in that,

Described classification transmitting device also is used for obtaining described forms according to the GDI of internal system.

13. classification transmission system as claimed in claim 9 is characterized in that,

Described classification transmitting device also is used for discerning element in the described zone according to the GDI of internal system, and described element comprises background.

14. classification transmission system as claimed in claim 9 is characterized in that,

Described classification transmitting device, the color category that also is used for described zone is comprised is discerned, and discerns element in the described zone according to the quantity of described color category, and described element comprises literal and/or picture at least.

15. classification transmission system as claimed in claim 14 is characterized in that,

Described classification transmitting device also is used to set a thresholding, and when the quantity of described color category during less than described thresholding, discerning described element is literal; When the quantity of described color category during greater than described thresholding, discerning described element is picture.

Images