CN117544741A

CN117544741A - Image processing method, device, equipment, medium and system

Info

Publication number: CN117544741A
Application number: CN202311494777.3A
Authority: CN
Inventors: 丁月; 宋琪; 周二龙
Original assignee: Suzhou Metabrain Intelligent Technology Co Ltd
Current assignee: Suzhou Metabrain Intelligent Technology Co Ltd
Priority date: 2023-11-10
Filing date: 2023-11-10
Publication date: 2024-02-09

Abstract

The invention discloses an image processing method, an image processing device, an image processing medium and an image processing system in the technical field of computers. When the invention meets the preset frame supplementing condition, the frame supplementing is not immediately carried out, but the frame supplementing efficiency can be improved by matching local pixels on the premise of allowing the real-time computing performance of the terminal where the client is located, the frame supplementing of the display picture of the server desktop can be carried out by adapting the terminal resource, the picture blocking phenomenon caused by the frame supplementing is avoided, and the resource occupation balance between the frame supplementing operation and other tasks on the terminal is realized.

Description

Image processing method, device, equipment, medium and system

Technical Field

The present invention relates to the field of computer technologies, and in particular, to an image processing method, apparatus, device, medium, and system.

Background

The BMC (Baseboard Management Controller ) is a dedicated controller for monitoring and managing servers. Typically, the BMC may capture a picture of the server desktop and then transmit it to the remote client so that the user of the remote client operates or manages the server. In the actual use process, the BMC can lose frames of the captured pictures of the server desktop due to insufficient video memory bandwidth or insufficient network bandwidth, so that the pictures transmitted to the remote client by the BMC are lost, and the display frame rate of the client is reduced. When the frame rate is reduced below 24 frames of the human eye resolution frame number, the blocking of the display picture can be obviously perceived.

Although the deep learning model can be adopted to supplement frames at the client at present, the deep learning model needs to run depending on a large amount of resources, and the terminal where the client is located has fewer computing resources and is difficult to support the running of the deep learning model, so that the deep learning model is not suitable for being applied to terminal integration, and terminal resource occupation and waste can be caused.

Therefore, how to adapt to the terminal resource to perform frame compensation of the display screen of the server desktop is a problem that needs to be solved by those skilled in the art.

Disclosure of Invention

In view of the above, the present invention aims to provide an image processing method, apparatus, device, medium and system for supplementing frames of a display screen of a server desktop by adapting terminal resources. The specific scheme is as follows:

in a first aspect, the present invention provides an image processing method applied to a client of a baseboard management controller, including:

receiving a current picture of a server desktop sent by the baseboard management controller;

if the preset frame supplementing condition is met currently, calculating pixel change information of the current picture and the received previous picture when equipment where the client is located currently has resources required for frame supplementing; wherein, the resources required by the frame compensation are as follows: the resources required by frame supplementing are carried out on the target picture which is missing between the current picture and the previous picture;

If the pixel change information is larger than a preset change threshold, determining a local picture area according to the picture difference of the current picture and the previous picture, restricting the pixel matching operation of the current picture and the previous picture to the local picture area, and carrying out frame supplementing operation on the target picture according to the pixel matching result of the current picture and the previous picture.

Optionally, the preset frame compensation condition includes:

the difference value between the receiving time stamp of the current picture and the receiving time stamp of the previous picture is larger than a preset timeout time; or the total receiving time length of the current picture is longer than the preset receiving time length; or the total number of pictures received in the preset period is smaller than a preset fixed value; or the number of the target pictures is smaller than a preset deletion threshold value.

Optionally, the method further comprises:

if the preset frame supplementing condition is not met currently, storing the current picture into a buffer area;

and reading the current picture from the buffer area and displaying the current picture.

Optionally, not satisfying the preset frame compensation condition includes:

the difference value between the receiving time stamp of the current picture and the receiving time stamp of the previous picture is not more than a preset timeout time; or the total receiving time length of the current picture is not more than the preset receiving time length; or the total number of the pictures received in the preset period is not smaller than a preset fixed value; or the target number of the target pictures is not smaller than a preset deletion threshold value.

Optionally, the device currently has resources required for the complementary frame, including:

acquiring idle equipment resources of real-time statistics of a target process running on the equipment;

if the idle resources of the equipment are not smaller than the resources required by the frame supplement, determining that the equipment currently has the resources required by the frame supplement;

or (b)

Calculating the resource occupation amount of a task currently operated by the equipment;

determining the difference value between the rated resource quantity of the equipment and the resource occupation quantity as equipment idle resources;

and if the idle resources of the equipment are not smaller than the resources required by the frame supplement, determining that the equipment currently has the resources required by the frame supplement.

Optionally, the idle resources of the device are not smaller than the resources required by the complementary frame, including:

inquiring the frame supplementing configuration information to obtain the maximum frame supplementing number applicable to the equipment;

and if the number of the complementary frames is not greater than a preset complementary frame threshold, determining that the idle resources of the equipment are not less than the resources required by the complementary frames.

Optionally, if the number of complementary frames is greater than the preset complementary frame threshold, determining that the idle resources of the device are smaller than the resources required by the complementary frames.

Optionally, the method further comprises:

when the idle resources of the equipment are smaller than the resources required by the frame filling, skipping frame filling operation, and storing the current picture into a buffer area;

Optionally, the determining process of the resources required by the complementary frame includes:

determining a target number of the target pictures;

inquiring frame supplementing configuration information to obtain the resource usage amount required by the equipment for carrying out frame supplementing operation on the target quantity;

and determining the resource usage amount as the resource required by the complementary frame.

Optionally, the calculating pixel change information of the current picture and the received previous picture includes:

calculating the total current pixel amount of the current picture;

calculating the total amount of the previous pixels of the previous picture;

and determining the difference value between the current pixel total and the previous pixel total as the pixel change information.

Optionally, the method further comprises:

if the pixel change information is not greater than the preset change threshold value, determining the target number of the target pictures;

obtaining a target number of target pictures according to the current picture and the previous picture;

storing the current picture and the target number of target pictures into a buffer memory area;

and reading the target number of target pictures and the current picture from the cache area, and displaying the target number of target pictures and the current picture in sequence.

Optionally, the obtaining a target number of target pictures according to the current picture and the previous picture includes:

copying the current picture for the target number of times to obtain a target number of target pictures;

or (b)

Copying the previous picture for the target number of times to obtain a target number of target pictures;

or (b)

And calculating pixel mean values of the current picture and the previous picture, and generating a target number of target pictures based on the pixel mean values.

Optionally, the determining the local picture area according to the picture difference between the current picture and the previous picture includes:

determining edge points of pixels which change by comparing the current picture with the previous picture;

and determining the minimum rectangle covering the area surrounded by all edge points as the local picture area.

Optionally, the constraining the pixel matching operation of the current picture and the previous picture to the local picture region includes:

intercepting the current picture and the previous picture by utilizing the local picture area to obtain a current intercepting surface of the current picture and a previous intercepting surface of the previous picture;

performing pixel matching operation on the current cut-out surface and the previous cut-out surface by using a block matching algorithm to determine the same pixel area positioned at different positions of the current picture and the previous picture;

And recording the current position information of the same pixel area in the current picture and the previous position information of the same pixel area in the previous picture to obtain the pixel matching result.

Optionally, the performing a frame-filling operation on the target picture according to the pixel matching result of the current picture and the previous picture includes:

determining a previous coordinate position of a target pixel in the previous position information;

determining a current coordinate position of the target pixel in the current position information;

determining a displacement vector from the previous coordinate position to the current coordinate position;

determining the moving direction of the same pixel area according to the direction of the displacement vector;

and carrying out frame supplementing operation on the target picture in the moving direction.

Optionally, the frame-supplementing operation is performed on the target picture in the moving direction, including:

determining the positions of the same pixel areas in the target frames in the target number according to the target number of the target frames, the previous coordinate position and the current coordinate position;

filling pixels at positions corresponding to the current picture or the previous picture in other positions except the current same pixel region aiming at the same pixel region at each position to obtain a frame of target picture corresponding to the same pixel region at each position;

Optionally, after the sequentially displaying the target number of target frames and the current frame, the method further includes:

and deleting the target number of target pictures and the current picture from the cache area.

Optionally, if the pixel change information is greater than a preset change threshold, detecting the current picture and the previous picture by using a preset neural network model to obtain an interested region in the current picture and an interested region in the previous picture;

and carrying out frame supplementing operation on the target picture based on the region of interest in the current picture and the region of interest in the previous picture.

In a second aspect, the present invention provides an image processing apparatus applied to a client of a baseboard management controller, including:

the receiving module is used for receiving the current picture of the server desktop sent by the baseboard management controller;

the detection module is used for calculating pixel change information of the current picture and the received previous picture when the equipment where the client is located currently has resources required for frame filling if the preset frame filling condition is met; wherein, the resources required by the frame compensation are as follows: the resources required by frame supplementing are carried out on the target picture which is missing between the current picture and the previous picture;

And the frame supplementing module is used for determining a local picture area according to the picture difference of the current picture and the previous picture if the pixel change information is larger than a preset change threshold value, restricting the pixel matching operation of the current picture and the previous picture to the local picture area, and carrying out frame supplementing operation on the target picture according to the pixel matching result of the current picture and the previous picture.

Optionally, the preset frame compensation condition includes: the difference value between the receiving time stamp of the current picture and the receiving time stamp of the previous picture is larger than a preset timeout time; or the total receiving time length of the current picture is longer than the preset receiving time length; or the total number of pictures received in the preset period is smaller than a preset fixed value; or the number of the target pictures is smaller than a preset deletion threshold value.

Optionally, the method further comprises: the display module is used for storing the current picture into a buffer area if the preset frame supplementing condition is not met currently; and reading the current picture from the buffer area and displaying the current picture.

Optionally, not satisfying the preset frame compensation condition includes: the difference value between the receiving time stamp of the current picture and the receiving time stamp of the previous picture is not more than a preset timeout time; or the total receiving time length of the current picture is not more than the preset receiving time length; or the total number of the pictures received in the preset period is not smaller than a preset fixed value; or the target number of the target pictures is not smaller than a preset deletion threshold value.

Optionally, the detection module is specifically configured to: acquiring idle equipment resources of real-time statistics of a target process running on the equipment; if the idle resources of the equipment are not smaller than the resources required by the frame supplement, determining that the equipment currently has the resources required by the frame supplement; or calculating the resource occupation amount of the task currently operated by the equipment; determining the difference value between the rated resource quantity of the equipment and the resource occupation quantity as equipment idle resources; and if the idle resources of the equipment are not smaller than the resources required by the frame supplement, determining that the equipment currently has the resources required by the frame supplement.

Optionally, the detection module is specifically configured to: inquiring the frame supplementing configuration information to obtain the maximum frame supplementing number applicable to the equipment; and if the number of the complementary frames is not greater than a preset complementary frame threshold, determining that the idle resources of the equipment are not less than the resources required by the complementary frames. And if the number of the complementary frames is larger than the preset complementary frame threshold value, determining that the idle resources of the equipment are smaller than the resources required by the complementary frames.

Optionally, the display module is further configured to: when the idle resources of the equipment are smaller than the resources required by the frame filling, skipping frame filling operation, and storing the current picture into a buffer area; and reading the current picture from the buffer area and displaying the current picture.

Optionally, the determining process of the resources required by the complementary frame includes: determining a target number of the target pictures; inquiring frame supplementing configuration information to obtain the resource usage amount required by the equipment for carrying out frame supplementing operation on the target quantity; and determining the resource usage amount as the resource required by the complementary frame.

Optionally, the detection module is specifically configured to: calculating the total current pixel amount of the current picture; calculating the total amount of the previous pixels of the previous picture; and determining the difference value between the current pixel total and the previous pixel total as the pixel change information.

Optionally, the frame compensation module is further configured to: if the pixel change information is not greater than the preset change threshold value, determining the target number of the target pictures; obtaining a target number of target pictures according to the current picture and the previous picture; storing the current picture and the target number of target pictures into a buffer memory area; and reading the target number of target pictures and the current picture from the cache area, and displaying the target number of target pictures and the current picture in sequence.

Optionally, the frame compensation module is further configured to: copying the current picture for the target number of times to obtain a target number of target pictures; or copying the previous picture for the target number of times to obtain a target number of target pictures; or calculating the pixel mean value of the current picture and the previous picture, and generating a target number of target pictures based on the pixel mean value.

Optionally, the frame supplementing module is specifically configured to: determining edge points of pixels which change by comparing the current picture with the previous picture; and determining the minimum rectangle covering the area surrounded by all edge points as the local picture area.

Optionally, the frame supplementing module is specifically configured to: intercepting the current picture and the previous picture by utilizing the local picture area to obtain a current intercepting surface of the current picture and a previous intercepting surface of the previous picture; performing pixel matching operation on the current cut-out surface and the previous cut-out surface by using a block matching algorithm to determine the same pixel area positioned at different positions of the current picture and the previous picture; and recording the current position information of the same pixel area in the current picture and the previous position information of the same pixel area in the previous picture to obtain the pixel matching result.

Optionally, the frame supplementing module is specifically configured to: determining a previous coordinate position of a target pixel in the previous position information; determining a current coordinate position of the target pixel in the current position information; determining a displacement vector from the previous coordinate position to the current coordinate position; determining the moving direction of the same pixel area according to the direction of the displacement vector; and carrying out frame supplementing operation on the target picture in the moving direction.

Optionally, the frame supplementing module is specifically configured to: determining the positions of the same pixel areas in the target frames in the target number according to the target number of the target frames, the previous coordinate position and the current coordinate position; and filling pixels at positions corresponding to the current picture or the previous picture in other positions except the current same pixel area aiming at the same pixel area at each position to obtain a frame of target picture corresponding to the same pixel area at each position. The display module is also used for: storing the current picture and the target number of target pictures into a buffer memory area; and reading the target number of target pictures and the current picture from the cache area, and displaying the target number of target pictures and the current picture in sequence.

Optionally, the method further comprises: and the cache cleaning module is used for deleting the target number of target pictures and the current picture from the cache area.

In a third aspect, the present invention provides an electronic device, comprising:

a memory for storing a computer program;

and a processor for executing the computer program to implement the image processing method disclosed above.

In a fourth aspect, the present invention provides a readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the image processing method disclosed above.

In a fifth aspect, the present invention provides a system comprising: the system comprises a server, a baseboard management controller connected with the server and terminal equipment, wherein the terminal equipment is provided with a client of the baseboard management controller, and the client is provided with the image processing method disclosed by the prior art. The client is a kernel-based virtual machine client.

As can be seen from the above solution, the present invention provides an image processing method applied to a client of a baseboard management controller, including: receiving a current picture of a server desktop sent by the baseboard management controller; if the preset frame supplementing condition is met currently, calculating pixel change information of the current picture and the received previous picture when equipment where the client is located currently has resources required for frame supplementing; wherein, the resources required by the frame compensation are as follows: the resources required by frame supplementing are carried out on the target picture which is missing between the current picture and the previous picture; if the pixel change information is larger than a preset change threshold, determining a local picture area according to the picture difference of the current picture and the previous picture, restricting the pixel matching operation of the current picture and the previous picture to the local picture area, and carrying out frame supplementing operation on the target picture according to the pixel matching result of the current picture and the previous picture.

The beneficial effects of the invention are as follows: when the preset frame supplementing condition is met, the frame supplementing is not immediately carried out, but is carried out on the premise that the real-time computing performance of the terminal where the client is located allows, namely: when the equipment where the client is located currently has resources required for frame compensation, calculating pixel change information of a current picture and a received previous picture; if the pixel change information is larger than the preset change threshold value, determining a local picture area according to the picture difference of the current picture and the previous picture, so that the pixel matching operation of the current picture and the previous picture is limited to the local picture area, and performing frame supplementing operation on the target picture according to the pixel matching result of the current picture and the previous picture, thereby completing the frame supplementing operation by matching local pixels. Because the content generally displayed on the desktop of the server is simple, the pixel matching efficiency can be improved by matching local pixels, so that the frame supplementing efficiency is improved. According to the scheme, on the premise that the real-time computing performance of the terminal where the client is located allows, the frame supplementing operation is performed by matching local pixels, so that the terminal resources can be adapted to supplement frames of display pictures of the server desktop, the phenomena of picture blocking and the like caused by frame supplementing can be avoided, and the resource occupation balance between the frame supplementing operation and other tasks on the terminal is realized as much as possible.

It should be noted that, if the frame is directly complemented without considering the computing performance of the terminal, it may happen that: because a large amount of resources of the terminal are occupied by other tasks and frame compensation is difficult to carry out, the phenomenon of picture blocking and the like caused by frame compensation operation of a client side picture can directly influence the normal operation of the client side.

Correspondingly, the image processing device, the medium and the system provided by the invention also have the technical effects.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of an image processing method disclosed by the invention;

FIG. 2 is a schematic diagram illustrating a comparison of a current frame and a previous frame according to the present disclosure;

FIG. 3 is a schematic view of a displacement vector according to the present disclosure;

FIG. 4 is a schematic diagram of a target frame according to the present disclosure;

FIG. 5 is a schematic diagram of a frame-filling process according to the present disclosure;

FIG. 6 is a schematic diagram of a frame filling operation according to the present disclosure;

FIG. 7 is a schematic diagram of a system architecture according to the present disclosure;

FIG. 8 is a schematic diagram of an image processing apparatus according to the present disclosure;

FIG. 9 is a schematic diagram of an electronic device according to the present disclosure;

FIG. 10 is a diagram illustrating a server configuration according to the present invention;

fig. 11 is a diagram of a terminal structure according to the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

At present, although the frame can be supplemented at the client by adopting models such as deep learning, the deep learning model needs to run by depending on a large amount of resources, and the terminal where the client is located has fewer computing resources and is difficult to support the running of the deep learning model, so that the models such as deep learning are not suitable for being applied to terminal integration, and terminal resource occupation and waste can be caused. Therefore, the invention provides an image processing scheme which can adapt to terminal resources to supplement frames of display pictures of a server desktop.

Referring to fig. 1, an embodiment of the present invention discloses an image processing method applied to a client of a baseboard management controller, including:

s101, receiving a current picture of a server desktop sent by a baseboard management controller.

In this embodiment, the server desktop is similar to a computer desktop, in which relevant operation icons of the server are displayed.

S102, if the preset frame supplementing condition is met currently, calculating pixel change information of a current picture and a received previous picture when equipment where the client is located currently has resources required for frame supplementing.

In one embodiment, the preset frame-supplementing condition includes: the difference value between the receiving time stamp of the current picture and the receiving time stamp of the previous picture is larger than the preset timeout time; or the total receiving time length of the current picture is longer than the preset receiving time length; or the total number of pictures received in the preset period is smaller than a preset fixed value; or the number of target pictures is less than a preset deletion threshold. The above several frame-filling conditions may be combined as desired. If the number of the target pictures is smaller than the preset deletion threshold value, the method shows that: the number of the missing frames is too large, and the characteristic of small resources of the terminal is considered, so that the frame supplement can be directly abandoned, and other problems caused by too many frames can be avoided. Accordingly, failing to satisfy the preset frame complement condition includes: the difference value between the receiving time stamp of the current picture and the receiving time stamp of the previous picture is not more than the preset timeout time; or the total receiving time length of the current picture is not more than the preset receiving time length; or the total number of the pictures received in the preset period is not smaller than a preset fixed value; or the target number of the target pictures is not less than a preset deletion threshold.

In this embodiment, further comprising: if the current frame supplementing condition is not met, storing the current picture into a buffer memory area; and reading the current picture from the buffer area and displaying the current picture. That is: when the preset frame compensation condition is not satisfied, the current picture is directly displayed without frame compensation.

In one embodiment, the device currently has resources required for frame replenishment, comprising: acquiring idle equipment resources of real-time statistics of a target process running on equipment; if the idle resources of the equipment are not less than the resources required by the frame filling, determining that the equipment currently has the resources required by the frame filling; or calculating the resource occupation amount of the task currently operated by the equipment; determining the difference value between the rated resource quantity and the resource occupation quantity of the equipment as the idle resource of the equipment; if the idle resources of the equipment are not smaller than the resources required by the frame supplement, determining that the equipment currently has the resources required by the frame supplement. The device free resources include: CPU (Central Processing Unit ) idle resources, GPU (Graphics Processing Unit, graphics processor) idle resources, and memory space left, etc.

Further, the idle resources of the device are not smaller than the resources required by the frame complement, including: inquiring the frame supplementing configuration information to obtain the maximum frame supplementing number applicable to the equipment; if the number of the complementary frames is not greater than the preset complementary frame threshold, determining that the idle resources of the equipment are not less than the resources required by the complementary frames. The maximum number of frames to be complemented for each type of device may be recorded in advance in the frame complementing configuration information, and then after determining the number of frames to be complemented, it may be determined whether the current device is suitable for executing the frame complementing operation by comparing the number of frames to be complemented with the maximum number of frames to be complemented. In one embodiment, if the number of frames to be supplemented is greater than a predetermined frame supplementing threshold, it is determined that the idle resources of the device are less than the resources required for frame supplementing.

In one embodiment, when the idle resources of the device are smaller than the resources required by the frame filling, the frame filling operation is skipped, and the current picture is stored in the buffer area; and reading the current picture from the buffer area and displaying the current picture. That is: when the device has less idle resources, the current picture is directly displayed without frame compensation.

The process for determining the resources required by the frame compensation comprises the following steps: determining the target number of target pictures; inquiring the frame supplementing configuration information to obtain the resource usage amount required by the equipment for carrying out frame supplementing operation on the target quantity; and determining the resource usage as the resource required by the complementary frame. Wherein, the resources required by the frame filling are as follows: and the resources required for frame compensation are carried out on the target picture which is missing between the current picture and the previous picture. The frame-supplementing configuration information can also record the resource usage amount required by N frames of frame-supplementing operation of the current equipment in the past, wherein the value of N is a natural number such as 1, 2, 3 and the like. Such as: the current device supplements the resource usage needed by the display picture of the server desktop with one frame, two frames, three frames or four frames, so that the resource usage needed by supplementing the target number of frames can be obtained by inquiring the configuration information of the supplemented frames.

In one embodiment, calculating pixel change information for a current picture and a received previous picture includes: calculating the total current pixel of the current picture; calculating the total amount of the previous pixels of the previous picture; the difference between the current pixel total and the previous pixel total is determined as pixel change information.

And S103, if the pixel change information is larger than a preset change threshold, determining a local picture area according to the picture difference of the current picture and the previous picture, restricting the pixel matching operation of the current picture and the previous picture to the local picture area, and carrying out frame supplementing operation on the target picture according to the pixel matching result of the current picture and the previous picture.

It should be noted that, when the pixel change information is greater than the preset change threshold, it indicates that: the picture difference between the current picture and the previous picture is larger; when the pixel change information is not greater than the preset change threshold value, the method shows that: if the difference between the current picture and the previous picture is small, in one embodiment, if the pixel change information is not greater than the preset change threshold, determining the target number of the target pictures; obtaining a target number of target pictures according to the current picture and the previous picture; storing the current picture and the target number of target pictures into a buffer memory area; and reading the target number of target pictures and the current picture from the buffer memory area, and displaying the target number of target pictures and the current picture in sequence. The method for obtaining the target number of target pictures according to the current picture and the previous picture comprises the following steps: copying the current picture for the target number of times to obtain target number of target pictures; or copying the previous picture for the target number of times to obtain a target number of target pictures; or calculating the pixel mean value of the current picture and the previous picture, and generating a target number of target pictures based on the pixel mean value.

In one embodiment, determining a local picture region from a picture difference of a current picture and a previous picture includes: determining edge points where pixels change by comparing a current picture with a previous picture; the smallest rectangle covering the area surrounded by all edge points is determined as a local picture area. As shown in fig. 2, the current screen and the previous screen are both displayed with window a, except that the position of window a in the two screens is different. Window a is within region S, which is shown in fig. 2 as: the smallest rectangle covering the area enclosed by all edge points is also the local picture area. The region O is the entire screen display region including the region S. Thus, pixel matching need only be performed for region S, and not for region O.

In one embodiment, restricting the pixel matching operation of the current picture and the previous picture to the local picture region includes: intercepting a current picture and a previous picture by utilizing a local picture area to obtain a current intercepting surface (a part corresponding to the local picture area in the current picture) of the current picture and a previous intercepting surface (a part corresponding to the local picture area in the previous picture) of the previous picture; performing pixel matching operation on the current cut plane and the previous cut plane by using a block matching algorithm to determine the same pixel area (such as a window A in FIG. 2) positioned at different positions of the current picture and the previous picture; and recording the current position information of the same pixel area in the current picture and the previous position information of the same pixel area in the previous picture to obtain a pixel matching result. It can be seen that the window a in the current picture illustrated in fig. 2 and the window a in the previous picture illustrated in fig. 2 can be determined by a block matching algorithm.

Further, performing a frame-filling operation on the target frame according to the pixel matching result of the current frame and the previous frame, including: determining a previous coordinate position of the target pixel in the previous position information; determining a current coordinate position of the target pixel in the current position information; determining a displacement vector from a previous coordinate position to a current coordinate position; determining the moving direction of the same pixel area according to the direction of the displacement vector; and performing frame supplementing operation on the target picture in the moving direction. Referring to the current picture and the previous picture provided in fig. 2, the displacement vector and its direction are shown in fig. 3. The previous coordinate position and the current coordinate position are positions of the same pixel in two pictures. Of course, the displacement vector may be determined from the lower left vertex of the same pixel region in the previous frame (A1 shown in fig. 6) to the upper right vertex of the same pixel region in the previous frame (A2 shown in fig. 6).

In this embodiment, performing a frame-filling operation on a target screen in a moving direction includes: determining the positions of the same pixel areas in the target frames in the target number according to the target number of the target frames, the previous coordinate position and the current coordinate position; filling pixels at corresponding positions of a current picture or a previous picture in other positions except the current same pixel region aiming at the same pixel region at each position to obtain a frame of target picture corresponding to the same pixel region at each position; storing the current picture and the target number of target pictures into a buffer memory area; and reading the target number of target pictures and the current picture from the buffer memory area, and displaying the target number of target pictures and the current picture in sequence. Referring to the current picture and the previous picture provided in fig. 2, when the target number value is 1, 1 position needs to be determined for the window a, so that the window a is fixed at one position in the picture and pixels at other positions are filled to restore the whole picture, and a frame of target picture can be obtained. Accordingly, 1 frame (i.e., 1) of the target picture that needs to be supplemented according to the current picture and the previous picture provided in fig. 2 is shown in fig. 4.

In one embodiment, after sequentially displaying the target number of target frames and the current frame, the method further includes: and deleting the target number of target pictures and the current picture from the cache area.

As can be seen, when the preset frame-supplementing condition is satisfied, the embodiment does not immediately supplement the frame, but rather supplements the frame on the premise that the real-time computing performance of the terminal where the client is located allows, that is: when the equipment where the client is located currently has resources required for frame compensation, calculating pixel change information of a current picture and a received previous picture; if the pixel change information is larger than the preset change threshold value, determining a local picture area according to the picture difference of the current picture and the previous picture, so that the pixel matching operation of the current picture and the previous picture is limited to the local picture area, and performing frame supplementing operation on the target picture according to the pixel matching result of the current picture and the previous picture, thereby completing the frame supplementing operation by matching local pixels. Because the content generally displayed on the desktop of the server is simple, the pixel matching efficiency can be improved by matching local pixels, so that the frame supplementing efficiency is improved. According to the scheme, on the premise that the real-time computing performance of the terminal where the client is located allows, the frame supplementing operation is performed by matching local pixels, so that the terminal resources can be adapted to supplement frames of display pictures of the server desktop, the phenomena of picture blocking and the like caused by frame supplementing can be avoided, and the resource occupation balance between the frame supplementing operation and other tasks on the terminal is realized as much as possible. The frames are supplemented to a certain extent, so that the frames can be displayed more coherently, the visual experience is better, and misoperation of server management can be reduced to a certain extent.

As described above, the present invention can calculate the motion vector of the specified region in the previous and subsequent frames by matching the local pixels to perform the dynamic adaptive interpolation frame, and then generate the interpolation frame accordingly, and can supplement the frame lower than 30fps (frames/second) to 30fps by the adaptive interpolation frame.

In one example, a frame number threshold of 5 may be set first, and if calculated at a frame rate of 30fps, the processing time per frame of picture is around 33 milliseconds (ms), and the processing time of 5 frames is about 165ms. If the client normally receives 5 frames of pictures in 165ms, the current frame number is considered to be normal, and the condition of insufficient frame number does not exist. The statistical time duration can be restarted after receiving a frame of picture, so as to judge whether the frame number is missing. And the following steps: when no new frame data is received within a specified time interval, or the transmission time of a certain frame exceeds the transmission time of a normal frame data (i.e. the time of the transmission of the original next frame is occupied), the situation of insufficient frame number is considered to occur. Hereby it can be set that: if the specified number of frames is received within 33ms, then no supplemental frames are deemed necessary, and the received picture is directly stored in the specified image display buffer.

In one example, a frame-filling flow is shown in FIG. 5. After receiving a frame of picture, judging whether the situation of insufficient frame number exists or not; if yes, the number of missing frames is recorded and received continuously, if the number of missing frames exceeds the threshold, the number of frames to be complemented exceeds the set frame complementing threshold, and frame complementing recovery cannot be carried out, so that the process is ended directly. If the missing frame number does not exceed the threshold value, the frame number difference N between the last received frame picture (current picture) and the previous received frame picture (previous picture) is calculated, wherein N is not less than 1 and not more than 4. And then the N frames of pictures are subjected to frame filling. If the condition of insufficient frame number does not occur, the screen filling is directly carried out by utilizing the currently received frame of picture so as to finish the display of the picture at the client.

Wherein, the frame supplementing the N frames of pictures comprises: calculating the pixel difference between the last received frame of picture and the last received frame of picture so as to judge the pixel point change of the two frames of pictures; if the pixel difference value is not greater than the pixel threshold value, the pixel point of the two frames of pictures is not changed greatly, and then the latest received frame of picture or the previous frame of picture is directly copied to complete frame supplementing. If the pixel difference value is larger than the pixel threshold value, the pixel point of the two frames of pictures is considered to be changed greatly, and if operations such as window movement or mouse movement occur, the displacement vectors need to be matched and calculated by using a block matching algorithm.

In one example, the block matching algorithm does not use the global pixel point of the whole frame to perform matching, but starts from the initial position where the difference occurs between the two frames, and determines the local matching area (i.e., the local picture area) according to the edge point of the pixel point position where the difference occurs. Reference is made in particular to fig. 2. Wherein, pixel matching can be performed according to a block size of 32×32, and the block size can be configured according to the image resolution size.

In the pixel matching process, if the sum of absolute errors of two pixel blocks at corresponding positions in front and rear pictures is smaller than a set threshold value, the two pixel blocks are considered to be consistent, the pixel block belongs to the same pixel block in front and rear pictures in a local picture area, and the pixel block is considered to belong to a non-interested area. If the sum of absolute errors of two pixel blocks at corresponding positions in the front and rear pictures is not smaller than the set threshold, the target pixel block is considered to be matched, that is, the two pixel blocks which are currently matched with each other are at the same position of the front and rear pictures, but the pixel contents are different, as shown in fig. 2, the current pixel block X and the previous pixel block Y. A region of interest (e.g., window a) may be determined by counting all the matched target pixel blocks. Such as: and calculating the positions of the edge pixel points of the region of interest according to the vertex positions of all the target pixel blocks, and counting the positions of all the edge pixel points to obtain a rectangular frame range (shown as a region S in FIG. 2) corresponding to the region of interest.

After the region of interest is determined, the motion displacement vector of the pixel block can be calculated from the two target pixel blocks that match each other at the same location. And carrying out subsequent frame supplementing operation by referring to the motion displacement vector. As shown in fig. 6, in the previous frame im1, the windows window is at the position a, in the next frame im2, the windows window is at the position B, if the number of missing frames is n=1, then a window is inserted between the position a and the position B according to the direction of the motion displacement vector, the windows window and other pixels around the window can form the missing whole picture, and the other pixels around the window are pixels corresponding to the positions in im1 or im 2.

The position of the inserted windows window in the whole picture is also determined at present, and the specific process is as follows: acquiring left lower corner vertex positions A1 (A1, b 1) of a window of im1 and right upper corner vertex positions A2 (A2, b 2) of a window of im2, moving A1 (A1, b 1) to positions (x, y) along the direction of a motion displacement vector, wherein x=a1+ (A2-A1)/2 represents the transverse direction and y=b1+ (b 2-b 1)/2 represents the longitudinal direction; and simultaneously, other pixels of the windows in im1 are moved by the same distance as A1 in the same way, the windows after the movement is completed are positioned at the AO position, the windows at the AO position and other pixels around the windows can form a missing whole picture im3, and the other pixels around the windows are pixels at corresponding positions in im1 or im 2.

The padding of other pixels around can be performed according to the following rule: setting windows in im1 as ROI1, windows in im2 as ROI2, windows in im3 as ROI3, and filling pixels except for ROI 1U-ROI 2U-ROI 3 with pixels in corresponding positions in im1 or im 2. Specifically, for the pixels inside the ROI1 that are not inside the ROI2, the corresponding position pixels in im2 are used for padding, and for the pixels inside the ROI2 that are not inside the ROI1, the corresponding position pixels in im1 are used for padding. ROI (Region Of Interest) the region of interest.

Similarly, if N frames are complemented, the following calculation is performed from the lower left corner vertex position A1 of ROI1 of im1 and the upper right corner vertex position A2 (A2, b 2) of ROI2 of im 2: the transverse distance a1+ (a 2-a 1)/N+1 and the longitudinal distance b1+ (b 2-b 1)/N+1 are respectively used for obtaining the transverse distance and the longitudinal distance of two adjacent positions, and N positions are uniformly determined along the direction of the motion displacement vector according to the transverse distance and the longitudinal distance. This process is similar to uniformly dividing a distance into n+1 segments, one division position being one position corresponding to ROI 1.

And after finishing one frame supplementing operation, restarting timing the current frame number, and continuously judging whether the frame number of the subsequent display picture exists or not. Therefore, the embodiment can perform self-adaptive optimization frame filling at the BMC client, and fill the frame number of the display picture through a software algorithm when the real frame number of the display picture is small, so that the display picture effect is smoother and the visual experience effect is better through occupying small resources.

In one example, a system as shown in FIG. 7 may be designed to complement frames, including: the device comprises a receiving buffer area, a frame rate detection module, a block matching module, a frame supplementing module and a display buffer area. The receiving buffer area can store 5 frames of pictures at least, and the data of the receiving buffer area is directly stored in the display buffer area for display when the frame rate is normal. And when the frame rate is abnormal, judging a frame supplementing operation according to the data in the buffer area, and writing the data with the frame supplementing completion into the display buffer area. For example: when the first frame im1 is received in the current time and then the next frame im2 is not received in 33ms, the receiving is continued, and if a new frame im3 is received, im1 and im3 are used for calculation to obtain im2. When im1, im2, im3 images are ready, i.e. there are at least 3 frames in the image buffer, these 3 frames are sent sequentially to the display buffer in sequence, and there is a call for external KVM (Kernel-based Virtual Machine) client display. Therefore, when the server is remotely operated by the KVM client, display images such as a mouse and a window are smoother, the phenomenon of instantaneous movement in the images is reduced, the display frame rate of the images is improved, the operation impression experience can be improved to a certain extent, and misoperation is reduced. The method improves the display frame rate through self-adaptive optimization, and the frame supplementing rate is faster. And the local matching algorithm is adopted, so that the consumption of system computing resources is reduced.

Optionally, if the pixel change information is greater than the preset change threshold, detecting the current frame and the previous frame by using a preset neural network model to obtain a region of interest (a window a in the current frame illustrated in fig. 2) in the current frame and a region of interest (a window a in the previous frame illustrated in fig. 2) in the previous frame; and performing frame supplementing operation on the target picture based on the region of interest in the current picture and the region of interest in the previous picture. It can be seen that the window a in the current picture illustrated in fig. 2 and the window a in the previous picture illustrated in fig. 2 can be determined through a preset neural network model.

As described above, if the difference between two frames is large or the number of corresponding pixel changes is large, it is considered that the two frames of images before and after each other have image changes, for example: window movement or mouse movement etc. occurs, and at this time, the ROI area needs to be determined, namely, determination: the position of which part of pixels in the front and rear frames is to be shifted. The foregoing embodiment employs a block matching algorithm to determine the ROI area, and the present embodiment employs a neural network model to determine the ROI area. Neural network models such as: a convolutional neural network model.

Before the neural network model is used, the actual hardware performance of the device running the KVM client can be measured, and if the performance of the device is good, the current picture and the previous picture are detected by using the preset neural network model to determine the ROI area in the front picture and the rear picture under the condition that the pixel change information is larger than the preset change threshold value. The performance of the device may be determined based on the current amount of free resources of the device. The neural network model can also detect objects such as windows, mice, etc. in the picture.

The neural network model may specifically include an input layer, a hidden layer, and an output layer. The training model uses the activating function Sigmoid to train 1000 rounds of iteration. If the recognition accuracy of the trained model to the mouse and the window reaches more than 90%, the model is determined to be applicable to online application. It should be noted that, the simpler the composition structure of the neural network model is, the easier it is to apply and integrate in the KVM client, and the model can be quantitatively cut if necessary, so that the model parameters are fewer, and the model structure is simpler. The data set used in training is: server desktop images at different resolutions of self-screenshot, each image including window, mouse, etc.

After model training is completed, a detection model which can be specially used for detecting windows in the image can be obtained, and window information in the image can be identified. The model is also identified if there are multiple display windows in a frame. When the window information is identified, whether the window is a moving window or not, namely, the ROI area can be further detected, the area size of the window can be calculated according to the position vertex data of the identified window for matching, and the structural similarity (SSIM, structural Similarity Index) of pixels in the window can be calculated for determining.

In one example, before the frame-filling operation is performed on the target frame based on the region of interest in the current frame and the region of interest in the previous frame, the two regions of interest are area-matched or structural similarity calculated according to the region of interest in the current frame and the region of interest in the previous frame to determine whether the two regions of interest are moving windows in the two frames of the previous and the subsequent frames.

An image processing apparatus according to an embodiment of the present invention is described below, and the image processing apparatus described below and other embodiments described herein may be referred to with each other.

Referring to fig. 8, an embodiment of the present invention discloses an image processing apparatus applied to a client of a baseboard management controller, including:

a receiving module 801, configured to receive a current picture of a server desktop sent by a baseboard management controller;

the detection module 802 is configured to calculate pixel change information of a current frame and a received previous frame when the device where the client is located currently has resources required for frame compensation if a preset frame compensation condition is currently met; wherein, the resources required by the frame filling are as follows: the resources required by frame supplementing are carried out on the target picture which is missing between the current picture and the previous picture;

And a frame compensation module 803, configured to determine a local frame region according to a frame difference between the current frame and the previous frame if the pixel change information is greater than the preset change threshold, restrict the pixel matching operation of the current frame and the previous frame to the local frame region, and perform the frame compensation operation on the target frame according to the pixel matching result of the current frame and the previous frame.

In one embodiment, the preset frame-supplementing condition includes: the difference value between the receiving time stamp of the current picture and the receiving time stamp of the previous picture is larger than the preset timeout time; or the total receiving time length of the current picture is longer than the preset receiving time length; or the total number of pictures received in the preset period is smaller than a preset fixed value; or the number of target pictures is less than a preset deletion threshold.

In one embodiment, the method further comprises: the display module is used for storing the current picture into the buffer area if the current picture does not meet the preset frame supplementing condition; and reading the current picture from the buffer area and displaying the current picture.

In one embodiment, not meeting the preset frame complement condition includes: the difference value between the receiving time stamp of the current picture and the receiving time stamp of the previous picture is not more than the preset timeout time; or the total receiving time length of the current picture is not more than the preset receiving time length; or the total number of the pictures received in the preset period is not smaller than a preset fixed value; or the target number of the target pictures is not less than a preset deletion threshold.

In one embodiment, the detection module is specifically configured to: acquiring idle equipment resources of real-time statistics of a target process running on equipment; if the idle resources of the equipment are not less than the resources required by the frame filling, determining that the equipment currently has the resources required by the frame filling; or calculating the resource occupation amount of the task currently operated by the equipment; determining the difference value between the rated resource quantity and the resource occupation quantity of the equipment as the idle resource of the equipment; if the idle resources of the equipment are not smaller than the resources required by the frame supplement, determining that the equipment currently has the resources required by the frame supplement.

In one embodiment, the detection module is specifically configured to: inquiring the frame supplementing configuration information to obtain the maximum frame supplementing number applicable to the equipment; if the number of the complementary frames is not greater than the preset complementary frame threshold, determining that the idle resources of the equipment are not less than the resources required by the complementary frames. If the number of the complementary frames is larger than the preset complementary frame threshold, determining that the idle resources of the equipment are smaller than the resources required by the complementary frames.

In one embodiment, the display module is further configured to: when the idle resources of the equipment are smaller than the resources required by the frame filling, the frame filling operation is skipped, and the current picture is stored in a buffer area; and reading the current picture from the buffer area and displaying the current picture.

In one embodiment, the determining the resources required for the complementary frame includes: determining the target number of target pictures; inquiring the frame supplementing configuration information to obtain the resource usage amount required by the equipment for carrying out frame supplementing operation on the target quantity; and determining the resource usage as the resource required by the complementary frame.

In one embodiment, the detection module is specifically configured to: calculating the total current pixel of the current picture; calculating the total amount of the previous pixels of the previous picture; the difference between the current pixel total and the previous pixel total is determined as pixel change information.

In one embodiment, the frame-filling module is further configured to: if the pixel change information is not greater than the preset change threshold value, determining the target number of the target pictures; obtaining a target number of target pictures according to the current picture and the previous picture; storing the current picture and the target number of target pictures into a buffer memory area; and reading the target number of target pictures and the current picture from the buffer memory area, and displaying the target number of target pictures and the current picture in sequence.

In one embodiment, the frame-filling module is further configured to: copying the current picture for the target number of times to obtain target number of target pictures; or copying the previous picture for the target number of times to obtain a target number of target pictures; or calculating the pixel mean value of the current picture and the previous picture, and generating a target number of target pictures based on the pixel mean value.

In one embodiment, the frame-filling module is specifically configured to: determining edge points where pixels change by comparing a current picture with a previous picture; the smallest rectangle covering the area surrounded by all edge points is determined as a local picture area.

In one embodiment, the frame-filling module is specifically configured to: intercepting a current picture and a previous picture by utilizing a local picture area to obtain a current intercepting surface of the current picture and a previous intercepting surface of the previous picture; performing pixel matching operation on the current cut-out plane and the previous cut-out plane by using a block matching algorithm to determine the same pixel areas positioned at different positions of the current picture and the previous picture; and recording the current position information of the same pixel area in the current picture and the previous position information of the same pixel area in the previous picture to obtain a pixel matching result.

In one embodiment, the frame-filling module is specifically configured to: determining a previous coordinate position of the target pixel in the previous position information; determining a current coordinate position of the target pixel in the current position information; determining a displacement vector from a previous coordinate position to a current coordinate position; determining the moving direction of the same pixel area according to the direction of the displacement vector; and performing frame supplementing operation on the target picture in the moving direction.

In one embodiment, the frame-filling module is specifically configured to: determining the positions of the same pixel areas in the target frames in the target number according to the target number of the target frames, the previous coordinate position and the current coordinate position; and filling pixels at positions corresponding to the current picture or the previous picture in other positions except the current same pixel region aiming at the same pixel region at each position to obtain a frame of target picture corresponding to the same pixel region at each position. The display module is also used for: storing the current picture and the target number of target pictures into a buffer memory area; and reading the target number of target pictures and the current picture from the buffer memory area, and displaying the target number of target pictures and the current picture in sequence.

In one embodiment, the method further comprises:

and the cache cleaning module is used for deleting the target number of target pictures and the current picture from the cache area.

The more specific working process of each module and unit in this embodiment may refer to the corresponding content disclosed in the foregoing embodiment, and will not be described herein.

Therefore, the embodiment provides an image processing device, which can improve frame supplementing efficiency by matching local pixels on the premise of allowing real-time computing performance of a terminal where a client is located, and can also adapt to terminal resources to supplement frames of display pictures of a server desktop, so that picture blocking and other phenomena caused by frame supplementing are avoided, and resource occupation balance between frame supplementing operation and other tasks on the terminal is realized.

An electronic device provided in the embodiments of the present invention is described below, and an electronic device described below may refer to other embodiments described herein.

Referring to fig. 9, an embodiment of the present invention discloses an electronic device, including:

a memory 901 for storing a computer program;

a processor 902 for executing the computer program to implement the method disclosed in any of the embodiments above.

Further, the embodiment of the invention also provides electronic equipment. The electronic device may be a server as shown in fig. 10 or a terminal as shown in fig. 11. Fig. 10 and 11 are structural diagrams of an electronic device according to an exemplary embodiment, and the contents of the drawings should not be construed as any limitation on the scope of use of the present invention.

Fig. 10 is a schematic structural diagram of a server according to an embodiment of the present invention. The server specifically may include: at least one processor, at least one memory, a power supply, a communication interface, an input-output interface, and a communication bus. Wherein the memory is configured to store a computer program that is loaded and executed by the processor to implement the relevant steps in image processing disclosed in any of the foregoing embodiments.

In this embodiment, the power supply is configured to provide a working voltage for each hardware device on the server; the communication interface can create a data transmission channel between the server and external equipment, and the communication protocol to be followed by the communication interface is any communication protocol applicable to the technical scheme of the invention, and the communication protocol is not particularly limited; the input/output interface is used for acquiring external input data or outputting data to the external, and the specific interface type can be selected according to the specific application requirement, and is not limited in detail herein.

In addition, the memory may be a read-only memory, a random access memory, a magnetic disk, an optical disk, or the like as a carrier for storing resources, where the resources stored include an operating system, a computer program, data, and the like, and the storage mode may be transient storage or permanent storage.

The operating system is used for managing and controlling each hardware device and computer program on the Server to realize the operation and processing of the processor on the data in the memory, and the operation and processing can be Windows Server, netware, unix, linux and the like. The computer program may further include a computer program that can be used to perform other specific works in addition to the computer program that can be used to perform the image processing method disclosed in any of the foregoing embodiments. The data may include data such as information on a developer of the application program in addition to data such as update information of the application program.

Fig. 11 is a schematic structural diagram of a terminal according to an embodiment of the present invention, where the terminal may specifically include, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, or the like.

Generally, the terminal in this embodiment includes: a processor and a memory.

The processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing ), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor may also include a main processor, which is a processor for processing data in an awake state, also called a CPU (Central Processing Unit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor may incorporate a GPU (Graphics Processing Unit, image processor) for rendering and rendering of content required to be displayed by the display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.

The memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory is at least used to store a computer program, where the computer program, after being loaded and executed by the processor, can implement relevant steps in the image processing method performed by the terminal side as disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory can also comprise an operating system, data and the like, and the storage mode can be short-term storage or permanent storage. The operating system may include Windows, unix, linux, among others. The data may include, but is not limited to, update information for the application.

In some embodiments, the terminal may further include a display screen, an input-output interface, a communication interface, a sensor, a power supply, and a communication bus.

Those skilled in the art will appreciate that the structure shown in fig. 11 is not limiting of the terminal and may include more or fewer components than shown.

A readable storage medium provided by embodiments of the present invention is described below, and the readable storage medium described below may be referred to with respect to other embodiments described herein.

The embodiment of the invention discloses a readable storage medium for storing a computer program, wherein the computer program is executed by a processor to realize the image processing method disclosed in the previous embodiment. The readable storage medium is a computer readable storage medium, and can be used as a carrier for storing resources, such as read-only memory, random access memory, magnetic disk or optical disk, wherein the resources stored on the readable storage medium comprise an operating system, a computer program, data and the like, and the storage mode can be transient storage or permanent storage.

A system provided by embodiments of the present invention is described below, and a system described below may be referred to with respect to other embodiments described herein.

The embodiment of the invention discloses a system, which comprises: the system comprises a server, a baseboard management controller connected with the server and terminal equipment, wherein the terminal equipment is provided with a client of the baseboard management controller, and the client realizes the image processing method disclosed by the prior art. The client is a kernel-based virtual machine client.

In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of readable storage medium known in the art.

The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.

Claims

1. An image processing method, applied to a client of a baseboard management controller, comprising:

2. The method of claim 1, wherein the preset supplemental frame conditions comprise:

3. The method as recited in claim 1, further comprising:

4. A method according to claim 3, wherein not meeting the preset supplemental frame condition comprises:

5. The method of claim 1, wherein the device currently has resources required for the supplemental frame, comprising:

or (b)

6. The method of claim 5, wherein the device free resources are not less than resources required for the supplemental frame, comprising:

7. The method of claim 6, wherein if the number of frames is greater than the predetermined frame filling threshold, determining that the device free resources are less than the resources required for the frames filling.

8. The method as recited in claim 5, further comprising:

9. The method of claim 1, wherein the determining of the resources required for the complementary frame comprises:

Determining a target number of the target pictures;

10. The method of claim 1, wherein said calculating pixel change information for the current picture and the received previous picture comprises:

calculating the total current pixel amount of the current picture;

calculating the total amount of the previous pixels of the previous picture;

11. The method as recited in claim 1, further comprising:

12. The method of claim 11, wherein the obtaining a target number of target pictures from the current picture and the previous picture comprises:

or (b)

13. The method according to any one of claims 1 to 12, wherein said determining a local picture area from a picture difference of the current picture and the previous picture comprises:

14. The method according to any one of claims 1 to 12, wherein said restricting the pixel matching operation of the current picture and the previous picture to the partial picture region comprises:

15. The method of claim 14, wherein the performing a frame-filling operation on the target picture according to the pixel matching result of the current picture and the previous picture comprises:

16. The method of claim 15, wherein the frame-filling the target frame in the moving direction comprises:

17. The method of claim 16, wherein after the sequentially displaying the target number of target pictures and the current picture, further comprising:

18. The method according to any one of claims 1 to 12, further comprising:

if the pixel change information is larger than a preset change threshold value, detecting the current picture and the previous picture by using a preset neural network model to obtain an interested region in the current picture and an interested region in the previous picture;

19. An image processing apparatus, applied to a client of a baseboard management controller, comprising:

20. An electronic device, comprising:

a memory for storing a computer program;

a processor for executing the computer program to implement the method of any one of claims 1 to 18.

21. A readable storage medium for storing a computer program, wherein the computer program when executed by a processor implements the method of any one of claims 1 to 18.

22. A system, comprising: server, baseboard management controller connected to the server, and terminal equipment, the terminal equipment being equipped with a client of the baseboard management controller, the client implementing the method according to any one of claims 1 to 18.

23. The system of claim 22, wherein the client is a kernel-based virtual machine client.