JP7126539B2 - Video playback method, terminal and system - Google Patents

Description

The present invention relates to the field of video surveillance, in particular to a video playback method, terminal and system in the field of video surveillance.

High definition video has now become an important technological trend in the field of video surveillance. And cameras with 720p or 1080p resolution are more and more widely used in the field of video surveillance. With the continual increase in camera resolution, the range that a single camera can monitor has become larger and the image details have become clearer. At the same time, intelligent analysis techniques for video images are gradually being implemented. Technological developments in hardware devices have enabled hardware performance to meet the demands of performing intelligent analysis on multiple regions of interest within the same image. This saves significant costs compared to manual monitoring.

Current video surveillance clients typically play back video images from multiple cameras at the same time. However, with increasing video image resolution, the total resolution of the video images from multiple cameras often exceeds the resolution range of the client's monitor. Taking a 22 inch display as an example, this display generally supports a maximum resolution of 1920×1080. That is, the display is only capable of reproducing 1080p images from one feed. If multiple feeds of 1080p images are played on this monitor at the same time, the images must be zoomed out. In addition to the playback window, there are also multiple auxiliary function panels on a typical video surveillance client interface, such as the title bar, camera list, and pan-tilt-zoom control panel, which further display the video image. Reduce the display area of . Therefore, the image that can be reproduced within the reproduction window is much smaller than the original image.

In particular, when an event (eg, an event triggered by intelligent analysis) occurs within a video image, the area of the image where the event occurs is even smaller. This is because the image is downsized during playback, which is inconvenient for the user to view.
When an observer monitors an image with the naked eye, it is difficult for the observer to notice changes in details, and as a result important information is missed.

Most clients today offer the ability to zoom in on a selected area of an image. That is, an area of the video playback image is selected by sliding the mouse and the selected area is zoomed in. This improves the image quality of the region of interest to some extent. However, digital zooming of video images causes loss of some pixel information, thus affecting image quality and further affecting the effectiveness of user observation of image details. In addition, if the function of zooming within the selected area is used, in one aspect, manual operation by the user is required, and when the event occurs suddenly, the user does not have time to perform any operation, thus miss an event. In another aspect, it is not possible to zoom in on several regions at the same time if the events occur in different regions of the image. Therefore, the user experience is relatively poor.

Embodiments of the present invention provide a video playing method, terminal and system, which can improve user experience.

In a first aspect, embodiments of the present invention provide a video playback method, comprising the steps of: dividing an original playback image into at least two regions of interest; determining a first region of interest in which an event occurs; obtaining decoded data of a first video image displayed in the first region of interest; obtaining decoded data of the first video image; and drawing to a designated playback window for playback.

In a first possible implementation of the first aspect, the method further comprises determining a correspondence between each region of interest of the at least two regions of interest and the designated playback window. and rendering the decoded data of the first video image into a designated playback window for playback includes rendering the decoded data of the first video image to the first object for playback according to the correspondence relationship. There is the step of drawing in a designated playback window corresponding to the region.

In relation to the first aspect or the first possible implementation of the first aspect, in the second possible implementation of the first aspect, the triggering event occurs in at least two regions of interest Determining the first region of interest comprises determining a triggering action to be performed by the user on the region of interest in the original reproduced image, the triggering action being a click action, a double-click action. , or the act of selecting a region of interest, further comprising determining the region of interest on which the triggering action is to be performed as the first region of interest.

In a third possible implementation of the first aspect, in relation to the first aspect or the first possible implementation of the first aspect, the triggering event occurs in at least two regions of interest The step of determining the first region of interest includes obtaining coordinate metadata of the trigger event occurrence point in the original reproduced image; and determining as a region.

In relation to the first aspect or any one of the first through third possible implementations of the first aspect, in the fourth possible implementation of the first aspect, the first region of interest obtaining the decoded data of the first video image displayed in the step of obtaining the decoded data of the original reproduced image; and obtaining the decoded data of the first video image according to the decoded data of the original reproduced image. and determining.

In relation to the first aspect or any one of the first through fourth possible implementations of the first aspect, in the fifth possible implementation of the first aspect, Rendering the decoded data into the designated playback window for playback comprises rendering the decoded data of the first video image into the designated playback window for playback in a zoomed-in manner. Thus, the specified playback window is larger than the first region of interest.

In relation to the first aspect or any one of the first through fifth possible implementations of the first aspect, in the sixth possible implementation of the first aspect, Rendering the decoded data into a designated playback window for playback comprises popping up a separate playback window and rendering the decoded data of the first video image into a separate playback window for playback. and a step.

In a second aspect, embodiments of the present invention provide a video playback terminal, the terminal comprising a segmentation module configured to segment an original playback image into at least two regions of interest, and a segmentation module selected by the segmentation module. a first determining module configured to determine a first region of interest in which a triggering event occurs among the at least two regions of interest issued; and a first target determined by the first determining module. an acquisition module configured to acquire decoded data of a first video image displayed in the region; a playback module configured to draw to the playback window.

In a first possible implementation of the second aspect, the terminal is configured to determine correspondence between each target area of the at least two target areas and a specified playback window. Further comprising a second determining module, the reproducing module converts the decoded data of the first video image obtained by the obtaining module to the first video image for reproduction according to the correspondence determined by the second determining module. is further configured to draw to a specified playback window corresponding to the region of interest of the

Relating to the second aspect or the first possible implementation of the second aspect, in the second possible implementation of the second aspect, the first determining module comprises: a first determining unit configured to determine a triggering action performed by a user on the target area, the triggering action being a clicking action, a double-clicking action, or an action of selecting the target area; and further comprising a second determining unit configured to determine, as the first target area, a target area in which the triggering action determined by the first determining unit is to be performed.

In relation to the second aspect or the first possible implementation of the second aspect, in the third possible implementation of the second aspect, the first determining module comprises: a first acquisition unit configured to acquire coordinate metadata of the trigger event occurrence point; and determining a region of interest to which the trigger event occurrence point belongs to the first target according to the coordinate metadata acquired by the first acquisition unit. and a third determining unit configured to determine as a region.

In a fourth possible implementation of the second aspect, in conjunction with the second aspect or any one of the first through third possible implementations of the second aspect, the acquisition module comprises: a second acquisition unit configured to obtain decoded data of the reconstructed image; and for determining decoded data of the first video image according to the decoded data of the original reconstructed image acquired by the second acquisition unit. and a third decision unit configured as:

In a fifth possible implementation of the second aspect, in relation to the second aspect or any one of the first through fourth possible implementations of the second aspect, the regeneration module comprises: is further configured to render the decoded data of the video image in a specified playback window for playback in a zoomed-in manner, the specified playback window being larger than the first region of interest.

In relation to the second aspect or any one of the first to fifth possible implementations of the second aspect, in the sixth possible implementation of the second aspect, the regeneration module comprises an independent a display unit configured to display a playback window; and a playback unit configured to render decoded data of the first video image into a separate playback window popped up by the display unit for playback. have.

In a third aspect, embodiments of the present invention provide a video playback system comprising a terminal according to the second aspect of the present invention and a video image captured by capturing and encoding the video image. a video capture system configured to generate a media stream; a server configured to obtain the media stream generated by the video capture system and provide the media stream to a terminal; a storage device configured to store a media stream, the terminal comprising: a segmentation module configured to segment an original reproduced image into at least two target regions; a first determining module configured to determine a first region of interest in which the triggering event occurs among the at least two regions of interest; and within the first region of interest determined by the first determining module. an acquisition module configured to acquire decoded data of a first video image displayed in a specified playback window for playback; a playback module configured to render.

Based on the above technical solution, with the video playback method, terminal and system according to the embodiments of the present invention, the original playback image is divided into a plurality of target regions, and the target region where the trigger event occurs is Images inside are displayed separately. Thus, in one aspect, the user is able to observe more distinct image detail within the regions of interest, and in another aspect, the user is able to track image detail within multiple regions of interest simultaneously. can be used, which greatly improves the user experience.

To describe the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings required for describing the embodiments of the present invention. Apparently, the accompanying drawings in the following description only show some embodiments of the present invention, and those skilled in the art can further derive other drawings from these accompanying drawings without creative efforts. can be done.

A video playback method, terminal and system in the field of video surveillance are provided.

1 is a schematic block diagram of an exemplary application scenario, according to an embodiment of the invention; FIG. 1 is a schematic flow diagram of a video playback method according to an embodiment of the present invention; FIG. Fig. 4 is another schematic flow diagram of a video playing method according to an embodiment of the present invention; Figure 4 is a schematic flow diagram of a method for segmenting an original reconstructed image into regions of interest, according to an embodiment of the present invention; Fig. 3 is another schematic flow diagram of a method for segmenting an original reconstructed image into regions of interest according to an embodiment of the present invention; 1 is a schematic flow diagram of a method for determining a region of interest in which a trigger event occurs, according to one embodiment of the invention; FIG. FIG. 5 is another schematic flow diagram of a method for determining a region of interest in which a trigger event occurs, according to an embodiment of the invention; FIG. 4 is a schematic flow diagram of a method for obtaining decoded data for a region of interest, according to an embodiment of the present invention; 1 is a schematic flow diagram of a method for reconstructing an image within a region of interest, according to one embodiment of the present invention; FIG. FIG. 4 is a schematic flow diagram of a video playing method according to another embodiment of the present invention; FIG. 4 is another schematic flow diagram of a video playing method according to another embodiment of the present invention; FIG. 4 is another schematic flow diagram of a video playing method according to another embodiment of the present invention; FIG. 4 is a schematic diagram of playing a region of interest according to an embodiment of the present invention; FIG. 4 is a schematic diagram of playing a region of interest according to an embodiment of the present invention; 1 is a schematic block diagram of a terminal according to an embodiment of the present invention; FIG. FIG. 4 is another schematic block diagram of a terminal according to an embodiment of the present invention; Fig. 2 is a first decision module schematic block diagram according to an embodiment of the present invention; FIG. 2B is another schematic block diagram of the first decision module, according to an embodiment of the present invention; FIG. 4 is a schematic block diagram of an acquisition module, according to one embodiment of the present invention; FIG. 4 is a schematic block diagram of a regeneration module, according to one embodiment of the present invention; 1 is a system schematic block diagram, according to one embodiment of the present invention; FIG. FIG. 4 is a schematic block diagram of a terminal according to another embodiment of the present invention;

The following clearly and completely describes the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by persons skilled in the art without creative efforts based on the embodiments of the present invention shall fall within the protection scope of the present invention.

FIG. 1 is a schematic block diagram of an exemplary application scenario, according to one embodiment of the present invention. As shown in FIG. 1, a video surveillance system to which embodiments of the present invention are applicable includes a terminal having a video capture device, a central server, a storage device, and a client, the video capture device for capturing video images. and can generate a media stream by encoding the video images to transmit the video images over a network. For example, video capture devices may include devices such as network cameras, analog cameras, encoders, and digital video recorders (DVRs for short). After being connected to the central server, the client of the terminal can request the video stream, decode and display this video stream, and present the on-site video image to the user.

Central servers may include management servers and media servers. A media server may be responsible for receiving media streams, recording and storing media stream data in storage devices, and forwarding media streams to clients for playback on demand. A management server may serve functions such as user login, authentication, and service scheduling, for example. The central server may also be accessed by multiple clients, such as managing network connections between various video surveillance systems. The storage device may be, for example, a disk array. Disk arrays can serve to store video data, and can be network attached storage (NAS for short), storage area network (SAN for short), Or the server itself can be used.

It should be understood that the video surveillance system shown in FIG. 1 is merely an embodiment to which the method of the present invention can be applied and is not intended to limit the purpose and functionality of the present invention. . The present invention should not be described as dependent on any one or any combination of the components of the video surveillance system shown in FIG. However, in order to elaborate the invention more clearly, embodiments of the invention are described below by taking the application scenario of a video surveillance system as an example. However, the invention is not so limited.

The technical solutions for video data transmission in the embodiments of the present invention can be applied to various communication networks or communication systems, such as the Global System for Mobile Communications ("GSM" for short). "), Code Division Multiple Access (abbreviated as "CDMA") systems, Wideband Code Division Multiple Access (abbreviated as "WCDMA") systems, general packet radio services ( general packet radio service ("GPRS" for short) system, Long Term Evolution ("LTE" for short) system, LTE frequency division duplex ("FDD" for short) system, LTE time Time division duplex ("TDD" for short) system, Universal Mobile Telecommunication System ("UMTS" for short) or Worldwide Interoperability for Microwave Access It should be understood that a Microwave Access, abbreviated "WiMAX") communication system may be used. Embodiments of the invention are not limited to these.

FIG. 2 is a schematic flow chart of a video playback method 100, according to one embodiment of the invention.
This method 100 may be performed by a video playback device. This device is for example a terminal or a client. As shown in FIG. 2, the method 100 has the following steps.
S110: Divide the original reconstructed image into at least two regions of interest.
S120: Determining a first region of interest in which a trigger event occurs among the at least two regions of interest.
S130: Obtain decoded data of the first video image displayed in the first region of interest.
S140: Draw the decoded data of the first video image into the designated playback window for playback.

In order not to affect the quality of the reproduced video and to improve the effect of user observation on image details, especially when multiple video images are displayed in the same window in a zoomed-out manner: The video playback device may first divide the original playback image into multiple regions of interest. Then, decoded data of the video image displayed in the region of interest where the trigger event occurs is obtained. Accordingly, the video playback device can render the decoded data of the video image into a separate designated playback window for playback. In this way, image details that are of interest to the user can be displayed in separate windows. And the effect of user observation on image details is improved.

Therefore, the video playback method according to an embodiment of the present invention divides the original playback image into a plurality of target regions and separately displays the images in the target region where the trigger event occurs. Thus, in one aspect, the user is able to observe more distinct image detail within regions of interest, and in another aspect, the user is able to track image detail in multiple regions of interest simultaneously. , which greatly improves the user experience.

It should be understood that in embodiments of the present invention, video not only includes video files, but also real-time video streams. Embodiments of the present invention are described by taking playback of a real-time video stream as an example, but embodiments of the present invention are not so limited.

In an embodiment of the present invention, optionally as shown in FIG. 3, method 100 further comprises the following steps.
S150: Determine the correspondence between each target region of the at least two target regions and the designated playback window.

Rendering the decoded data of the first video image into a designated playback window for playback comprises the steps of: a.
S141: Rendering the decoded data of the first video image into the specified playback window corresponding to the first target area for playback according to the correspondence.

That is, each region of interest may be associated with one or more playback windows for playing back images within the region of interest when a triggering event occurs within the region of interest. The specified window may be the largest playback window on the display device and may be part of the largest playback window. The specified window may be a current existing playback window or part of an existing playback window, or may be a new pop-up playback window or a newly created playback window. Embodiments of the invention are not limited to these.

Referring to FIGS. 4 to 12B, the video playing method according to an embodiment of the present invention will now be described in detail.

In S110, optionally dividing the original reproduced image into at least two target regions comprises dividing the original reproduced image into at least two target regions in an equal division method or a free division method. ing.

Specifically, a single playback window may be pre-divided into multiple regions of interest on the client. The size of the regions of interest may be the same or different, and the regions of interest may be set as irregular regions. Additionally, in embodiments of the present invention, correspondences between regions of interest and playback windows may be determined. The step of dividing the original reproduced image into regions of interest may be performed manually by the user or automatically configured by the client software, which configuration is saved on the client.

The image can be split with an equal split method or a free split method. A specific configuration process is shown in FIGS. 4 and 5. FIG. For example, as shown in FIG. 4, a method for dividing an image into regions of interest with an equal division method comprises the following steps.
S111: Click the right-click menu or toolbar button to display the configuration window.
S112: Set the number of target regions in the pop-up configuration window. For example, set the number of regions of interest to sixteen.
S113: Right-click on the target area to set a playback window combined with the target area.
S114: Select a playback window to play the video in the target region when the trigger event occurs within the target region.

As shown in FIG. 5, a method for dividing an image into target regions by a free dividing method may include, for example, the following steps.
S115: Click the right-click menu or toolbar button to pop up a configuration window.
S116: In the pop-up configuration window, drag the mouse to draw the target area. The size and shape of the regions of interest may be the same or different.
S117: Right-click on the target area to set a playback window combined with the target area.
S118: Select a play window to play the video in the target region when the trigger event occurs within the target region.

In embodiments of the present invention, the original playback image that is divided into regions of interest may be the entire playback image within the largest playback window on the display device, or multiple playback images that are played simultaneously within the largest playback window. It should be understood that there may be one or more images in the image of .
Embodiments of the invention are not limited to these.

At S120, the video playback device determines a first target region in which the trigger event occurs among the at least two target regions, and displays an image within the first target region in another playback window. , thereby improving the display effect of image details.

In embodiments of the present invention, the user can manually trigger events to determine the region of interest, or detect automatically generated trigger events to determine the region of interest. These are described below with reference to FIGS. 6 and 7, respectively.

Optionally, as shown in FIG. 6, the step of determining, among the at least two regions of interest, a first region of interest in which the triggering event occurs comprises the steps of: a.
S121: Determining the triggering action to be performed by the user on the region of interest in the original reproduced image.
Triggering actions include clicking, double-clicking, or selecting an area of interest.
S122: Determining the region of interest on which the trigger action is to be performed as the first region of interest.

Specifically, when a user watches a video and detects the occurrence of an event, the user can act on the client interface. For example, by performing a triggering action on a region of interest within the original playback image, such that the image within the region of interest where the event occurs is played back within a pre-designated playback window, or popped up independently. Displayed in the playback window. When events occur in multiple regions of interest, the user can trigger multiple windows for display. A trigger action is, for example, a click action, a double-click action, or an action to select a region of interest. Embodiments of the invention are not limited to these.

FIG. 7 shows another schematic flow chart of a method for determining a region of interest in which a trigger event occurs, according to an embodiment of the invention. Optionally, as shown in FIG. 7, the step of determining, among the at least two regions of interest, a first region of interest in which the triggering event occurs comprises the steps of: a.
S123: Acquire the coordinate metadata of the trigger event occurrence point in the original reproduced image.
S124: Determine the target region to which the trigger event occurrence point belongs as the first target region according to the coordinate metadata.

Specifically, for example, a user can pre-configure regions that require automatic event detection and configure event detection rules, such as motion detection or intelligent analysis detection. When an event occurs, the client software can determine a corresponding pre-configured region of interest according to the coordinate metadata of the trigger event occurrence point, so that the corresponding image is played back in the pre-designated playback window. or displayed in a separate pop-up playback window. When events occur in multiple regions of interest, the client software can launch multiple windows for display.

It will be appreciated that within an irregular intelligent analysis region, a trigger event may cover multiple regions of interest, and at this time, multiple regions of interest may be determined as the first region of interest in which the trigger event occurs. It should be. Embodiments of the invention are not so limited.

In embodiments of the present invention, a video playback device can determine whether a trigger event occurs within a region of interest by motion detection, intelligent analysis detection, or the like. A central server can also perform detection to determine if a triggering event occurs within the region of interest. And when the trigger event is detected, the central server can feed back the coordinate metadata of the trigger event occurrence point to the video playback device, so that the video playback device can generate the trigger event according to the coordinate metadata. A first region of interest can be determined. Embodiments of the invention are not so limited.

At S130, the video playback device obtains the decoded data of the first video image displayed within the first region of interest so that the first video image is played back in the designated playback window.
Optionally, in an embodiment of the present invention, as shown in Figure 8, obtaining decoded data of a first video image to be displayed in the first region of interest comprises the steps of: a.
S131: Acquire the decoded data of the original reproduced image.
S132: Determining the decoded data of the first video image according to the decoded data of the original reproduced image.

Specifically, for example, the video playback device receives events manually triggered by the user in the manner of clicks, double-clicks, toolbar button clicks, or shortcut keys. The apparatus intercepts the content of the data belonging to the region of interest from the decoded YUV data of the original playback window and, according to a preconfigured correspondence, replaces a portion of this content with the prespecified playback. It can be played within a frame (or a separate playback window can be displayed to play part of it). Multiple playback windows use the same YUV data source. Therefore, the device does not need to bring or add additional multiple video streams.

For example, assume that the resolution of the original reproduced image is Width×Height. For the target area, let StartX be the horizontal coordinate of the start point, StartY be the vertical coordinate of the start point, EndX be the horizontal coordinate of the end point, and EndY be the vertical coordinate of the end point. The YUV data of the original reconstructed image are in the array Org[Width×Height] and the YUV data of the region of interest are in Dst[ROIWidth×ROIHeight] where n is any point in the region of interest. . The YUV data within the region of interest can then be determined according to the following equations.
ROIWidth = EndX-StartX
ROIHeight = EndY-StartY
Dst[n] = Org[(Width*(StartY+n/ROIWidth)+StartX+n%ROIWidth)]
The division operation "/" indicates rounding to the nearest integer, and the symbol "%" indicates a REM operation.

At S140, the video playback device renders the decoded data of the first video image into the designated playback window for playback.

Specifically, the video playback device plays the first video image in a pop-up window, displays the first video image in a new playback window, or displays the first video image in the original playback window. A video image can be displayed and a digital zoom can be performed on the first video image to fit the size of the playback window. That is, in embodiments of the present invention, the designated window may be the largest playback window on the display device or a portion of the largest playback window. The specified window may be the current existing playback window or part of an existing playback window, or it may be a new popup or a newly created playback window. The designated window may be one window or multiple windows. Embodiments of the invention are not limited to these.

In an embodiment of the invention, optionally rendering the decoded data of the first video image into a designated playback window for playback comprises:
Rendering decoded data of the first video image in a designated playback window for playback in a zoomed-in manner, the designated playback window being larger than the first region of interest.

In an embodiment of the present invention, for example as shown in FIG. 9, rendering decoded data of a first video image into a designated playback window for playback comprises the following steps.
S142: Display an independent playback window.
S143: Draw the decoded data of the first video image into a separate playback window for playback.

In an embodiment of the present invention, the designated playback window is an independent pop-up playback window, and the independent pop-up playback window may be larger than the first region of interest, so that the first video image is zoomed-in. It should be understood that, although reproduced in a manner, embodiments of the present invention are not so limited. For example, the independent playback window may be smaller than or equal to the first region of interest.

In embodiments of the present invention, "B corresponding to A" indicates that B is related to A, and it should be understood that B can be determined according to A. However, it should also be understood that determining B according to A does not mean that B is determined according to A only, but that B may be determined according to A and/or other information.

It should be understood that in embodiments of the present invention, the sequential numbering of the aforementioned processes does not indicate the order of execution. The execution order of the processes should be determined according to their functions and their inherent logic, and should not impose any restrictions on the implementation of the embodiments of the present invention.

Therefore, the video playback method according to an embodiment of the present invention divides the original playback image into a plurality of target regions, and separately displays the image of the target region where the trigger event occurs. Thus, in one aspect, the user is able to observe more distinct image detail in the region of interest, and in another aspect, the user is able to track image detail in multiple regions of interest simultaneously, which greatly improve the user experience. In addition, embodiments of the present invention use the original decoded data to reconstruct the image of the region of interest, but without adding an additional video stream.

10 to 12B, the video playing method according to the embodiment of the present invention will now be described in detail.

As shown in FIG. 10, video playback method 200 may be performed by a video playback device.
And this device is, for example, a terminal or a client. The method 200 may have the following steps.
S201: Display the graphical user interface (GUI for short) of the client.
S202: Determine whether to set the playback image segmentation method. If it is determined to set the playback image segmentation method, the process proceeds to S203. Otherwise, the process proceeds to S204.
S203: Set the playback image division method. The process then proceeds to S204.
S204: Determine whether the user starts playing. If the user decides to start playing, the process goes to S205. Otherwise, the process proceeds to S201.
S205: Enable the network port.
S206: Receive a media stream, decode the media stream, and render the decoded media stream on a display device for display.
S207: Determine whether the user manually triggers the event. If the user decides to manually trigger the event, the process proceeds to S208. Otherwise, the process proceeds to S209.
S208: When the user decides to manually trigger the event, display the event occurrence area in the designated window in a zoom-in manner. The process then proceeds to S206.
S209: Determine whether the device automatically triggers the event. If it is determined that the device automatically triggers the event, the process proceeds to S210. Otherwise, the process proceeds to S211.
S210: When the device determines to automatically trigger an event, display the event occurrence area in a designated window in a zoom-in manner. The process then proceeds to S206.
S211: Determine whether the user finishes playing. If the user decides to finish playing, the process proceeds to S212. Otherwise, the process proceeds to S206.
S212: Determine whether the user closes the client. If the user decides to close the client, the process proceeds to S213. Otherwise, the process proceeds to S201.
S213: Clean up system resources. Then, video playback ends.

It should be understood that in embodiments of the present invention, the sequential numbering of the aforementioned processes does not indicate the order of execution. The execution order of the processes should be determined according to their functions and their inherent logic, and should not impose any restrictions on the implementation of the embodiments of the present invention.

FIG. 11A shows a schematic flow chart of a region-of-interest manually triggered playback method 300, according to one embodiment of the present invention. Method 300 may be performed by a video playback device. And this device is, for example, a terminal or a client. As shown in FIG. 11A, method 300 includes the following steps.
S301: Draw and play back each frame of the video image in the original playback window normally.
S302: Determine whether the user manually triggers the event. If the user decides to manually trigger the event, the process proceeds to S303. Otherwise, the process proceeds to S301.
S303: Obtain a region of interest where the user event is located.
S304: Check the playback window connected to the target area.
S305: Compute the YUV data of the video image covered by the region of interest for each frame of the video image.
S306: For each frame of the video image, draw the YUV data of the region of interest into the designated playback window for playback. For example, as shown in FIG. 12A, the entire playback window includes an original playback image window having the same size as the original playback image window and three designated playback windows, where the original playback image has 16 regions of interest. divided into An image of the region of interest where the manual trigger event occurs is then played back in a zoomed-in manner on one of the designated playback windows.
S307: Determine whether the user finishes playing. If the user decides to finish playing, the process proceeds to S308. Otherwise, the process proceeds to S305.
S308: Stop video playback. Then terminate the process.

FIG. 11B is a schematic flow chart of a playback method 400 in which regions of interest are automatically triggered by events, according to one embodiment of the present invention. The method 400 may have the following steps.
S401: Draw and play back each frame of the video image in the original playback window normally.
S402: Perform intelligent analysis to determine whether a trigger event occurs.
If it is determined that a trigger event will occur, the process proceeds to S403. Otherwise, the process proceeds to S401.
S403: Calculate the correspondence between the intelligent analysis area and the target area.
S404: Obtain a region of interest (or regions of interest) covered by the analysis event.
S405: Check the playback window connected to the target area.
S406: Compute the YUV data of the video image covered by the region of interest for each frame of the video image.
S407: For each frame of the video image, draw the YUV data of the region of interest into the designated playback window for playback. For example, as shown in FIG. 12B, the entire playback window includes an original playback image window having the same size as the original playback image window and three designated playback windows, where the original playback image has 16 regions of interest. divided into The image of the region of interest where the trigger event occurs is then played back in a zoomed-in manner on one of the designated playback windows.
S408: Determine whether the user finishes playing. If the user decides to finish playing, the process proceeds to S409. Otherwise, the process proceeds to S406.
S409: Stop video playback. Then terminate the process.

It should be understood that in embodiments of the present invention, the sequential numbering of the aforementioned processes does not indicate the order of execution. The execution order of the processes should be determined according to their functions and their inherent logic, and should not impose any restrictions on the implementation of the embodiments of the present invention.

Therefore, the video playback method according to an embodiment of the present invention divides the original playback image into a plurality of target regions, and separately displays the image of the target region where the trigger event occurs. Thus, in one aspect, the user is able to observe more distinct image detail in the region of interest, and in another aspect, the user is able to track image detail in multiple regions of interest simultaneously, which greatly improve the user experience.

The video playback method according to the embodiment of the present invention has been described in detail above with reference to FIGS. 1 to 12B. Below, the video playback terminal and system according to the embodiments of the present invention will be described in detail with reference to FIGS. 13 to 20. FIG.

FIG. 13 shows a schematic block diagram of terminal 500 according to an embodiment of the present invention. As shown in FIG. 13, the terminal 500
a segmentation module 510 configured to segment the original reproduced image into at least two regions of interest;
a first determining module 520 configured to determine a first target region in which a trigger event occurs among the at least two target regions selected by the segmentation module 510; an acquisition module 530 configured to acquire decoded data of a first video image displayed within the determined first region of interest;
and a playback module 540 configured to render the decoded data of the first video image acquired by the acquisition module 530 into a designated playback window for playback.

Therefore, the video playback terminal according to the embodiment of the present invention divides the original playback image into a plurality of target areas and separately displays the image in the target area where the trigger event occurs. Thus, in one aspect, the user is able to observe more distinct image detail in the region of interest, and in another aspect, the user is able to track image detail in multiple regions of interest simultaneously, which greatly improve the user experience.

It should be understood that in the embodiments of the present invention, the video playing terminal can not only play video files, but also play real-time video streams. Embodiments of the present invention are described based on an example in which a terminal plays a real-time video stream, but embodiments of the present invention are not limited thereto.

In an embodiment of the present invention, optionally, as shown in FIG. 14, terminal 500 may:
further comprising a second determining module 550 configured to determine correspondence between each region of interest of the at least two regions of interest and the designated playback window;
The playback module 540 converts the decoded data of the first video image acquired by the acquisition module 530 to the designated corresponding first region of interest for playback according to the correspondence determined by the second determination module 550 . and is further configured to draw to the playback window.

Optionally, in an embodiment of the present invention, the segmentation module 510 is further configured to segment the original reconstructed image into at least two target regions with an equal segmentation method or a free segmentation method.

In an embodiment of the invention, optionally, as shown in Figure 15, the first decision module 520:
a first determining unit 521 configured to determine a triggering action to be performed by a user on a region of interest in the original reproduced image, the triggering action being a click action, a double-click action or comprising the act of selecting a region of interest;
Furthermore, it comprises a second determining unit 522 adapted to determine the region of interest on which the triggering action determined by the first determining unit 521 is performed as the first region of interest.

In an embodiment of the present invention, optionally, as shown in Figure 16, the first decision module 520:
a first acquisition unit 523 configured to acquire the coordinate metadata of the trigger event occurrence point in the original playback image;
and a third determining unit 524 configured to determine the target region to which the trigger event occurrence point belongs as the first target region according to the coordinate metadata acquired by the first acquiring unit 523 .

In an embodiment of the invention, optionally, as shown in Figure 17, the acquisition module 530:
a second obtaining unit 531 configured to obtain the decoded data of the original reproduced image;
and a third determining unit 532 configured to determine the decoded data of the first video image according to the decoded data of the original reproduced image obtained by the second obtaining unit 531 .

Optionally, in an embodiment of the present invention, the playback module is further configured to render the decoded data of the first video image into a specified playback window for playback in a zoomed-in manner, and the specified The playback window is larger than the first region of interest.

In an embodiment of the present invention, optionally, as shown in FIG. 18, regeneration module 540:
a display unit 541 configured to display an independent playback window;
and a playback unit 542 configured to render the decoded data of the first video image into a separate playback window displayed by the display unit for playback.

That the video playback terminal 500 according to embodiments of the present invention may correspond to the video playback device in the previous embodiments of the present invention, and the operation and/or functions of these and other terminal 500 modules, respectively, from FIG. It should be understood that they are used to perform the corresponding processes of methods 100-400 in FIG. 12B, which are not repeated here for the sake of brevity.

Therefore, the video playback terminal according to the embodiment of the present invention divides the original playback image into a plurality of target areas and separately displays the image of the target area where the trigger event occurs. Thus, in one aspect, the user is able to observe more distinct image detail within the regions of interest, and in another aspect, the user is able to track image detail within multiple regions of interest simultaneously. , which greatly improves the user experience.

FIG. 19 is a schematic block diagram of a system 600 according to one embodiment of the invention. As shown in FIG. 19, system 600 includes:
a terminal 610 according to an embodiment of the invention;
a video capture system 620 configured to capture a video image and generate a media stream by encoding the video image;
a server 630 configured to obtain the media stream produced by the video capture system and provide the media stream to the terminal 620;
and a storage device 640 configured to store media streams obtained by the server 630 .

The terminal 610 included in the video playback system 600 according to embodiments of the present invention may correspond to the video playback terminal 500 in the previous embodiments of the present invention, and the operation of the modules in the above and other terminals 610 and/or functions are used to perform the corresponding processes of methods 100-400 in FIGS. Do not repeat.

Therefore, the video playback system according to embodiments of the present invention divides the original playback image into multiple regions of interest and separately displays the image of the region of interest where the trigger event occurs. Thus, in one aspect, the user is able to observe more distinct image detail within the regions of interest, and in another aspect, the user is able to track image detail within multiple regions of interest simultaneously. , which greatly improves the user experience.

Embodiments of the present invention further provide a video playback terminal. As shown in FIG. 20, terminal 700 comprises processor 710 , memory 720 and bus system 730 , processor 710 and memory 720 being interconnected through bus system 730 . Memory 720 is configured to store instructions, and processor 710 is configured to execute the instructions stored in memory 720 . Processor 710 divides the original reconstructed image into at least two regions of interest, determines a first region of interest among the at least two regions of interest in which the trigger event occurs, and displays in the first region of interest. and rendering the decoded data of the first video image into a designated playback window for playback.

Therefore, the video playback terminal according to the embodiment of the present invention divides the original playback image into a plurality of target areas and separately displays the image of the target area where the trigger event occurs. Thus, in one aspect, the user is able to observe more distinct image detail within the regions of interest, and in another aspect, the user is able to track image detail within multiple regions of interest simultaneously. , which greatly improves the user experience.

In an embodiment of the present invention, the processor 710 can be a Central Processing Unit (CPU for short), and the processor 710 can be any other general processor, a digital signal processor (for short). DSP), application-specific integrated circuit (ASIC for short), field programmable gate array (FPGA for short), or other programmable logic device, discrete gate or transistor logic device, It should be understood that they may be separate hardware components or the like. A common processor may be a microprocessor, or this processor may be any common processor, or the like.

Memory 720 , which may include read-only memory and random-access memory, provides instructions and data to processor 710 . A portion of memory 720 may also include non-volatile random access memory. For example, memory 720 may also store device type information.

In addition to data buses, bus system 730 may also include power buses, control buses, status signal buses, and the like. However, for clarity of explanation, all buses are shown as bus system 730 in the figure.

During implementation, the steps of the methods described above may be implemented through integrated logic circuitry in hardware within processor 710 or in the form of instructions in software. The steps of the methods disclosed in the embodiments of the present invention can be directly implemented by a hardware processor or by a combination of hardware and software modules within the processor. A software module may reside in any storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, and registers. The storage media is located in memory 720 . Processor 710 reads the information in memory 720 and in conjunction with its hardware implements the steps of the method. To avoid repetition, the details are not repeated here.

In one embodiment, optionally the processor 710 is further configured to determine a correspondence relationship between each region of interest in the at least two regions of interest and the designated playback window. The processor 710 rendering the decoded data of the first video image into the designated playback window for playback involves rendering the decoded data of the first video image to the first video image for playback according to the correspondence relationship. Including drawing to the specified playback window corresponding to the region of interest.

As an embodiment, optionally dividing the original reproduced image into at least two target regions by the processor 710 comprises dividing the original reproduced image into at least two target regions in an equal division method or a free division method. including doing

As an embodiment, optionally the processor 710 determining a first region of interest in which the trigger event occurs among the at least two regions of interest is performed by a user on the region of interest in the original reproduced image. determining a triggering action to be performed, the triggering action including a clicking action, a double-clicking action, or an action of selecting a target area; and further determining the target area on which the triggering action is performed as a first target area. Including deciding.

Optionally, in one embodiment, the processor 710 determining a first region of interest, among the at least two regions of interest, where the trigger event occurs is based on the coordinates of the point of occurrence of the trigger event in the original reconstructed image. obtaining metadata; and determining a region of interest to which the trigger event occurrence point belongs as a first region of interest according to the coordinate metadata.

In one embodiment, optionally obtaining the decoded data of the first video image displayed in the first region of interest by the processor 710 includes obtaining the decoded data of the original reproduced image; determining decoded data for the first video image according to decoded data for the original reconstructed image.

Optionally, in an embodiment, the terminal 700 is further configured to render the decoded data of the first video image in a specified playback window for playback in a zoomed-in manner, and the specified playback is performed. The window is larger than the first region of interest.

In one embodiment, terminal 700 optionally further comprises a display 740 . The processor 710 rendering the decoded data of the first video image into the designated playback window for playback includes displaying the independent playback window, the display 740 displaying the first video image. It is configured to render the decoded data into a separate playback window for playback.

Video playback terminal 700 according to embodiments of the present invention may correspond to video playback terminal 500 or terminal 610 in previous embodiments of the present invention, and operation and/or functionality of modules within these and other terminals 700 are used to perform the corresponding processes of methods 100-400 in FIGS. 1-12B, respectively, which are not repeated here for the sake of brevity.

Therefore, the video playback terminal according to the embodiment of the present invention divides the original playback image into a plurality of target areas and separately displays the image of the target area where the trigger event occurs. Thus, in one aspect, the user is able to observe more distinct image detail within the regions of interest, and in another aspect, the user is able to track image detail within multiple regions of interest simultaneously. , which greatly improves the user experience.

Those skilled in the art will realize that the units and algorithm steps can be implemented by electronic hardware, computer software, or a combination thereof, in combination with the examples described in the embodiments disclosed herein. obtain. To clearly state the compatibility between hardware and software, the above outlines the configuration and steps of each example according to function. Whether a function is performed by hardware or by software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods to implement the described functionality for each particular application, but this implementation should not be considered beyond the scope of the invention.

For convenience and concise description, to the detailed operating processes of the aforementioned systems, devices, and units, reference can be made to the corresponding processes in the method embodiments, and the details are here again. What is not described can be clearly understood by those skilled in the art.

It should be understood that in some of the embodiments presented in this application, the disclosed systems, devices, and methods may be implemented otherwise. For example, the described apparatus embodiment is merely exemplary. For example, the division of units is merely logical function division, and may be other divisions in actual implementation. For example, multiple units or components may be combined or incorporated into other systems, or some features may be ignored or not performed. In addition, mutual couplings or direct couplings or communication connections shown or described may be implemented through some interfaces. Indirect couplings or communicative connections between devices or units may be implemented electronically, mechanically, or otherwise.

Units described as separate parts may or may not be physically separate. And the parts shown as units may or may not be physical units and may be in one location or distributed over multiple network units. Part or all of the units here can be selected according to actual needs to achieve the solution objectives of the embodiments of the present invention.

In addition, the functional units in the embodiments of the present invention may be incorporated within one processing unit, each of the units may physically exist alone, or two or more processing units may exist. Units may be incorporated into one unit. Embedded units may be implemented in the form of hardware or in the form of software functional units.

When the embedded unit is embodied in the form of a software functional unit and sold or used as a stand-alone product, the embedded unit can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present invention essentially or the part involved in the prior art, or all or part of the technical solution can be implemented in the form of software products. . A computer software product is stored on a storage medium and includes a number of instructions for instructing a computer device (it may be a personal computer, a server, a network device, etc.), and is described in the embodiments of the present invention. perform all or part of the steps of the method The aforementioned storage medium is any medium capable of storing program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM for short), a random access memory (RAM), Also includes magnetic or optical disks.

The foregoing descriptions are merely specific embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any equivalent variation or replacement readily figured out by a person skilled in the art within the technical scope of the present invention shall fall within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

The invention can be used for video surveillance.

500 terminal 510 segmentation module 520 first determination module 530 acquisition module 540 playback module 600 system 610 terminal 620 video capture system 630 server 640 storage device

Claims (16)

receiving a live video stream;
dividing the live video stream into at least two regions of interest;
determining the first region of interest containing the triggering event by applying coordinate metadata identifying a first region of interest associated with the triggering event among the at least two regions of interest; When,
determining decoded data associated with the first region of interest from the live video stream;
rendering the decoded data in a separate playback window to allow viewing of a zoomed version of the first region of interest of the live video stream;
rendering the live video stream in a second playback window to allow viewing of the live video stream concurrently with the zoomed version of the first region of interest of the live video stream;
splitting the live video stream occurs prior to determining the first region of interest containing the trigger event;
Rendering the decoded data in the separate playback window to allow viewing of a zoomed version of the first region of interest of the live video stream comprises: zooming in on the decoded data; rendering the data to the independent playback window. 2. The method of claim 1, further comprising: determining a correspondence between each region of interest among the at least two regions of interest and the independent playback window. receiving from a server the trigger event and the coordinate metadata associated with the trigger event;
2. The step of claim 1, wherein the step of receiving the trigger event and the coordinate metadata associated with the trigger event comprises determining whether the trigger event has occurred with a motion detector. the method of. obtaining from a server the trigger event and the coordinate metadata associated with the trigger event;
The step of receiving the trigger event and the coordinate metadata associated with the trigger event includes determining whether the trigger event was caused by an automatically generated trigger event. The method of claim 1. 2. The method of claim 1, wherein said independent playback window is larger than said first region of interest. 2. The method of claim 1, wherein said independent playback window is smaller than said first region of interest. 2. The method of claim 1, wherein all of the at least two regions of interest together comprise less than the entire live video stream. 2. The method of claim 1, wherein at least two of the at least two regions of interest have unequal sizes. 2. The method of claim 1, wherein the step of dividing the live video stream into at least two regions of interest occurs in response to manual user interaction. of the at least two regions of interest, by applying coordinate metadata associated with the second trigger event identifying the second region of interest that includes the second trigger event; determining a region;
determining second decoded data associated with the second region of interest from the live video stream;
rendering said second decoded data in said separate playback window to allow viewing of a zoomed version of said second region of interest of said live video stream;
rendering the second decoded data in the separate playback window to allow viewing of a zoomed version of the second region of interest of the live video stream;
2. The method of claim 1, comprising drawing the second decoded data into the separate playback window in a zoomed -in manner. 2. The method of claim 1, wherein the trigger event satisfies an automatic event detection rule. a processor;
and a non-persistent computer-readable medium storing instructions that, when executed by said processor, implement the method of any of claims 1-11. A video playback terminal,
a segmentation module configured to segment a video stream into at least two regions of interest;
a first determining module configured to determine a first region of interest among the at least two regions of interest in which a trigger event occurs, the first determining module comprising: a first determining unit configured to determine a triggering action to be performed over a region of interest by the a first decision module;
a first acquisition module configured to acquire, from a central server, coordinate metadata of points of origin associated with the trigger event;
a second acquisition module configured to acquire decoded data for video in the first region of interest determined by the first determination module;
a playback module configured to render the decoded data acquired by the second acquisition module into a separate playback window;
a second playback module configured to render the video stream into a second playback window at the same time that the decoded data is rendered into the separate playback window;
the segmentation module is further configured to segment the video stream prior to the first determination module determining the first region of interest in which the trigger event occurs;
A video playback terminal, wherein the second playback module is further configured to render the decoded data into the separate playback window in a zoom-in manner. 14. The method of claim 13, further comprising: a second determining module configured to determine correspondence between each region of interest among the at least two regions of interest and the independent playback window. terminal. The playback module is
a display unit configured to display independent playback windows;
14. The terminal of claim 13, comprising: a playback unit configured to render the decoded data into the separate playback window displayed by the display unit. a video playback device;
a video capture system configured to capture and generate a media stream by encoding the video stream and responsive to a motion detector;
a server configured to obtain the media stream generated by the video capture system and to provide the media stream to the video playback terminal;
a storage device configured to store the media stream obtained by the server;
The video playback terminal is
a segmentation module configured to segment the video stream into at least two regions of interest;
a first determining module configured to determine a first region of interest among the at least two regions of interest divided by the dividing module, from the server a trigger caused by the motion detector; Obtaining coordinate metadata of an occurrence point in the media stream based on the event, and determining the target area to which the trigger event occurrence point belongs as the first target area according to the coordinate metadata. a first decision module, further configured to:
an acquisition module configured to acquire decoded data of the first region of interest determined by the first determination module;
a playback module configured to render the decoded data acquired by the acquisition module into a separate playback window;
a second playback module configured to render the video stream into a second playback window at the same time that the decoded data is rendered into the separate playback window;
the segmentation module is further configured to segment the video stream prior to the first determination module determining the first region of interest containing the trigger event;
A video playback system, wherein the second playback module is further configured to render the decoded data into the separate playback window in a zoom-in manner.

Images