CN114422735B - Video recorder, video data processing method and device and electronic equipment - Google Patents
Video recorder, video data processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN114422735B CN114422735B CN202210109292.7A CN202210109292A CN114422735B CN 114422735 B CN114422735 B CN 114422735B CN 202210109292 A CN202210109292 A CN 202210109292A CN 114422735 B CN114422735 B CN 114422735B
- Authority
- CN
- China
- Prior art keywords
- target video
- service
- video
- data
- video service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 243
- 238000000034 method Methods 0.000 claims description 106
- 230000008569 process Effects 0.000 claims description 76
- 239000000872 buffer Substances 0.000 claims description 40
- 238000006243 chemical reaction Methods 0.000 claims description 20
- 238000004891 communication Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 9
- 238000004148 unit process Methods 0.000 claims description 3
- 239000000523 sample Substances 0.000 claims 1
- 239000002699 waste material Substances 0.000 abstract description 66
- 238000010586 diagram Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 6
- RRAMGCGOFNQTLD-UHFFFAOYSA-N hexamethylene diisocyanate Chemical compound O=C=NCCCCCCN=C=O RRAMGCGOFNQTLD-UHFFFAOYSA-N 0.000 description 6
- 230000009467 reduction Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 239000002131 composite material Substances 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/907—Television signal recording using static stores, e.g. storage tubes or semiconductor memories
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The embodiment of the invention provides a video recorder, a video data processing method, a video data processing device and electronic equipment, which are applied to the technical field of data processing. The video recorder comprises a VI unit and at least one service processing unit; each service processing unit is used for processing video services; the VI unit is used for acquiring pre-configured service information; when receiving a video stream signal to be processed, converting the video stream signal into at least one path of video data; converting one path of video data corresponding to the target video service into target video data with video parameters required by the target video service, and writing the target video data of each target video service into a corresponding first storage position; the service processing unit corresponding to each target video service is used for acquiring data to be utilized aiming at the target video service and processing the data to be utilized according to the data processing mode corresponding to the target video service. Through the scheme, the resource waste in the video recorder can be reduced.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a video recorder, a video data processing method, a video data processing device, and an electronic device.
Background
Some video processing devices, such as video recorders (Digital Video Recorder, hard disk recorders), may implement multiple video services, which may include coded services, video output services, and so on. Video services refer to some specific processing of video data. The function of a DSP (Digital Signal Processing ) system of a Video recorder to implement a plurality of Video services depends on the VI (Video input) unit and VPE (Video Process Engine, image processing engine) unit that it contains.
The VI unit processes the received video stream signals and outputs one path of video data, wherein one path of video data output by the VI unit is required to be used as input data for any path of video service. However, there is a video service inconsistent with the video parameters of the one path of video data, which results in a problem of resource waste for the video service.
Disclosure of Invention
The embodiment of the invention aims to provide a video recorder, a video data processing method, a video data processing device and electronic equipment, so that resource waste aiming at video service in the video recorder is reduced. The specific technical scheme is as follows:
In a first aspect, an embodiment of the present invention provides a video recorder, including a video input VI unit and at least one service processing unit; each service processing unit is used for processing video services;
the VI unit is configured to obtain pre-configured service information, where the service information includes video parameters required by at least one target video service to be executed; when receiving a video stream signal to be processed, converting the video stream signal into at least one path of video data, wherein each path of video data corresponds to a target video service; for each target video service, converting one path of video data corresponding to the target video service into target video data with video parameters required by the target video service, and writing the target video data of each target video service into a first storage position corresponding to the target video service; the first storage position corresponding to each target video service is a storage position which is applied for the VI unit in advance and used for storing the data of the target video service;
and the service processing unit is used for acquiring data to be utilized for the target video service, and processing the data to be utilized according to a data processing mode corresponding to the target video service, wherein the data to be utilized is determined based on the target video data in the specified storage position corresponding to the target video service, and the specified storage position of each target video service is a first storage position corresponding to the target video service.
In a second aspect, an embodiment of the present invention provides a video data processing method, applied to a video recorder, where the method includes:
acquiring pre-configured service information, wherein the service information comprises video parameters required by at least one target video service to be executed;
when receiving a video stream signal to be processed, converting the video stream signal into at least one path of video data; wherein each path of video data corresponds to a target video service;
for each target video service, converting one path of video data corresponding to the target video service into target video data with video parameters required by the target video service;
aiming at each target video service, acquiring data to be utilized corresponding to the video service, and processing the data to be utilized according to a data processing mode corresponding to the target video service; wherein the data to be utilized is determined based on the target video data of the target video service.
In a third aspect, an embodiment of the present invention provides a video data processing apparatus, applied to a video recorder, including:
the information acquisition module is used for acquiring pre-configured service information, wherein the service information comprises video parameters required by at least one target video service to be executed;
The first conversion module is used for converting the video stream signal into at least one path of video data when receiving the video stream signal to be processed; wherein each path of video data corresponds to a target video service;
the second conversion module is used for converting one path of video data corresponding to each target video service into target video data with video parameters required by the target video service;
the data processing module is used for acquiring data to be utilized corresponding to each target video service, and processing the data to be utilized according to a data processing mode corresponding to the target video service; wherein the data to be utilized is determined based on the target video data of the target video service.
In a fourth aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;
a memory for storing a computer program;
a processor configured to implement the method steps of any of the second aspects when executing a program stored on a memory.
In a fifth aspect, embodiments of the present invention provide a computer-readable storage medium having stored therein a computer program which, when executed by a processor, implements the method steps of any of the second aspects.
The embodiment of the invention has the beneficial effects that:
the video recorder provided by the embodiment of the invention comprises a VI unit and at least one service processing unit, wherein each service processing unit is used for processing video service, the VI unit can convert video stream signals into at least one path of video data when receiving video stream signals to be processed, and for each target video service, convert one path of video data corresponding to the target video service into target video data with video parameters required by the target video service, write the target video data of each target video service into a first storage position corresponding to the target video service, and the service processing unit corresponding to each target video service can acquire the data to be utilized for the target video service and process the data to be utilized according to a data processing mode corresponding to the target video service. Because the VI unit can convert one path of video data for each target video service and convert one path of video data corresponding to each target video service into target video data with video parameters required by the target video service, each path of video data obtained by conversion can be ensured to be fully utilized, and therefore, the resource waste for the video service in the video recorder is reduced.
Of course, it is not necessary for any one product or method of practicing the invention to achieve all of the advantages set forth above at the same time.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are necessary for the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention and that other embodiments may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 (a) is a schematic diagram of a video service processing flow in a video recorder;
FIG. 1 (b) is a schematic diagram of a video recorder handling DDR bandwidth consumption of video traffic;
FIG. 2 is a schematic diagram of a video recorder according to an embodiment of the present invention;
FIG. 3 is a flowchart of a video data processing method according to an embodiment of the present invention;
FIG. 4 (a) is a schematic diagram illustrating a video data processing procedure according to an embodiment of the present invention;
FIG. 4 (b) is a schematic diagram showing DDR bandwidth consumption when a video recorder according to an embodiment of the present invention processes video services;
FIG. 5 is another flowchart of a video data processing method according to an embodiment of the present invention;
Fig. 6 is a schematic structural diagram of a video data processing apparatus according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Video recorders, such as DVR (Digital Video Recorder, hard disk video recorder), are a set of computer systems that perform image storage processing, and have functions of long-time video recording, audio recording, remote monitoring and control of images/voices. The Video recorder includes a VI (Video input) unit and a VPE (Video Process Engine, image processing engine) unit.
The VI unit is a hardware functional module for collecting video stream signals into video data of a specified type, and the VI unit may be composed of hardware and driving software, and the hardware may be a video collecting module. Wherein the video stream signal may be one type of digital signal and the specified type of video data may be another type of digital signal. The video stream signal carries image information which is collected to generate video data of a specified type. The video stream signal may be a digital video stream obtained by analog-to-digital conversion of an analog signal, for example, a BT656 video stream or a BT1120 video stream. BT656 defines a parallel hardware interface for transmitting a 4:2:2 YCbCr digital video stream, YCbCr being one of the color spaces commonly used for continuous processing of video in movies or in digital photographic systems, Y being the luminance (luminance) and Cb and Cr being the blue and red concentration offset components. While BT1120 is an HDTV (High Definition Television ) studio signal digital interface capable of encoding 4:4:4 and 4:4:2 video data into a video data stream with embedded synchronous timing reference codes. In addition, exemplary, the specified type of video data may be YUV (brightness, chromaticity, and density) type of video data, such as YUV420 format data, YUV422 format data, and the like; of course, the specified type of video data is not limited to YUV type of video data, and for example, the specified type of video data may be RGB (red, green, blue) format video data.
Specifically, the VI unit is configured to receive video stream signals through interfaces such as BT656 and BT1120 specified by technical alliance such as MIPI (Mobile Industry Processor Interface ), ITU-R (International Tecommunicat ion Union-R telecommunications, international telecommunications union-radio communication section), and convert the received video stream signals into video data, and write the converted video data into a buffer, and further the VI unit may process the video data in the buffer and write the processed video data into a specified memory area, for example: DDR memory area.
Before the processed video data is stored in the designated memory area, the VI unit may perform processing such as clipping (loop), horizontal vertical Scaling (Down Scaling), and horizontal vertical Flip (Mirror, flip) on the video data in the buffer, where the processing performed by the VI unit on the video data in the buffer may depend on the corresponding hardware unit and software driver. In addition, the VPE unit belongs to a functional module, and at least can perform image quality adjustment, such as noise reduction, sharpening, brightness adjustment, and the like, on the video data, and then compress the video data according to the resolution required by each service and output the video data, where the functional module can be implemented by a hardware+software mode or can also be implemented by software only. That is, the VPE unit supports that an input image is first subjected to image quality adjustment including noise reduction, sharpening, brightness adjustment, etc., and then scaled to a certain resolution and output through each output port; the VPE unit also contains functions of HDR (High Dynamic Range Imaging high dynamic range imaging), rotation, clipping, etc.; furthermore, the VPE unit relies on hardware, including a set of hardware resources within the SOC chip, such as decompression, clipping, noise reduction, de-interlacing, masking, etc., units, as well as software drivers.
Also, a VI unit in a video recorder has one VI channel (also referred to as VI chan or VI path), while a VPE unit may have multiple (e.g., four) VPE channels (VPE channel outputs). Because the VI unit only has one path of VI channel, in order to realize the multi-path video service at the same time, the VI unit transmits the video data to the VPE unit through one path of VI channel, and the VPE unit supports multiple paths of VPE channels, so that the VPE unit can output the received video data through multiple paths of VPE channels after receiving one path of video data transmitted by the VI, thereby ensuring that the multi-path video service can be realized at the same time. Wherein, the VI channel and the VPE channel are both logical concepts, and the VI unit has one path of VI channel: after receiving the video stream signal, the VI unit converts the video stream signal into one path of video data, stores the one path of video data in the buffer area, performs processing such as clipping, horizontal vertical shrinkage, horizontal vertical overturn and the like on the video data in the buffer area, and writes the processed one path of video data in a predetermined storage position, for example: a predetermined DDR memory area; the VPE unit has four VPE channels: the VPE unit reads video data from the preset storage position for each path of video service, and then respectively processes the read video data corresponding to the path of video service, wherein the processed video data is written into the storage position corresponding to the path of video service for each path of video service processing.
Therefore, the VI unit has only one VI channel, so that one video data output by the VI unit needs to be input data of multiple video services at the same time. However, since video parameters of video data required for each video service may Not be uniform, for example, video service 1 requires full-size and real-time (abbreviated RT) video data, in which the frame rate of real-time video data is the same as that of input data, and video service 2 requires full-size and non-real-time (Not real time, abbreviated NRT) video data, in which the frame rate of non-real-time video data is smaller than that of input video data. Therefore, in order to simultaneously meet the requirements of each path of video service, one path of video data output by the VI unit needs to be full-size and real-time video data. As can be seen, when the following video service does not necessarily need full-size and real-time video data, as in the video service 2, the video data output by the VI unit cannot be fully utilized by the following video service, which results in resource waste, that is, the video data output by the VI unit cannot be fully utilized by the control chip of the following video service, which exceeds the requirement of the processing module of the following video service, and results in resource waste.
Fig. 1 (a) is a schematic diagram of a video service processing flow in a typical video recorder. As shown in fig. 1 (a), the video service processing is implemented through interaction of a VI unit, a first-stage VPE unit located after the VI unit, and a second-stage VPE unit located at the rear end of the first-stage VPE unit. Four paths of VPE channels of the primary VPE unit and the secondary VPE unit are respectively represented by chn0, chn1, chn2 and chn3, each path of VPE channel is bound with a later-stage service, wherein the later-stage service bound by each VPE channel can be a control chip or a VPE unit of video service. It should be noted that "after" herein refers to a later order in the order of signal or data processing.
The above-mentioned first-stage VPE unit is responsible for processing large-size video services, including main stream coding, HDMI (High Definition Multimedia Interface ) or VGA (Video Graphics Array, video graphics array) large-screen preview, JPEG (Joint Photographic Experts Group ) coding, etc., and the second-stage VPE unit is responsible for processing small-size video services, including: subcode stream coding, CVBS (Composite Video Broadcast Signal ) preview, HDMI or VGA small picture preview, traffic streaming, etc.
The large-size video service refers to a video service with a required image size larger than a specified image size, and the small-size video service refers to a video service with a required image size not larger than a specified image size, wherein the specified image size can be determined according to practical requirements, for example, the video service with the required image size larger than 640x480 is used as the large-size video service, and the video service with the required image size not larger than 640x480 is used as the small-size video service.
The main stream codes are used for coding the video data which is transmitted and stored locally, and the sub stream codes are used for coding the video data which is transmitted in a network.
The above-described encoding-related services, such as main-stream encoding, JPEG encoding, sub-stream encoding, etc., are services that convert video data in a storage location into video data of a specified format, and JPEG encoding, for example, converts video data in a storage location into video data of a JPEG format.
The above-mentioned preview-related services (which may also be referred to as video output services), such as HDMI or VGA large-screen preview, CVBS preview, and HDMI or VGA small-screen preview, are services that present video data in a storage location on a display in accordance with image parameters required for the preview, wherein the image size of an image presented by the large-screen preview is larger than a specified image size, and the image size presented by the small-screen preview is not larger than the specified image size
The resource waste in the video recorder is described below in connection with a specific video service, where the video recorder is divided into a coding real-time device and a coding non-real-time device.
The video stream signal output by the video camera (which may be a digital signal or may be an analog signal) may be the maximum frame rate, wherein the video stream signal output by the video camera may be an analog signal, including at least one of an HDTVI (High Definition Transport Video Interface, high-definition image transmission interface) coaxial high-definition signal, an AHD (Analog High Definition, analog high-definition) signal, an HDCVI (High Definition Composite Video Interface, high-definition composite image interface) signal, and an analog signal such as a CVBS, and the video camera may also be an SDI (Serial Digital Interface, digital component serial interface) high-definition camera, and the video stream signal output by the video camera may be a digital signal, for example, an SDI signal, where the SDI signal may be converted into a BT656 or BT1120 video stream signal by some modules, for example, an FPGA is generally a module in a video recorder for signal acquisition, and converts the SDI signal into the BT or BT1120 video stream signal in the video recorder.
The minimum encoding frame rate of the encoding real-time device may be 1/2fps or less, or the like. The maximum coding frame rate of the coder in the coding real-time device is the same as the signal frame rate of the video stream signal, so that the coding real-time device has the capability of generating the video data with the same frame rate as the signal frame rate of the video stream signal by using the coder. Of course, the encoding real-time device may also encode video data with an encoder to generate video data with a frame rate less than the signal frame rate of the video stream signal, as desired. For example, if the video stream signal of the real-time apparatus 1 is encoded at a signal frame rate of 120fps, video data having a frame rate of 120fps may be encoded by an encoder according to a user's selection, or video data having a frame rate of 60fps may be encoded by an encoder.
The above-mentioned encoding non-real time apparatus refers to an encoding apparatus in which the upper limit of the frame rate of the video data generated by the encoder is limited, and the maximum encoding frame rate of the encoder in the encoding non-real time apparatus is smaller than the signal frame rate of the video stream signal, i.e., the encoding non-real time apparatus does not have the capability of encoding video data having the same generating frame rate as the signal frame rate. By way of example, if the signal frame rate of the video stream signal of the encoding non-real time device 2 is 120fps and the maximum encoding frame rate of the encoder in the encoding non-real time device 2 is 60fps, the encoding non-real time device can only encode video data having a frame rate of 60fps at maximum, and cannot encode video data having a frame rate of 120fps by the encoder.
It should be emphasized that, for both the encoding real-time device and the encoding non-real-time device, before the video data is generated by encoding with the encoder, the VI unit is required to receive the video stream signal, and output the video data with the same frame rate as the signal frame rate of the video stream signal, so that the encoder encodes the video data according to the configuration information to obtain the locally stored or network transmitted video. For the encoding real-time device, the encoder may encode the video data output by the VI unit according to a signal frame rate of the video stream signal, or may encode the video data output by the VI unit according to a frame rate smaller than the signal frame rate of the video stream signal. Whereas for a coded non-real time device the encoder can only encode the video data output by the VI unit at a frame rate that is smaller than the signal frame rate of the video stream signal.
As shown in table 1, the resource waste situation in the encoding real-time device and the encoding non-real-time device of different video services is given.
TABLE 1
| Video service | Encoding real-time equipment | Encoding non-real time device |
| Main code stream coding (maximum coding frame rate) | No resource waste | Has resource waste |
| Main code stream coding (non-maximum coding frame rate) | Has resource waste | Has resource waste |
| Subcode stream coding (maximum coding frame rate) | Has resource waste | Has resource waste |
| Subcode stream coding (non-maximum coding frame rate) | Has resource waste | Has resource waste |
| High definition preview (Multi-picture) | Has resource waste | Has resource waste |
The following describes each video service in table 1:
first video service: main code stream coding (maximum coding frame rate)
The main code stream coding (maximum coding frame rate) service is a service coded according to the maximum coding frame rate of the coding real-time equipment or the coding non-real-time equipment.
For the encoding real-time device, the maximum encoding frame rate of the encoder is the same as the signal frame rate of the video stream signal, so that when the encoding real-time device processes the main code stream encoding (maximum encoding frame rate) service, the encoder in the encoding real-time device encodes the video data output by the VI unit according to the signal frame rate of the video stream signal. The frame rate of the VI unit output video data is the same as the signal frame rate, so that each video frame in the VI unit output video data can be utilized by the encoder, thereby avoiding resource waste.
For a coding non-real time device, the maximum coding frame rate of the encoder is smaller than the signal frame rate of the video stream signal, so that when the coding non-real time device processes the main code stream coding (maximum coding frame rate) service, the encoder in the coding non-real time device will code the video data output by the VI unit at a frame rate smaller than the signal frame rate of the video stream signal. Since the frame rate of the VI unit output video data is the same as the signal frame rate of the video stream signal, that is, only a part of the video frames in the VI output video data can be utilized by the encoder in the encoding non-real time device, there is a waste of resources.
Second video service: main code stream coding (non-maximum coding frame rate)
Wherein, the main code stream coding (non-maximum coding frame rate) service is a service coded at a frame rate less than the maximum coding frame rate of the coding real-time device or the coding non-real-time device. It means that the encoder encodes the video data output by the VI unit at a frame rate smaller than the signal frame rate of the video stream signal when processing the main stream encoding (non-maximum encoding frame rate) service, regardless of the encoding real-time device or the encoding non-real-time device.
The frame rate of the video data output by the VI unit is the same as the signal frame rate of the video stream signal, that is, when the encoding real-time device or the encoding non-real-time device processes the main code stream encoding (non-maximum encoding frame rate) service, only part of the video frames in the video data output by the VI can be utilized by the encoder in the encoding real-time device or the encoding non-real-time device, so that the encoding real-time device or the encoding non-real-time device has resource waste when processing the main code stream encoding (non-maximum encoding frame rate) service.
Third video service: subcode stream coding (maximum coding frame rate)
The sub-stream coding (maximum coding frame rate) service is a service of coding according to the maximum coding frame rate of the coding real-time equipment or the coding non-real-time equipment, and the image size of the coded video is smaller than the image size of the video stream signal.
For subcode stream coding (maximum coding frame rate), it is necessary to analyze the resource waste from two dimensions, namely the frame rate and the image size.
For the frame rate dimension, the resource waste condition of the encoding real-time device or the encoding non-real-time device when processing the sub-code stream encoding (maximum encoding frame rate) service is the same as the resource waste condition of the encoding real-time device or the encoding non-real-time device when processing the main code stream encoding (maximum encoding frame rate) service, that is, in the frame rate dimension, the encoding real-time device has no resource waste, and the encoding non-real-time device has resource waste, and the relevant description is referred to the main code stream encoding (maximum encoding frame rate) part and is not repeated herein.
For the image size dimension, since for subcode stream coding (maximum coding frame rate) traffic the image size of the video after coding is smaller than the image size of the video stream signal, this means that in the image size dimension the image size of the video output by the encoder is smaller than the image size of the video stream signal, whether it is a real-time or non-real-time device.
The VI unit outputs video data having the same image size as the video stream signal, and the encoded real-time device or the encoded non-real-time device has an image size smaller than the image size of the video stream signal when processing the sub-stream encoding (maximum encoding frame rate) service, so that only a part of data in each video frame encoded by the encoder is utilized by the encoder, resulting in resource waste when processing the sub-stream encoding (maximum encoding frame rate) service in the image size dimension.
When the frame rate dimension and the image size dimension are integrated, resource waste exists in both the encoding real-time equipment and the encoding non-real-time equipment when processing sub-code stream encoding (maximum encoding frame rate) service.
Fourth video service: subcode stream coding (non-maximum coding frame rate)
The sub-stream coding (non-maximum coding frame rate) service is a service in which a frame rate smaller than a signal frame rate of a video stream signal is coded, and an image size of a coded video is smaller than an image size of the video stream signal.
Similarly, for subcode stream coding (not the maximum coding frame rate), it is also necessary to analyze the resource waste from two dimensions, namely the frame rate and the image size.
For the frame rate dimension, the resource waste condition of the encoding real-time device or the encoding non-real-time device when processing the sub-stream encoding (non-maximum encoding frame rate) service is the same as the resource waste condition of the encoding real-time device or the encoding non-real-time device when processing the main stream encoding (non-maximum encoding frame rate) service, that is, in the frame rate dimension, the resource waste exists in the encoding real-time device and the encoding non-real-time device, and the relevant description is referred to the main stream encoding (non-maximum encoding frame rate) part and is not repeated herein.
Regarding the image size dimension, the resource waste condition of the encoding real-time device or the encoding non-real-time device when processing the sub-code stream encoding (non-maximum encoding frame rate) service is the same as the resource waste condition of the encoding real-time device or the encoding non-real-time device when processing the sub-code stream encoding (maximum encoding frame rate) service, that is, the resource waste exists in the image size dimension, the encoding real-time device and the encoding non-real-time device, and the relevant description refers to the image size dimension part in the sub-code stream encoding (maximum encoding frame rate) and is not repeated herein.
When the frame rate dimension and the image size dimension are integrated, resource waste exists in both the encoding real-time equipment and the encoding non-real-time equipment when processing sub-code stream encoding (non-maximum encoding frame rate) service.
Fifth video service: high definition preview (Multi-picture)
The high-definition preview (multi-picture) is a service of performing high-definition preview output on multiple paths of video data, that is, displaying the multiple paths of video data with an image size larger than a specified image size. The image size of the high-definition preview picture of each path of video data for high-definition preview output is determined according to the number of multiple pictures, for example, for a 4-picture high-definition preview service, the image size of the high-definition preview picture of each path is 1/4 of the image size of the input image data. It can be seen that for high definition preview (multi-picture), the image size of the high definition preview image of each path of video data is smaller than the image size of the video stream signal.
The image size of the video data output by the VI unit is the same as the image size of the video stream signal, and the encoding real-time device or the encoding non-real-time device only outputs a high-definition preview picture with the image size smaller than the image size of the video stream signal when processing the high-definition preview (multi-picture), so that only part of the video data output by the VI unit is utilized by the encoder, and resource waste exists when the encoding real-time device and the encoding non-real-time device process the high-definition preview (multi-picture) service.
The above-mentioned waste of resources may be waste of DDR bandwidth and memory, and hereinafter, the DDR bandwidth is taken as an example, to further describe the consumption situation of DDR bandwidth when the video recorder processes video service.
Fig. 1 (b) is a schematic diagram of a video recorder processing the DDR bandwidth consumption of a video service.
In the first stage, the VI unit outputs full-size and real-time video data to the primary VPE unit.
In this process, the VI unit writes the full-size and real-time video data to the DDR, and the primary VPE unit reads the full-size and real-time video data from the DDR. It can be seen that in the first stage, the occupation amount of DDR bandwidth is the data amount of full-sized and real-time video data.
In the second stage, the first stage VPE unit outputs video data to the later stage service through four VPE channels.
In this process, each VPE channel of the primary VPE unit writes video data required by the secondary service into the DDR, and the secondary service reads the video data from the DDR.
It can be seen that, in the second stage, the occupation amount of the DDR bandwidth of each VPE channel of the first stage VPE unit is the data amount of the video data output by the VPE channel, where the data amount of the video data output by each VPE channel depends on the frame rate and the image size required by the post-stage service, that is, the data amount of the video data transferred from the front stage to the post-stage is equal to the image size of the single-frame image data multiplied by the frame rate required by the post-stage service.
And in the third stage, the second-stage VPE unit outputs video data to the later-stage service through four-way VPE channels.
In this process, each VPE channel of the two-stage VPE unit writes video data required by the post-stage service into the DDR, and the post-stage service reads the video data from the DDR.
In the third stage, the occupation amount of the DDR bandwidth of each VPE channel of the two-stage VPE unit is the data amount of the video data output by the VPE channel, where the data amount of the video data output by each VPE channel depends on the frame rate and the image size required by the post-stage service.
Based on the above, in order to reduce resource waste existing in a video recorder for video services, embodiments of the present invention provide a video recorder, a video data processing method, a video data processing device, and an electronic device.
The video recorder provided by the embodiment of the invention comprises a video input VI unit and at least one service processing unit; each service processing unit is used for processing video services;
VI unit, which is used to obtain the pre-configured service information, wherein the service information includes the video parameters needed by at least one target video service to be executed; when receiving a video stream signal to be processed, converting the video stream signal into at least one path of video data, wherein each path of video data corresponds to a target video service; for each target video service, converting one path of video data corresponding to the target video service into target video data with video parameters required by the target video service, and writing the target video data of each target video service into a first storage position corresponding to the target video service; the first storage position corresponding to each target video service is a storage position which is applied for a VI unit in advance and used for storing data of the target video service;
And the service processing unit is used for acquiring data to be utilized of the target video service, and processing the data to be utilized according to a data processing mode corresponding to the target video service, wherein the data to be utilized is determined based on the target video data in the designated storage position corresponding to the target video service, and the designated storage position of each target video service is a first storage position corresponding to the target video service.
In the above scheme provided by the embodiment of the invention, the VI unit can convert one path of video data for each target video service and convert one path of video data corresponding to each target video service into the target video data with video parameters required by the target video service, so that each path of video data obtained by conversion can be ensured to be fully utilized, and resource waste for the video service in the video recorder is reduced.
Furthermore, the VI unit can output video data for multiple video services at the same time, so that the VPE unit is not required to be relied on, and only one path of video data is required to be output for each video service which does not need to be subjected to image processing by the VPE unit, so that the resource consumption in the video recorder can be reduced. Wherein reducing the resource consumption in the video recorder comprises: the System On Chip (SOC) utilization rate in the video recorder is reduced, the DDR memory occupation in the video recorder is reduced, the DDR bandwidth occupation in the video recorder is reduced, and the CPU utilization rate in the video recorder is reduced.
The following describes a video recorder according to an embodiment of the present invention in detail with reference to the accompanying drawings.
As shown in fig. 2, a video recorder according to an embodiment of the present invention includes a video input VI unit 201 and at least one service processing unit 202; each service processing unit 202 is configured to process video services;
VI unit 201, configured to obtain pre-configured service information, where the service information includes video parameters required by at least one target video service to be executed; when receiving a video stream signal to be processed, converting the video stream signal into at least one path of video data, wherein each path of video data corresponds to a target video service; for each target video service, converting one path of video data corresponding to the target video service into target video data with video parameters required by the target video service, and writing the target video data of each target video service into a first storage position corresponding to the target video service; the first storage position corresponding to each target video service is a storage position which is applied for a VI unit in advance and used for storing data of the target video service;
and a service processing unit 202 corresponding to each target video service, configured to obtain data to be utilized for the target video service, and process the data to be utilized according to a data processing manner corresponding to the target video service, where the data to be utilized is determined based on the target video data in the specified storage location corresponding to the target video service, and the specified storage location of each target video service is the first storage location corresponding to the target video service.
Wherein, the target video service may be at least one of the following services:
1. a main code stream coding service; the main code stream coding service is a service for realizing main code stream coding through an encoder and is used for generating locally stored video data, and comprises main code stream coding with the maximum coding frame rate and main code stream coding with the non-maximum coding frame rate.
2. Sub-code stream coding service; the sub-code stream coding service is a service for realizing sub-code stream coding through an encoder and is used for generating video data for network transmission, and the sub-code stream coding service comprises sub-code stream coding with the maximum coding frame rate and sub-code stream coding with the non-maximum coding frame rate.
3. High definition preview service; the high-definition preview service is a service for performing high-definition display output on video data, for example, output in HDMI or VGA. The high-definition preview service refers to that the image size of the outputted or displayed image picture is larger than the specified image size. The high-definition preview service comprises single-picture high-definition preview and multi-picture high-definition preview, wherein the single-picture high-definition preview outputs a full-size single video frame, namely the size of the displayed image picture is the same as the image size of the video stream signal, and the multi-picture high-definition preview needs to combine video frames of multiple paths of video data into the single video frame for outputting.
For each video frame in a multi-picture high definition preview, image size scaling of each video frame is required in order to combine multiple video frames into a single video frame output. For example, for a 4-palace high definition preview service, the video frames of 4 paths of video need to be combined into a single video frame, and then the image size of the video frame of each path of video is one quarter of the full size.
4. Standard definition preview service; the standard-definition preview service is a service for performing standard-definition preview output on video data, for example, output by a CVBS. The standard definition preview service refers to that the image size of the outputted or displayed image frame is not larger than the specified image size. The single video frame image size output by the standard definition preview service is smaller than the image size of the video stream signal, and thus it is necessary to scale on the basis of the video stream signal.
5. A streaming service is fetched; the streaming service refers to a task of providing input data for a later-stage service, and the later-stage task may be an intelligent service, where the intelligent service needs to acquire video data from a video recorder to perform algorithm analysis.
6. JPEG coding service; the JPEG coding service comprises a single-step mode and a binding mode. The single-step mode refers to JPEG encoding only one piece of image data at a time, for example, acquiring only one frame of video data at a time, and JPEG encoding the acquired one frame of video data. The binding mode designates a time capture service, for example, multiple frames of video data are acquired at the speed of n frames/second, wherein n is equal to or greater than 0.
7. Non-homologous output traffic; wherein non-homologous output traffic refers to traffic that outputs video data from different video sources, which may be cameras, through HDMI, VGA or CVBS.
In this embodiment, the VI unit may generate at least one path of target video data for one path of video stream signal for outputting. It can be understood that when the number of paths of the target video data is multiple, the VI unit has a one-out-multiple processing function, that is, for one path of video stream signal, multiple paths of target video data are generated and output; when the number of the target video data is one, the VI unit has a processing function of outputting one, that is, one target video data is generated for one video stream signal and output.
It should be emphasized that each of the above-mentioned service processing units 202 is used for processing video services, and is merely used for indicating that each video service needs to be processed by one of the service processing units 202, and not limiting that each of the service processing units 202 is used for processing only one video service, that is, each of the service processing units 202 may be used for processing at least one video service. In addition, the video service mentioned in the embodiment of the present invention is essentially a processing manner of video data, for example, regarding the encoded video service, such as main stream encoding, sub stream encoding, etc., and is essentially a process of compressing video data in a specified storage location according to a specified format. Each service processing unit is a final execution unit of the corresponding video service; for coding services, such as a main code stream coding service, a subcode stream coding service, a JPEG coding service, etc., the service processing unit may be an encoder; for preview services, such as high definition preview service, standard definition preview service, and non-homologous output service, the service processing unit may be a corresponding controller or processing chip associated with the video interface, for example, for HDMI preview service, the service processing unit may be an HDMI controller associated with the HDMI interface, and for VGA preview service, the service processing unit may be a VGA conversion chip associated with the VGA interface.
And, the at least one target video service to be executed may be a video service configured to be executed from among video services executable by the video recorder. For example, video services executable by a video recorder include: video service 1, video service 2, and video service 3, and video service configured to be executed includes only video service 1 and video service 2, video service 1 and video service 2 are target video services to be executed. It will be appreciated that, by way of example, in one implementation, at least one video service may be configured by an associated person via a GUI (Graphical User Interface ) as video services to be performed, each video service configured to be performed may serve as a target video service to be performed.
In this embodiment, the VI unit may obtain preconfigured service information, where the preconfigured service information includes video parameters required by at least one target video service to be executed. The at least one target video service to be executed may be distinguished by service identifiers, where the pre-configured service information may specifically include a correspondence between each service identifier and a video parameter, so that the video parameter required by the target video service with each service identifier may be obtained.
Optionally, the service information may further include a first storage location corresponding to at least one target video service to be executed, so that after the VI unit obtains the pre-configured service information, the VI unit may determine at least one target video service to be executed, video parameters required by each target video service, and the first storage location corresponding to each video service at the same time. The first storage location corresponding to each target video service is a storage location applied for VI units in advance and used for storing data of the target video service, for example, may be a storage area specified in the DDR, and the storage area may be an offset address in the DDR. Of course, the first storage location corresponding to each target video service may be acquired independently of the service information. Optionally, the VI unit may acquire, before writing the target video data of each target video service into the first storage location corresponding to the target video service, location information of the first storage location applied by the upper layer application for each target video service. The upper layer application may be an application layer program, such as application software for controlling a video data processing procedure of the VI unit.
Alternatively, the video parameters required for each of the target video services described above may include a specified frame rate and/or a specified image size, which may also be referred to as a specified image resolution, i.e., the image size is characterized by the image resolution. For example, the video parameters of video service 1 are: the frame rate is 30fps and the image size is 1024×768. It will be appreciated that the video parameters also include other parameters such as data format. Illustratively, the data format is used to specify the manner in which the Y/U/V components of the video data are arranged in memory, e.g., the data formats include YUV420, YUV422, YUV444, and RAW (RAW) formats, etc. The YUV is a color coding method, where "Y" represents brightness, and "U" and "V" represent chromaticity, where "U" and "V" are used to describe image color and saturation, and are used to specify the color of a pixel. In YUV420 format, the data amount of one frame of image data=w×h×3/2. In the YUV422 format, the data amount of one frame of image data=w×h×2, where w is the width of the image and h is the height of the image.
It will be appreciated that the video parameters required for each target video service may be determined in accordance with the requirements of the video service. In an optional video parameter determining manner, the video parameter of the video service may be determined through a subsequent device associated with a service processing unit of the video service, for example, the video service is a high-definition preview service, the subsequent device associated with the service processing unit is a display 1, and the resolution of the display is 1024×768, where the video parameter of the high-definition preview service may be determined to include an image size 1024×768. In another alternative manner of determining video parameters, the video parameters of the video service may be configured by the relevant personnel via the GUI. In this case, the service information may be configured by the relevant person through a GUI of an upper layer application, and the upper layer application may issue video parameters configured by the relevant person to the functional modules such as the VI unit, the service processing unit, and the like.
After acquiring the pre-configured service information, the VI unit may receive a video stream signal to be processed. The video stream signal to be processed received by the VI unit may be a video stream signal collected by a camera, and the video stream information collected by the camera may also be referred to as a video stream signal.
Alternatively, the VI unit may receive the video stream signal sent by the camera in real time, as a video stream signal to be processed. Alternatively, the VI unit may read the video stream signal from a target storage location, which may be a video stream signal transmitted by a camera stored in advance, as the video stream signal to be processed. In one implementation, the VI unit may receive the video stream signal to be processed through interfaces such as BT656 and BT1120 specified by the technical alliance protocols such as MIPI, ITU-R, etc.
After receiving the video stream signal to be processed, the VI unit may convert the video stream signal into at least one path of video data, where each path of video data corresponds to a target video service. Illustratively, when the target video service to be performed is 3, then the VI unit may convert the video stream signal into 3-way video data.
In one implementation, converting a video stream signal into video data, and writing the converted video data into at least one predetermined buffer area respectively to obtain at least one path of video data; wherein each predetermined buffer area uniquely corresponds to a target video service.
The at least one path of target video service includes a video service 1, a video service 2 and a video service 3, wherein a predetermined buffer area of the video service 1 is a buffer area 1, a predetermined buffer area of the video service 2 is a buffer area 2, and a predetermined buffer area of the video service 3 is a buffer area 3. After receiving the video stream signal, the VI unit may convert the video stream signal into video data, and then write the converted video data into the buffer 1, the buffer 2, and the buffer 3, respectively.
After converting the video stream signal into at least one path of video data, the VI unit may convert, for each target video service, one path of video data corresponding to the target video service into target video data having video parameters required for the target video service.
Optionally, the video parameters may include a specified frame rate and/or a specified image size, and at this time, the VI unit may process, for each target video service, one path of video data corresponding to the target video service according to the specified frame rate and/or the specified image size required by the target video service, to obtain target video data having the video parameters required by the target video service.
Optionally, when the video parameters required by a target video service include a specified frame rate, one path of video data corresponding to the target video service may be converted into target video data having the specified frame rate required by the target video service. The specific conversion manner will be described in detail in the following embodiments, and will not be described in detail here. It can be understood that if the specified frame rate is consistent with the frame rate of one path of video data corresponding to the target video service, processing is not required, and if the specified frame rate is inconsistent with the frame rate of one path of video data corresponding to the target video service, frame rate adjustment is required for the path of video data according to the specified frame rate required by the path of video service.
When the video parameters required by a target video service include a specified image size, one path of video data corresponding to the target video service can be converted into target video data with the specified image size. It will be appreciated that if the specified image size is consistent with the image size of the video data corresponding to the target video service, then no processing is required, and if the specified image size is inconsistent with the image size of the video data corresponding to the target video service, then the video frames in the video data need to be scaled according to the specified image size required by the video service.
It can be understood that when the video parameters required by a target video service include a specified frame rate and a specified image size, in one implementation manner, each video frame in one path of video data corresponding to the target video service can be scaled in size, and then the scaled video data is further subjected to frame extraction processing based on the specified frame rate; or in another implementation manner, the video frames to be utilized needed in the path of video data corresponding to the target video service can be determined in advance based on the designated frame rate, and then only the size of the video frames to be utilized is scaled.
Optionally, in the case that the VI unit converts the video stream signal into video data and writes the converted video data into at least one predetermined buffer area respectively to obtain at least one path of video data, the VI unit may convert, for each target video service, the video data in the predetermined buffer area corresponding to the target video service into target video data having video parameters required by the target video service.
In the embodiment of the present invention, the VI unit may perform processing such as cropping (Crop), horizontal vertical Scaling (Down Scaling), and/or horizontal vertical Flip (Mirror, flip) on the video data in the predetermined buffer, in addition to performing processing such as frame rate conversion and/or image size Scaling on the video data in the predetermined buffer.
After obtaining the target video data of each target video service, the VI unit may write the target video data of each target video service into a first storage location corresponding to the target video service. The first storage position corresponding to each target video service is a storage position which is applied for the VI unit in advance and used for storing the data of the target video service.
As before, the first storage location corresponding to the target video service may be recorded in the pre-configured service information, and at this time, after the service information is obtained, the VI unit may obtain the first storage location corresponding to each target video service. Alternatively, in another implementation, the VI unit may obtain the location information of the first storage location applied by the upper layer application for each target video service, where the VI unit may actively request the upper layer application, or the upper layer application actively issues to the VI unit.
After each target video data of the VI unit is written into the first storage location corresponding to the target video service, the service processing unit 202 corresponding to each target video service may acquire data to be utilized for the target video service, and process the data to be utilized according to the data processing manner corresponding to the target video service.
The data to be utilized is determined based on the target video data in the designated storage locations corresponding to the target video service, and the designated storage location of each target video service is the first storage location corresponding to the target video service.
In one implementation, the service processing unit 202 corresponding to each target video service may read, from the designated storage location corresponding to the target video service, the target video data of the target video service as the data to be utilized for the target video service.
Optionally, the service processing unit 202 corresponding to each target video service is further configured to read, from a designated storage location corresponding to the target video service, target video data of the target video service, and obtain, before the target video data is to be utilized for the target video service, location information of a first storage location applied by an upper layer application for the target video service.
For each service processing unit 202 corresponding to each target video service, the service processing unit may receive, in advance, the first storage location allocated to the target video service and sent by the upper layer application, and further in a subsequent operation process, the service processing unit may read data from the first storage location.
After the VI unit writes the target video data of the target video service into the first storage location, the service processing unit may read the target video data from the first storage location, where the target video data in the first storage location is the data to be utilized of the target video service.
In addition, the data processing modes corresponding to different target video services are different, and the modes for processing the data to be utilized by the service processing units corresponding to different target video services are different. The embodiment of the invention does not limit the specific implementation mode of processing the data to be utilized according to the data processing mode corresponding to the target video service by the service processing unit.
In the above scheme provided by the embodiment of the invention, the VI unit can convert one path of video data for each target video service, and convert one path of video data corresponding to each target video service into the target video data with video parameters required by the target video service, so that each path of video data obtained by conversion can be ensured to be fully utilized, and resource waste in the video recorder is reduced.
As shown in fig. 2, the video recorder of an embodiment of the invention may further comprise an image processing engine VPE unit 203.
The VPE unit 203 may read target video data of any target video service from a designated storage location corresponding to the target video service when first indication information corresponding to the target video service is detected, perform pre-configuration processing on the video data, and write the processed target video data into a second storage location corresponding to the target video service.
The first indication information corresponding to each target video service comprises: information characterizing the VPE units contained in each unit involved in executing the target video service.
When related personnel configure a target video service to be executed through a GUI, an upper layer application can configure each unit involved in executing the target video service, and when each unit involved in executing the target video service contains a VPE unit, the upper layer application can issue first indication information to the VPE unit.
When the first indication information corresponding to the target video service is obtained, the VPE unit can read the target video data of the target video service from the designated storage position corresponding to the target video service, perform pre-configuration processing on the video data, and write the processed target video data into the second storage position corresponding to the target video service.
In one implementation, the upper layer application may further send a first storage location and a second storage location corresponding to the target video service to the VPE unit, where the second storage location is a storage location that is applied by the upper layer application in advance for the VPE unit and is used for storing data of the target video service. After receiving the first storage location and the second storage location sent by the upper layer application, the VPE unit may determine, based on the acquired first storage location, a location where the data to be read is located, and may determine, based on the acquired second storage location, a storage location where the data is subjected to the pre-configuration processing. Optionally, the upper layer application may write the first storage location and the second storage location corresponding to the target video service into the first indication information, and issue the first indication information to the VPE unit at the same time, or may send the first storage location and the second storage location corresponding to the target video service to the VPE unit separately.
Under the condition that first indication information corresponding to any target video service is detected, the VPE unit can read target video data corresponding to the target video service from a first storage position corresponding to the target video service, perform pre-configuration processing on the read target video data, and finally write the processed target video data into a second storage position corresponding to the target video service. The pre-configuration processing comprises the following steps: one or more of image size scaling, video data duplication, adjustment for specified image parameters; wherein the specified image parameter is a parameter for characterizing the image quality. Illustratively, the specified image parameters may be: brightness, resolution, noise, etc. It should be noted that, the total number of at least one path of video data converted by the VI unit from the video stream signal may be smaller than the total number of paths of the target video service, at this time, the VPE unit may copy the video data, generate multiple paths of video data from one path of video data, and further, after performing corresponding processing on the multiple paths of video data, perform corresponding processing by the service processing unit. In addition, the pre-configuration process performed by the VPE unit may be a process other than the video parameters required for the target video service, i.e. the VI unit itself has the following capabilities: converting the video data into target video data with video parameters required by the target video service without VPE assistance; of course, the pre-configuration process performed by the VPE unit may also involve the processing of the video parameters required for the target video service, provided that the VI unit itself does not have the capability to convert the video data into target video data having the video parameters required for the target video service, as well.
After determining that each unit involved in executing the target video service includes a VPE unit, the upper layer application may also send the first indication information and the second storage location corresponding to the target video service to the service processing unit 202 corresponding to the target video service. At this time, the service processing unit 202 corresponding to the target video service reads the target video data of the target video service from the second storage location corresponding to the target video service as the data to be utilized for the target video service when detecting the first indication information corresponding to the target video service.
When the upper layer application determines that the units involved in executing the target video service do not include the VPE unit, the second instruction information corresponding to the target video service and the first storage location may be sent to the service processing unit 202 corresponding to the target video service, where the second instruction information corresponding to each target video service includes: information characterizing that no VPE unit is included in each unit involved in executing the target video service.
At this time, the service processing unit 202 corresponding to each target video service may read, when the second instruction information corresponding to the target video service is detected, the target video data of the target video service from the designated storage location corresponding to the target video service as the data to be utilized for the target video service.
Optionally, in one implementation, the first indication information corresponding to each target video service is: set in the case that the VI unit does not have the capability to process the video data required to obtain the target video service;
the second type of indication information corresponding to each target video service is: set in the case that the VI unit has the capability to process the video data required to obtain the target video service.
It should be noted that, the VI unit may not have the capability of processing video data required for obtaining some target video services, that is, for some video services, the VI unit may not be able to process video data consistent with the video parameters required for the target video service, where, by way of example, the video parameters required for the target video service a include a specified image size of 1/8 of the image size of the video stream signal, and the maximum scaling factor of the VI unit is 1/4, so that, after scaling the video data corresponding to the target video service a, the VI unit may only obtain the target video data with the scaling factor of 1/4, and may not obtain the video data with the scaling factor of 1/8 required for the target video service a.
In this case, the VPE unit or the service processing unit corresponding to the target video service must be required to further process the target video data.
In this case, if the VPE unit is required to further process the target video data, it is explained that the VPE unit is included in each unit involved in executing the target video service, and thus the first instruction information needs to be set.
In contrast, in the case where the VI unit has the capability of processing video data required for obtaining some target video service, the VPE unit may not be included in each unit involved in the target video service, so that the second instruction information may be set.
After acquiring the data to be utilized for the target video service, the service processing unit 202 corresponding to each target video service may process the data to be utilized according to the data processing manner corresponding to the target video service.
The data processing modes corresponding to different target video services are different, and for the coding services, for example, the main code stream coding service, the subcode stream coding service, the JPEG coding service and the like, after the data to be utilized for the target video service is obtained, the encoder corresponding to each coding service can perform coding processing on the data to be utilized for the target video service according to a specified format.
In this embodiment, the VI unit may convert one path of video data for each target video service, and convert one path of video data corresponding to each target video service into target video data having video parameters required by the target video service, so as to ensure that each path of video data obtained by conversion may be fully utilized, thereby reducing resource waste existing in the video recorder for the video service. Further, by means of the VPE unit, further image processing can be performed on each path of video data output by the VI unit, thereby generating service data required for each video service.
On the basis of the video recorder provided by the embodiment of the invention, the embodiment of the invention also provides a video data processing method which is applied to the video recorder, and optionally can be applied to the video recorder provided by the embodiment of the invention.
As shown in fig. 3, a video data processing method according to an embodiment of the present invention is applied to a video recorder, and includes the steps of:
s301, obtaining pre-configured service information, wherein the service information comprises video parameters required by at least one target video service to be executed;
the VI unit in the video recorder may obtain the pre-configured service information.
S302, when a video stream signal to be processed is received, converting the video stream signal into at least one path of video data; wherein each path of video data corresponds to a target video service;
the VI unit in the video recorder may convert the video stream signal into at least one path of video data when receiving the video stream signal to be processed.
S303, converting one path of video data corresponding to each target video service into target video data with video parameters required by the target video service;
the VI unit in the video recorder can convert, for each target video service, one path of video data corresponding to the target video service into target video data having video parameters required by the target video service.
S304, aiming at each target video service, acquiring data to be utilized corresponding to the video service, and processing the data to be utilized according to a data processing mode corresponding to the target video service; wherein the data to be utilized is determined based on the target video data of the target video service.
The service processing unit in the video recorder can acquire data to be utilized corresponding to each target video service, and process the data to be utilized according to a data processing mode corresponding to the target video service.
In the video access method according to the embodiment of the present invention, the description is relatively simple because the video access method corresponds to the video recorder, and the description of the video recorder according to the present invention is referred to.
Optionally, in an implementation manner, the obtaining data to be utilized corresponding to the video service may include:
and pre-configuring target video data of the video service to obtain data to be utilized corresponding to the video service.
The pre-configuration processing comprises the following steps: one or more of image size scaling, video data duplication, adjustment for specified image parameters; wherein the specified image parameter is a parameter for characterizing the image quality.
In an embodiment, S302 may include:
converting the video stream signal into video data, and writing the converted video data into at least one preset buffer zone respectively to obtain at least one path of video data; wherein each predetermined buffer area uniquely corresponds to a target video service;
at this time, the step S303 may include:
for each target video service, converting the video data in the preset buffer zone corresponding to the target video service into target video data with video parameters required by the target video service.
In the video data processing method according to the embodiment of the present invention, the description is relatively simple because the video data processing method corresponds to the video recorder, and the description of the video recorder according to the present invention is referred to in the relevant points.
In an embodiment, the video parameters include a specified frame rate and/or a specified image size, and for each target video service, processing one path of video data corresponding to the target video service according to the specified frame rate and/or the specified image size required by the target video service to obtain target video data with the video parameters required by the target video service.
Based on the above embodiment, the embodiment of the present invention further provides another video data processing method, where the process of processing, according to the specified frame rate required by the target video service, one path of video data corresponding to the target video service may include steps 1 to 2:
step 1, determining a video frame to be extracted from one path of video data corresponding to the target video service according to a specified frame rate required by the target video service;
when the designated frame rate required by the target video service is the same as the frame rate of one path of video data corresponding to the target video service, each video frame of one path of video data corresponding to the target video service can be determined as a video frame to be extracted.
When the specified frame rate required by the target video service is smaller than the signal frame rate of the video stream signal, the method only needs to indicate that part of video frames of one path of video data corresponding to the target video service are needed, so that the video frames to be extracted can be determined from one path of video data corresponding to the target video service according to the specified frame rate required by the target video service.
In the implementation manner of determining the video frames to be extracted, the inter-frame time length corresponding to the appointed frame rate required by the target video service can be calculated, and then the video frames to be extracted are determined from one path of video data corresponding to the target video service according to the inter-frame time length.
Alternatively, a video frame may be determined from the video data corresponding to the target video service as the video frame to be extracted, for each inter-frame duration.
The inter-frame time length corresponding to the specified frame rate required by the target video service is time length between adjacent video frames at the specified frame rate required by the target video service, and in general, the time length corresponding to the specified frame rate required by the target video service is:
where T is a unit time, e.g., 1000ms, F is a specified frame rate required by the target video service, e.g., 30fps, and T is a duration corresponding to the specified frame rate required by the target video service.
After determining the inter-frame time length corresponding to the designated frame rate required by the target video service, a video frame to be extracted can be determined at intervals of one inter-frame time length.
In order to improve the accuracy of video frame determination, a time stamp can be added to each video frame of one path of video data corresponding to the target video service, and then a frame of video frame to be extracted is determined at intervals of inter-frame time based on the time stamp of each video frame.
In the implementation manner, the video frames to be extracted can be conveniently and efficiently determined from one path of video data corresponding to the target video service through the inter-frame time length corresponding to the appointed frame rate required by the target video service.
In another implementation manner of determining video frames to be extracted, a ratio of a signal frame rate of a video stream signal to a specified frame rate required by the target video service may be calculated as a frame interval parameter, so that for each video frame in video data corresponding to the target video service, whether the video frame is a video frame to be extracted is determined based on the frame interval parameter.
The ratio of the signal frame rate of the video stream signal to the specified frame rate required by the target video service is:
where a is a frame interval parameter, srcfps is a signal frame rate of a video stream signal, and dstfps is a specified frame rate required for the target video service.
After the frame interval parameter is calculated, for each video frame in the path of video data corresponding to the target video service, whether the video frame is a video frame to be extracted can be determined based on the frame interval parameter.
Optionally, each frame of video in one path of video data corresponding to the target video service has a frame number, and the frame numbers of each frame form an arithmetic sequence, for example {1,2,3,4,5, …, n-1, n }, or {2,4,6,8, …,2 (n-1), 2n }.
At this time, the determining whether the video frame is a video frame to be extracted based on the frame interval parameter includes steps 11-13:
Step 11, determining the frame number of the video frame and the target number of the video frame to be extracted in one path of video data corresponding to the target video service before the video frame;
for each frame of video frame in one path of video data corresponding to the target video service, the frame sequence number of the frame can be determined first, and the number of targets serving as video frames to be extracted in one path of video data corresponding to the target video service is determined before the video frame. The video frame preceding the video frame may be a video frame with a frame number smaller than that of the video frame, for example, frames of each frame of video in one path of video data corresponding to the target video service need to form an arithmetic sequence {1,2,3,4,5, …, n-1, n }, and for a video frame with a frame number of 4, the video frame preceding the video frame includes a video frame 1, a video frame 2, and a video frame 3.
Step 12, calculating the product of the frame interval parameter, the tolerance of the arithmetic sequence and the target number;
after determining the frame interval parameter, the product of the frame interval parameter, the tolerance of the sequence of arithmetic differences, and the target number may be calculated as follows:
C＝A×Δ×S
where C is the product, Δ is the tolerance of the arithmetic sequence, and S is the target number.
And step 13, if the product is smaller than the frame number of the video frame, judging the video frame as the video frame to be extracted.
If the product is smaller than the frame number of the video frame, the video frame is judged to be the video frame to be extracted, otherwise, if the product is smaller than the frame number of the video frame, the video frame is discarded.
By way of example, assuming a signal frame rate of 30fps for the video stream signal, the specified frame rate required for the target video service is 15fps. The frame number of each video frame of one path of video data corresponding to the target video service is {1,2,3,4,5, …, n-1, n }.
The ratio of the signal frame rate of the video stream signal to the specified frame rate required for the target video service is:
the tolerance delta of the frame serial numbers of each frame of one path of video data corresponding to the target video service forming an arithmetic sequence is 1, and then the following steps are included:
C＝2×1×S＝2S
for the video frame 1 with the frame number of 1 in the video data corresponding to the target video service, since the number of the targets serving as the video frames to be extracted in the video data corresponding to the target video service is 0 before the video frame 1, the product C is 0, and the number of the targets is smaller than the number of the frames 1, which is required to be 1, the video frame 1 is the video frame to be extracted.
For the video frame 2 with the frame number of 2, since the number of the video frames to be extracted in the path of video data corresponding to the target video service is 1 (video frame 1) before the video frame 2, the product C is 2 and is not less than the frame number 2 of the video frame 2, and the video frame 2 is the video frame to be discarded.
For the video frame 3 with the frame number 3, since the number of the video frames to be extracted in the path of video data corresponding to the target video service is 1 (video frame 1) before the video frame 3, the product C is 2 and is smaller than the frame number 3 of the video frame 3, and the video frame 3 is the video frame to be extracted.
For the video frame 4 with the frame number of 4, since the number of the video frames to be extracted in the path of video data corresponding to the target video service is 2 (video frame 1 and video frame 3) before the video frame 4, the product C is 4 and is not less than the frame number of 4 of the video frame 4, and the video frame 4 is the video frame to be discarded.
For the video frame 5 with the frame number of 5, since the number of the video frames to be extracted in the path of video data corresponding to the target video service is 2 (video frame 1 and video frame 3) before the video frame 5, the product C is 4, which is smaller than the frame number of 5, and the video frame 5 is the video frame to be extracted.
Similarly, in each video frame of one path of video data corresponding to the target video service, a frame can be determined as a video frame to be extracted every other frame.
In the implementation manner, the video frame to be extracted can be accurately determined from one path of video data corresponding to the target video service through the frame interval parameter.
And 2, extracting the determined video frames to obtain video data with a frame rate of a target frame rate as target video data aiming at a target video service.
After determining the video frames to be extracted, the determined video frames may be extracted to obtain video data conforming to the specified frame rate required by the target video service.
The video data processing method of the embodiment of the invention can reduce the resource waste, further, the video frames to be extracted are determined from one path of video data corresponding to the target video service according to the specified frame rate required by the target video service, and the determined video frames are extracted to obtain the video data meeting the specified frame rate required by the target video service, thereby providing an implementation basis for reducing the resource waste in DVR.
The following describes a video processing procedure of the video recorder according to the embodiment of the present invention in conjunction with a specific video service.
Fig. 4 (a) is a schematic diagram of a video data processing flow according to an embodiment of the present invention. In fig. 4 (a), VI is a VI unit, where PATH0-PATH7 indicates that the VI unit is directed to a VI channel of each PATH of video service, and an image size and a frame rate of video data output by each VI channel of the VI unit are adjustable according to video parameters required by the video service. In fig. 4 (a), the video traffic includes main stream coding, sub stream coding, HDMI/VGA, CVBS, streaming service, HDMI non-homologous output, and JPEG coding.
The VPE marked by the dashed line in fig. 4 (a) may be omitted according to actual requirements, for example, when the VI unit can meet the service requirements of the video service, for example, main code stream coding, sub code stream coding, service stream taking, HDMI previewing, etc., the later unit of the VI unit may be directly a service processing device of the video service; the HDMI, VGA and VBS are input to the VPE unit, and the VPE unit performs further processing and then inputs the processed VPE unit to the corresponding service processing device. In fig. 4 (a), the dashed arrow marks represent scalable video traffic such as HDMI non-homologous output, JPEG encoding, etc.
As shown in the following table 2, the method for processing video data according to the embodiment of the present invention is used to reduce resource waste of each video service:
TABLE 2
The following describes each video service in table 2:
first video service: main code stream coding (maximum coding frame rate)
For the encoding real-time device, when the encoding real-time device processes the main code stream encoding (maximum encoding frame rate) service, the video data of the signal frame rate of the video stream signal is required by the target video service, but in the embodiment of the invention, the VI unit can output the video data according to the target frame rate required by the target video service, so that each video frame in the VI unit output video data can be utilized by the encoder, thereby no resource waste is caused.
For the encoding non-real time device, when the encoding non-real time device processes the main code stream encoding (maximum encoding frame rate) service, the target frame rate required by the target video service is smaller than the video data of the signal frame rate of the video stream signal.
Second video service: main code stream coding (non-maximum coding frame rate)
When the main code stream coding (non-maximum coding frame rate) service is processed, the video data with the frame rate smaller than the signal frame rate of the video stream signal is required by the target video service, but in the embodiment of the invention, the VI unit can output the video data according to the frame rate smaller than the signal frame rate of the video stream signal required by the target video service, so that each video frame in the VI unit output video data can be utilized by the encoder, thereby avoiding resource waste.
Third video service: subcode stream coding (maximum coding frame rate)
For subcode stream coding (maximum coding frame rate), it is necessary to analyze the resource waste from two dimensions, namely the frame rate and the image size.
For the frame rate dimension, the resource waste condition of the encoding real-time device or the encoding non-real-time device when processing the sub-stream encoding (maximum encoding frame rate) service is the same as the resource waste condition of the encoding real-time device or the encoding non-real-time device when processing the main stream encoding (maximum encoding frame rate) service in the embodiment of the present invention, that is, in the frame rate dimension, the resource waste does not exist in the encoding real-time device and the encoding non-real-time device, and the relevant description is referred to the main stream encoding (maximum encoding frame rate) part and is not repeated herein.
For the image size dimension, since for sub-stream coding (maximum coding frame rate) traffic, the image size of the video after coding is smaller than the image size of the video stream signal.
If the VI unit meets the service requirement, that is, if the VI unit can completely scale the image size of the original video data to the image size required by the target video service, in the dimension of the image size, no resource waste exists in the encoding real-time device and the encoding non-real-time device.
If the VI unit does not meet the service requirement, that is, the VI unit cannot completely scale the image size of the original video data to the image size required by the target video service, the VI unit outputs the video data scaled according to the maximum scaling capability, and further performs scaling by the VPE.
Fourth video service: subcode stream coding (non-maximum coding frame rate)
Similarly, for subcode stream coding (not the maximum coding frame rate), it is also necessary to analyze the resource waste from two dimensions, namely the frame rate and the image size.
For the frame rate dimension, the resource waste condition of the encoding real-time device or the encoding non-real-time device when processing the sub-stream encoding (non-maximum encoding frame rate) service is the same as the resource waste condition of the encoding real-time device or the encoding non-real-time device when processing the main stream encoding (non-maximum encoding frame rate) service in the embodiment of the present invention, that is, in the frame rate dimension, the resource waste does not exist in the encoding real-time device and the encoding non-real-time device, and the relevant description is referred to the above main stream encoding (non-maximum encoding frame rate) part and is not repeated here.
Regarding the dimension of the image size, the resource waste condition of the encoding real-time device or the encoding non-real-time device when processing the sub-code stream encoding (non-maximum encoding frame rate) service is the same as the resource waste condition of the encoding real-time device or the encoding non-real-time device when processing the sub-code stream encoding (maximum encoding frame rate) service in the embodiment of the invention. If the VI unit meets the service requirement, no resource waste exists in the image dimension, the coding real-time equipment and the coding non-real-time equipment; if the VI unit does not meet the service requirement, in the image size dimension, in the embodiment of the present invention, the encoding real-time device and the encoding non-real-time device can reduce resource waste.
Fifth video service: high definition preview (Multi-picture)
Although the size of each path of high-definition preview picture is smaller than the image size of the original video data, in the embodiment of the present invention, the VI unit may adjust the image size of the output video data according to the image size required by each path of high-definition preview picture.
If the VI unit meets the service requirement, that is, if the VI unit can completely scale the image size of the original video data to the image size required by the high-definition preview picture, no resource waste exists in the image size dimension, namely, the encoding real-time device and the encoding non-real-time device.
If the VI unit does not meet the service requirement, that is, the VI unit cannot completely scale the image size of the original video data to the image size required by the high-definition preview picture, the VI unit outputs the image picture scaled according to the maximum scaling capability, and then further performs scaling by the VPE.
Taking DDR bandwidth as an example, the consumption of DDR bandwidth when the video recorder processes video service is further described.
Fig. 4 (b) is a schematic diagram of DDR bandwidth consumption when a video recorder processes video services according to an embodiment of the present invention.
First kind: the main code stream encodes the service, and the VI unit outputs to the encoder;
in this process, the VI unit outputs video data according to a frame rate and an image size required for encoding the main stream. The process comprises the following steps: the video data is written into the DDR, and then the encoder encoded by the main code stream reads the video data from the DDR. In the encoding process of the main code stream, the occupation amount of the DDR bandwidth is the data amount of video data required by the encoding of the main code stream, namely the frame rate required by the encoding of the main code stream is multiplied by the image size required by the encoding of the main code stream.
Second kind: the sub-code stream codes the business, VI unit outputs to the encoder;
in this process, the VI unit outputs video data according to a frame rate and an image size required for sub-stream encoding. The process comprises the following steps: the video data is written into the DDR, and then the encoder of the subcode stream code reads the video data from the DDR. In the encoding process of the subcode stream, the occupation amount of the DDR bandwidth is the data amount of video data required by the subcode stream encoding, namely the frame rate required by the subcode stream encoding is multiplied by the image size required by the subcode stream encoding.
Third kind: the VI unit outputs the high-definition preview service to the HDMI/VGA;
in this process, the VI unit outputs video data according to a frame rate and an image size required for HDMI/VGA. The process includes two phases: the first stage, the VI unit writes the video data into DDR, read the video data from DDR by VPE; and in the second stage, after VPE processing, video data is written into the DDR, and the HDMI/VGA preview device reads the video data from the DDR.
If the VI unit meets the service requirement, the occupation amount of the DDR bandwidth in the first stage and the second stage is the image size required by the HDMI/VGA multiplied by the real-time frame rate, i.e., the size of the high-definition preview window multiplied by the real-time frame rate.
If the VI unit cannot meet the service requirement, the occupation amount of the DDR bandwidth in the first stage is the maximum scaled image size of the VI unit multiplied by the real-time frame rate, and the occupation amount of the DDR bandwidth in the second stage is the required image size of the HDMI/VGA multiplied by the real-time frame rate.
Fourth kind: standard definition preview service, VI unit outputs to CVBS;
in this process, the VI unit outputs video data according to the frame rate and image size required by the CVBS. The process includes two phases: the first stage, the VI unit writes the video data into DDR, read the video data from DDR by VPE; and in the second stage, after VPE processing, the video data is written into the DDR, and the CVBS preview device reads the video data from the DDR.
If the VI unit meets the service requirement, the occupation amount of the DDR bandwidth in the first stage and the second stage is the image size required by the CVBS multiplied by the real-time frame rate, i.e. the size of the standard definition preview window multiplied by the real-time frame rate.
If the VI units cannot meet the service requirement, the occupation amount of the DDR bandwidth in the first stage is the maximum scaled image size of the VI units multiplied by the real-time frame rate, and the occupation amount of the DDR bandwidth in the second stage is the required image size of the CVBS multiplied by the real-time frame rate.
Fifth: the flow taking service, the VI unit outputs to the intelligent processing unit
In this process, the VI unit outputs video data according to the frame rate and image size required for the streaming service. The process comprises the following steps: the video data is written into the DDR, and then the intelligent processing unit reads the video data from the DDR. It can be seen that, in the execution process of the streaming service, the occupation amount of the DDR bandwidth is the data amount of video data required by the streaming service, that is, the frame rate required by the streaming service is multiplied by the image size required by the streaming service.
Sixth: HMDI non-homologous output service, VI unit outputs to HMDI equipment;
in this process, the VI unit outputs video data according to the frame rate and image size required for HMDI non-homologous output service. The process includes two phases: the first stage, the VI unit writes the video data into DDR, read the video data from DDR by VPE; in the second stage, after VPE processing, the video data is written into the DDR, and the HMDI device reads the video data from the DDR.
If the VI unit meets the traffic demand, the DDR bandwidth occupancy of the first stage and the second stage is the image size required by the HMDI non-homologous output traffic multiplied by the real-time frame rate.
If the VI unit cannot meet the service requirement, the occupation amount of the DDR bandwidth in the first stage is the image size of the VI unit with the maximum zoom multiplied by the real-time frame rate, and the occupation amount of the DDR bandwidth in the second stage is the image size required by the HMDI non-homologous output service multiplied by the real-time frame rate.
Seventh: JPEG coding service, VI unit outputs to the encoder;
in this process the VI unit outputs video data according to the frame rate and image size required for the JPEG encoding service. The process comprises the following steps: writing the video data into DDR, and then reading the video data from DDR by an encoder of JPEG coding service. In the execution process of the streaming service, the occupation amount of the DDR bandwidth is the data amount of video data required by the JPEG coding service, namely the frame rate required by the JPEG coding service is multiplied by the image size required by the JPEG coding service.
Eighth: the method comprises the steps that ANY service, a VI unit outputs to a processing unit of the ANY service;
the ANY service is ANY expandable service, and can be used for the subsequent possible service expansion as redundancy. In this process, the VI unit writes the video data into the DDR, and the processing unit of the ANY service reads the video data from the DDR. In the execution process of the ANY service, the occupation amount of the DDR bandwidth is the data amount of video data required by the ANY service, that is, the frame rate required by the ANY service is multiplied by the image size required by the ANY service.
The technical scheme of the embodiment of the invention can reduce the consumption of DDR bandwidth and memory.
As shown in the following table 3, in order to adopt the video data processing method provided by the embodiment of the present invention, the consumption conditions of each resource are as follows:
TABLE 3 Table 3
As can be seen from table 3 above, for main stream encoding (maximum encoding frame rate), by adopting the technical scheme of the embodiment of the present invention, in encoding real-time equipment, the image size of VI output video data can be reduced, thereby reducing DDR bandwidth and memory consumption; in the encoding non-real time device, the code rate and the image size of the VI output video data can be reduced at the same time, so that the DDR bandwidth and the consumption of the memory are reduced.
For the main code stream coding (not the maximum coding frame rate), by adopting the technical scheme of the embodiment of the invention, the code rate and the image size of VI output video data can be reduced simultaneously in the coding real-time equipment and the coding non-real-time equipment, thereby reducing the consumption of DDR bandwidth and memory.
For subcode stream coding (maximum coding frame rate), by adopting the technical scheme of the embodiment of the invention, the code rate and the image size of VI output video data can be reduced simultaneously in coding real-time equipment and coding non-real-time equipment, thereby reducing the consumption of DDR bandwidth and memory.
For high-definition preview (multi-picture), the technical scheme of the embodiment of the invention can reduce the image size of VI output video data in both encoding real-time equipment and encoding non-real-time equipment, and can directly use the video data for high-definition preview service without VPE processing, thereby reducing DDR bandwidth and memory consumption.
For streaming service, by adopting the technical scheme of the embodiment of the invention, the code rate and the image size of VI output video data can be reduced simultaneously in the encoding real-time equipment and the encoding non-real-time equipment, thereby reducing the consumption of DDR bandwidth and memory.
Therefore, the video recorder provided by the embodiment of the invention has at least the following effective effects:
1. and the consumption of DDR bandwidth and memory by video service is reduced.
2. The amount of data that needs to be processed is reduced, thereby reducing the performance consumption of the VPE.
3. For the video recorder, the DDR bandwidth, the CPU utilization rate and the memory are related, one party reaches the bottleneck and can influence the overall performance of the video recorder, so that the embodiment of the invention can improve the overall performance of the video recorder by reducing the consumption of video service to the DDR bandwidth and the memory.
4. For the video recorder, the DDR bandwidth, the CPU utilization rate and the memory can be mutually converted, for example, after the CPU utilization rate in the video recorder reaches the maximum value, the consumption of the memory and/or the DDR bandwidth can be increased to replace the reduction of the CPU utilization rate through the adjustment of a flow or a framework, so that the overall performance of the device is improved, and the embodiment of the invention can improve the overall performance of the video recorder by reducing the consumption of the video service to the DDR bandwidth and the memory.
Optionally, when the video parameters required by at least one target video service only include the target frame rate, on the basis of the above embodiment of the present invention, another video recorder is further provided in the embodiment of the present invention, where the video recorder includes a video input VI unit and at least one service processing unit; each service processing unit is used for processing video services;
a VI unit, configured to obtain pre-configured service information, where the service information includes a target frame rate required by a target video service to be executed; when receiving a video stream signal to be processed, converting the video stream signal into video data; based on the target frame rate, performing frame rate adjustment on the video data to obtain target video data, and writing the target video data into a first storage position corresponding to the target video service; the first storage position corresponding to the target video service is a storage position which is applied for the VI unit in advance and used for storing data of the target video service;
and the service processing unit is used for acquiring data to be utilized of the target video service, and processing the data to be utilized according to a data processing mode corresponding to the target video service, wherein the data to be utilized is determined based on the target video data in the designated storage position corresponding to the target video service, and the designated storage position is a first storage position corresponding to the target video service.
In the above scheme provided by the embodiment of the invention, the VI unit can adjust the frame rate of the video data based on the target frame rate, so that the VI unit can output the target video data suitable for the target video service, and the duty ratio of the utilized part in the target video data is effectively improved, thereby reducing the resource waste in the video recorder for the video service.
In this embodiment, the target frame rate is the same as the specified frame rate mentioned in the previous embodiment. In addition, in this embodiment, the frame rate adjustment is performed on the video data based on the target frame rate to obtain the target video data, which is not limited to a specific strategy of determining the frame rate adjustment for the video data according to the target frame rate, but rather the frame rate of the target video data is necessarily changed relative to the frame rate of the video data obtained after the video stream signal conversion. For example, when the target frame rate is the same as the frame rate of the video data obtained after converting the video stream signal, the frame rate of the target video data obtained after performing the frame rate adjustment on the video data based on the target frame rate is the same as the frame rate of the video data before performing the frame rate adjustment; for another example, when the target frame rate is smaller than the frame rate of the video data obtained after the video stream signal is converted, a specific manner of performing frame rate adjustment on the video data may be determined based on the target frame rate, for example, the frame rate adjustment may be performed by the VI unit alone, or the frame rate adjustment may be performed by the VI unit with assistance of a VI unit later stage unit (service processing unit, VPE unit, etc.).
In this embodiment, the target video service may be single or multiple.
If the target video service is single, the VI unit can generate video data corresponding to one path of target video service aiming at one path of video stream signal, and then the frame rate is adjusted to obtain one path of target video data; in this case, the frame rate of the target video data may be greater than or equal to the target frame rate required for the target video service to be performed, and less than or equal to the frame rate of the video data before the frame rate adjustment, for example.
If the number of the target video services is multiple, the VI unit may generate video data corresponding to one target video service for one video stream signal, and then perform frame rate adjustment to obtain multiple target video data. In this case, for example, the frame rate of the target video data corresponding to each target video service may be greater than or equal to the target frame rate required for the target video service, and less than or equal to the frame rate of the video data before the frame rate adjustment.
In this embodiment, the VI unit may obtain pre-configured service information, where the pre-configured service information includes a target frame rate required for a target video service to be executed.
Optionally, if the target video service is single, the VI unit may convert the video stream signal into one path of video data, and may further perform frame rate adjustment on the path of video data based on the target frame rate of the target video service, to obtain the target video data.
Alternatively, if the target video service is plural, the VI unit may convert the video stream signal into multiple video data. In one implementation, the VI unit may convert the video stream signal into video data, and write the converted video data into a plurality of predetermined buffers, respectively, to obtain multiple paths of video data.
After converting the video stream signal into video data, the VI unit may perform frame rate adjustment on the video data in the predetermined buffer corresponding to the target video service based on the target frame rate of the target video service for each target video service, to obtain target video data corresponding to the video service. The specific frame rate adjustment method will be described in detail in the following embodiments, and will not be described herein.
It can be appreciated that if the target frame rate is consistent with the frame rate of the video data after the video stream signal conversion, the frame rate of the video data is adjusted based on the target frame rate to obtain the target video data, which may be expressed as a frame rate that does not need to process the video data (e.g., the frame rate of the video data is not changed); if the target frame rate is inconsistent with the frame rate of the video data converted by the video stream signal, the frame rate of the video data is adjusted based on the target frame rate, so as to obtain the target video data, wherein the target video data is expressed as the frame rate of the video data to be processed (for example, the frame rate of the video data is changed), namely, the frame rate of the video data is adjusted based on the target frame rate.
If there are multiple target video services, the VI unit may adjust, for each target video service, a frame rate of one path of video data corresponding to the target video service based on a target frame rate of the target video service, to obtain target video data corresponding to the video service.
Optionally, in the case that the VI unit writes the converted video data into a plurality of predetermined buffers respectively, the VI unit may perform frame rate adjustment on the video data in the predetermined buffer corresponding to the target video service based on the target frame rate of the target video service for each target video service, to obtain the target video data corresponding to the video service.
In one implementation, the VI unit may convert the frame rate of the video data to a target frame rate required for the target video service. Illustratively, the target frame rate is 30fps, then the VI unit converts the video data to target video data at a frame rate of 30 fps.
After the VI unit writes the target video data into the first storage location corresponding to the target video service, the service processing unit 202 corresponding to the target video service may perform service processing on the data to be utilized of the target video service according to the data processing manner corresponding to the target video service.
In the above scheme provided by the embodiment of the invention, the VI unit can adjust the frame rate of the video data based on the target frame rate, so that the VI unit can output the target video data suitable for the target video service, and the duty ratio of the utilized part in the target video data is effectively improved, thereby reducing the resource waste in the video recorder for the video service.
Referring to fig. 2, in case that the video recorder according to the embodiment of the present invention may further include an image processing engine VPE unit 203, the VPE unit 203 may be configured to read target video data from a designated storage location and perform a pre-configuration process on the target video data, and write the processed target video data to a second storage location corresponding to the target video service, in case that the first indication information corresponding to the target video service is detected.
The second storage position corresponding to the target video service is a storage position which is applied for the VPE unit in advance and used for storing data of the target video service; the first indication information corresponding to the target video service comprises: information characterizing the VPE units contained in each unit involved in executing the target video service.
When related personnel configure a target video service to be executed through a GUI, an upper layer application can configure each unit involved in executing the target video service, and when each unit involved in executing the target video service contains a VPE unit, the upper layer application can issue first indication information to the VPE unit.
When the first indication information corresponding to the target video service is obtained, the VPE unit can read the target video data from the designated storage location, perform pre-configuration processing on the target video data, and write the processed target video data into the second storage location corresponding to the target video service.
In one implementation manner, the upper layer application may further send a first storage location and a second storage location corresponding to the target video service to the VPE unit, where the second storage location corresponding to the target video service is a storage location that is applied by the upper layer application in advance for the VPE unit and is used for storing data of the target video service. After receiving the first storage location and the second storage location sent by the upper layer application, the VPE unit may determine, based on the acquired first storage location, a location where the data to be read is located, and determine, based on the acquired second storage location, a storage location where the data is subjected to the pre-configuration processing. Optionally, the upper layer application may write the first storage location and the second storage location corresponding to the target video service into the first indication information, and issue the first indication information to the VPE unit at the same time, or may send the first storage location and the second storage location corresponding to the target video service to the VPE unit separately.
When the number of the target video services is multiple, the VPE unit can read the target video data corresponding to the target video service from the first storage position corresponding to the target video service when detecting the first indication information corresponding to any target video service, so as to perform pre-configuration processing on the read target video data, and finally write the processed target video data into the second storage position corresponding to the target video service.
In one embodiment, the VI unit performs frame rate adjustment on the video data based on the target frame rate to obtain target video data, which may include:
converting the video data into video data having a first frame rate as target video data; wherein the first frame rate is greater than or equal to the target frame rate and less than a frame rate of the video data prior to frame rate adjustment.
It should be emphasized that in the present embodiment, the video data is converted into the video data having the first frame rate, and the frame rate of the target video data is not limited to be changed from the frame rate of the video data before the conversion, that is, the first frame rate may be equal to the frame rate of the video data before the frame rate adjustment is performed, but the strategy of performing the frame rate adjustment on the video data is determined according to the target frame rate. For example, when the target frame rate is the same as the frame rate of the video data obtained after converting the video stream signal, the first frame rate of the target video data obtained after converting may be the same as the frame rate of the video data before performing the frame rate adjustment; for another example, when the target frame rate is smaller than the frame rate of the video data obtained after the video stream signal is converted, a specific manner of performing frame rate adjustment on the video data may be determined based on the target frame rate, for example, the frame rate adjustment may be performed by the VI unit alone, or the frame rate adjustment may be performed by the VI unit with assistance of a VI unit later stage unit (service processing unit, VPE unit, etc.).
The frame rate of the video data before the frame rate adjustment is performed can be understood as the signal frame rate of the video stream signal. By way of example, the target frame rate is 30fps, and the frame rate of the video data before the frame rate adjustment is 60fps, then 30fps is less than or equal to the first frame rate which may be, for example, 40fps, 50fps, and another example, 30fps is less than or equal to 60 fps. At this time, the VI unit may convert the video data into target video data having a frame rate of 40fps or 50fps, and for example, the VI unit may convert the video data into target video data having a frame rate of 30fps.
If the VI unit has the capability to convert video data to a target frame rate required for a target video service, the first frame rate may be equal to the target frame rate. For example, the target frame rate is 30fps, then the VI unit has the capability to convert video data to the target frame rate required for the target video service, and the VI unit can convert video data to video data having 30fps.
If the VI unit does not have the capability of converting the video data to the target frame rate required by the target video service, the first frame rate may be greater than the target frame rate and less than or equal to the frame rate of the video data before the frame rate adjustment is performed. For example, the target frame rate is 30fps, the frame rate of the video data before the frame rate adjustment is 60fps, and then 30fps < the first frame rate is less than 60fps, and the first frame rate may be, for example, 40fps, 50fps, etc.
In this case, when the VI unit does not have the capability of converting video data into a target frame rate required for the target video service, the subsequent unit of the VI unit is required to perform further frame rate adjustment on the target video data in order to obtain the video data of the target frame rate required for the target video service.
In one implementation, the service processing unit corresponding to the target video service may perform further frame rate adjustment processing on the target video data. At this time, the service processing unit corresponding to the target video service is further configured to convert the frame rate of the data to be utilized of the target video service into the target frame rate before performing service processing on the data to be utilized according to the data processing manner corresponding to the target video service. The data to be utilized of the target video service is determined based on the target video data in the designated storage position corresponding to the target video service.
In another implementation, in the case that the video recorder further includes a VPE unit for performing a pre-configuration process on the target video data in the designated storage location, a further frame rate adjustment process may be performed on the target video data by the VPE unit. At this time, the pre-configuration processing of the target video data by the VPE unit may include converting a frame rate of the target video data into the target frame rate.
The target video data after the pre-configuration processing is stored in a second storage position corresponding to the target video service, and the target video data in the second storage position is used as data to be utilized for the target video service.
That is, in the case that the VI unit does not have the capability of converting video data into the target frame rate required for the target video service, in order to obtain video data having the target frame rate, the VPE unit or the service unit may be used in combination with the VI unit to complete the adjustment of the final implementation frame rate.
In another embodiment, when there are multiple target video services, the VI unit may perform frame rate adjustment on one path of the video data corresponding to the target video service based on the target frame rate of the target video service for each target video service, to obtain target video data corresponding to the video service.
The VI unit may convert, for each target video service, one path of the video data corresponding to the target video service into video data having a second frame rate corresponding to the video service, as target video data corresponding to the video service. Wherein, the second frame rate corresponding to each target video service: the frame rate of the video data of the path corresponding to the target video service is greater than or equal to the target frame rate of the target video service and is smaller than the frame rate of the video data of the path corresponding to the target video service before the frame rate adjustment is carried out.
Illustratively, the frame rate of the video data prior to frame rate adjustment is 60fps. The number of the target video services is 3, namely a target video service 1, a target video service 2 and a target video 3, wherein the target frame rate required by the target video service 1 is 30fps, the target frame rate required by the target video service 2 is 35fps and the target frame rate required by the target video service 3 is 40fps.
For the target video service 1, the second frame rate corresponding to the target video service 1 is 30fps less than or equal to 60fps, for example, the second frame rate corresponding to the target video service 1 may be 30fps, 40fps, 50fps, 60fps, and if 50fps, the vi unit converts the video data into video data with a frame rate of 50fps for the target video service 1 as the target video data corresponding to the target video service 1.
For the target video service 2, the second frame rate corresponding to the target video service 2 is 35fps less than or equal to 60fps, for example, the second frame rate corresponding to the target video service 2 may be 35fps, 40fps, 50fps, 60fps, and if 50fps, the vi unit converts the video data into video data with a frame rate of 50fps for the target video service 2 as the target video data corresponding to the target video service 2.
For the target video service 3, the second frame rate corresponding to the target video service 3 is equal to or less than 40fps and equal to or less than 60fps, for example, the second frame rate corresponding to the target video service 3 may be 40fps, 45fps, 50fps, 60fps, and if 50fps, the vi unit converts the video data into video data with a frame rate of 50fps for the target video service 3 as the target video data corresponding to the target video service 3.
At this time, in order to obtain video data of a target frame rate required for each target video service, a subsequent stage unit of the VI unit is required to make further frame rate adjustment for each target video data. That is, no matter how many paths of target video data are output by the VI unit, for each path of target video data output by the VI unit, the VPE unit or the service unit corresponding to the path of video service may assist in processing the frame rate of the path of target video data.
Optionally, for any path of target video service, if the second frame rate corresponding to the target video service is greater than the target frame rate of the target video service, in one implementation manner, the service processing unit corresponding to the target video service may perform assistance processing, where:
the service processing unit corresponding to the target video service is further configured to convert the frame rate of the data to be utilized of the video service into a target frame rate of the target video service before performing service processing on the data to be utilized of the target video service according to the data processing manner corresponding to the target video service.
In another implementation manner, in the case that the video recorder further includes a VPE unit configured to perform a pre-configuration process on the target video data corresponding to the target video service in the designated storage location corresponding to the target video service, the VPE unit may perform a further frame rate adjustment process on the target video data corresponding to the target video service. At this time, the pre-configuration process performed by the VPE unit for the video service may include converting the frame rate of the target video data corresponding to the target video service into a target frame rate required for the target video service. The target video data after the pre-configuration processing is stored in a second storage position corresponding to the target video service, and the target video data in the second storage position is used as data to be utilized for the target video service.
After acquiring the data to be utilized for the target video service, the service processing unit 202 corresponding to the target video service may perform service processing on the data to be utilized according to a data processing manner corresponding to the target video service.
Optionally, in an embodiment, the process of performing frame rate adjustment on the video data by the VI unit to obtain the target video data based on the target frame rate may include steps a-B:
Step A, determining a video frame to be extracted from video data according to a first frame rate; the first frame rate is greater than or equal to the target frame rate and less than or equal to a frame rate of the video data prior to frame rate adjustment;
when the first frame rate is the same as the frame rate of the video data, the VI unit may determine each video frame of the video data as a video frame to be decimated.
On the basis of the video recorder provided by the embodiment of the invention, the embodiment of the invention also provides a video data processing method which is applied to the video recorder, and optionally can be applied to the video recorder provided by the embodiment of the invention.
As shown in fig. 5, a video data processing method according to an embodiment of the present invention is applied to a video recorder, and includes the steps of:
s501, obtaining pre-configured service information, wherein the service information comprises a target frame rate required by a target video service to be executed;
the VI unit in the video recorder may obtain the pre-configured service information.
S502, when receiving a video stream signal to be processed, converting the video stream signal into video data;
the VI unit in the video recorder may convert the video stream signal into video data when receiving the video stream signal to be processed.
S503, adjusting the frame rate of the video data based on the target frame rate to obtain target video data;
the VI unit in the video recorder may adjust the frame rate of the video data based on the target frame rate to obtain the target video data.
S504, obtaining data to be utilized of a target video service; wherein the data to be utilized is data determined based on the target video data;
the target video service corresponding service processing unit in the video recorder can acquire data to be utilized of the target video service; wherein the data to be utilized is data determined based on the target video data.
S505, the data to be utilized is processed according to the data processing mode corresponding to the target video service.
The service processing unit corresponding to the target video service in the video recorder can process the data to be utilized according to the data processing mode corresponding to the target video service.
The VI unit can adjust the frame rate of the video data based on the target frame rate, so that the VI unit can output the target video data suitable for the target video service, the duty ratio of the utilized part in the target video data is effectively improved, and the resource waste aiming at the video service in the video recorder can be reduced
In the video access method according to the embodiment of the present invention, the description is relatively simple because the video access method corresponds to the video recorder, and the description of the video recorder according to the present invention is referred to.
Optionally, in an implementation manner, the determining manner of the data to be utilized of the target video service includes:
and performing pre-configuration processing on the target video data to obtain data to be utilized corresponding to the target video data.
Optionally, in an implementation manner, the adjusting the frame rate of the video data based on the target frame rate to obtain target video data includes:
converting the video data into video data having a first frame rate as target video data; wherein the first frame rate is greater than or equal to the target frame rate and less than a frame rate of the video data prior to frame rate adjustment.
Optionally, in one implementation, the first frame rate is greater than the target frame rate;
after the data to be utilized of the target video service is obtained, before the data to be utilized is processed according to the data processing mode corresponding to the target video service, the method further comprises:
Converting the frame rate of the data to be utilized of the target video service into the target frame rate;
or,
in the case that the determining manner of the data to be utilized of the target video service includes performing a pre-configuration process on the target video data to obtain the data to be utilized, the pre-configuration process includes: the frame rate of the target video data is converted to the target frame rate.
Optionally, in an implementation manner, the adjusting the frame rate of the video data based on the target frame rate to obtain target video data includes:
the frame rate of the video data is converted to the target frame rate.
Optionally, in one implementation manner, the target video service is multiple;
the converting the video stream signal into video data includes:
converting the video stream signal into multiple paths of video data; wherein each path of video data corresponds to a target video service;
the step of adjusting the frame rate of the video data based on the target frame rate to obtain target video data includes:
and aiming at each target video service, based on the target frame rate of the target video service, carrying out frame rate adjustment on one path of video data corresponding to the target video service to obtain target video data corresponding to the video service.
Optionally, in one implementation manner, the adjusting the frame rate of the video data corresponding to the target video service based on the target frame rate of the target video service to obtain target video data corresponding to the video service includes:
converting one path of video data corresponding to the target video service into video data with a second frame rate corresponding to the video service, and taking the video data as target video data corresponding to the video service; wherein, the second frame rate corresponding to each target video service: the frame rate of the video data of the path corresponding to the target video service is smaller than or equal to the frame rate before the frame rate adjustment is carried out.
Optionally, in one implementation, the second frame rate corresponding to the target video service is greater than the target frame rate of the target video service;
after the data to be utilized of the target video service is obtained, before the data to be utilized of the target video service is processed according to the data processing mode corresponding to the target video service, the method further comprises: converting the frame rate of the data to be utilized of the target video service into the target frame rate;
Or,
in the case that the determining manner of the data to be utilized of the target video service includes performing a pre-configuration process on the target video data corresponding to the target video service to obtain the data to be utilized of the target video service, the pre-configuration process includes: and converting the frame rate of the target video data corresponding to the target video service into the target frame rate required by the target video service.
Optionally, in an implementation manner, the converting the video stream signal into multiple paths of video data includes:
converting the video stream signal into video data, and writing the converted video data into a plurality of preset buffer areas respectively to obtain a plurality of paths of video data; wherein each predetermined buffer area uniquely corresponds to a target video service;
the frame rate adjustment is performed on one path of video data corresponding to each target video service based on the target frame rate of the target video service to obtain target video data corresponding to the video service, including:
and for each target video service, based on the target frame rate of the target video service, carrying out frame rate adjustment on the video data in a preset buffer zone corresponding to the target video service to obtain target video data corresponding to the video service.
Optionally, in an implementation manner, the adjusting the frame rate of the video data based on the target frame rate to obtain target video data includes:
determining a video frame to be extracted from the video data according to a first frame rate; the first frame rate is greater than or equal to the target frame rate and less than or equal to a frame rate of the video data prior to frame rate adjustment;
and extracting the determined video frames to obtain video data with the frame rate of the first frame rate as target video data aiming at the target video service.
Based on the video data processing method according to the embodiment of the present invention, as shown in fig. 6, an embodiment of the present invention further provides a video data processing device, which is applied to a video recorder, and the device includes:
an information obtaining module 601, configured to obtain pre-configured service information, where the service information includes video parameters required by at least one target video service to be executed;
the first conversion module 602 is configured to convert, when receiving a video stream signal to be processed, the video stream signal into at least one path of video data; wherein each path of video data corresponds to a target video service;
A second conversion module 603, configured to convert, for each target video service, one path of the video data corresponding to the target video service into target video data having video parameters required by the target video service;
the data processing module 604 is configured to obtain, for each target video service, data to be utilized corresponding to the video service, and process the data to be utilized according to a data processing manner corresponding to the target video service; wherein the data to be utilized is determined based on the target video data of the target video service.
Optionally, the data processing module includes:
and the data processing sub-module is used for carrying out pre-configuration processing on the target video data of the video service to obtain data to be utilized corresponding to the video service.
Optionally, the pre-configuration process includes:
one or more of image size scaling, video data duplication, adjustment for specified image parameters; wherein the specified image parameter is a parameter for characterizing image quality.
Optionally, the first conversion module is specifically configured to convert the video stream signal into video data, and write the converted video data into at least one predetermined buffer area respectively, so as to obtain at least one path of video data; wherein each predetermined buffer area uniquely corresponds to a target video service;
The second conversion module is specifically configured to convert, for each target video service, the video data in a predetermined buffer area corresponding to the target video service into target video data having video parameters required by the target video service.
Optionally, the video parameters include a specified frame rate and/or a specified image size;
the second conversion module is specifically configured to process, for each target video service, one path of video data corresponding to the target video service according to a specified frame rate and/or a specified image size required by the target video service, so as to obtain target video data with video parameters required by the target video service.
According to the video data processing device provided by the embodiment of the invention, as the video input unit can output one path of video data for each video service and each path of video data is processed according to the video parameters required by the video service, each path of video data can be ensured to be completely utilized by the video service, and resource waste is reduced.
The embodiment of the present invention further provides an electronic device, as shown in fig. 7, including a processor 701, a communication interface 702, a memory 703 and a communication bus 704, where the processor 701, the communication interface 702, and the memory 703 perform communication with each other through the communication bus 704,
A memory 703 for storing a computer program;
the processor 701 is configured to implement the steps of the video data processing method provided in the embodiment of the present invention when executing the program stored in the memory 703.
The communication bus mentioned above for the electronic devices may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the electronic device and other devices.
The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processing, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
In yet another embodiment of the present invention, there is also provided a computer readable storage medium having stored therein a computer program which when executed by a processor implements the steps of any of the video data processing methods described above.
In yet another embodiment of the present invention, there is also provided a computer program product containing instructions that, when run on a computer, cause the computer to perform the video data processing method of any of the above embodiments.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus, devices, system embodiments, the description is relatively simple as it is substantially similar to method embodiments, with reference to the description of method embodiments in part.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims (11)
1. A video recorder comprising a video input VI unit and at least one service processing unit; each service processing unit is used for processing video services;
the VI unit is configured to obtain pre-configured service information, where the service information includes video parameters required by at least one target video service to be executed; when receiving a video stream signal to be processed, converting the video stream signal into at least one path of video data, wherein each path of video data corresponds to a target video service; for each target video service, converting one path of video data corresponding to the target video service into target video data with video parameters required by the target video service, and writing the target video data of each target video service into a first storage position corresponding to the target video service; the first storage position corresponding to each target video service is a storage position which is applied for the VI unit in advance and used for storing the data of the target video service;
The service processing unit is used for acquiring data to be utilized for each target video service and processing the data to be utilized according to a data processing mode corresponding to the target video service;
in the case that the units involved in executing the target video service include an image processing engine VPE unit, the video recorder further includes: an image processing engine VPE unit;
the VPE unit is used for reading target video data of the target video service from a designated storage position corresponding to the target video service under the condition that first indication information corresponding to any target video service is detected, performing pre-configuration processing on the target video data, and writing the processed target video data into a second storage position corresponding to the target video service;
the second storage position corresponding to each target video service is a storage position which is applied for the VPE unit in advance and used for storing data of the target video service; the first indication information corresponding to each target video service comprises: information characterizing the inclusion of the VPE unit in each unit involved in executing the target video service; the first indication information is set under the condition that the VI unit does not have the capability of processing video data required by the target video service;
The service processing unit corresponding to each target video service obtains data to be utilized for the target video service, including:
under the condition that the first indication information corresponding to the target video service is detected, reading target video data of the target video service from a second storage position corresponding to the target video service as data to be utilized for the target video service; and/or the number of the groups of groups,
under the condition that second indication information corresponding to the target video service is detected, reading target video data of the target video service from a designated storage position corresponding to the target video service as data to be utilized for the target video service; the appointed storage position is a first storage position corresponding to the target video service;
the second indication information corresponding to each target video service comprises: information characterizing that the VPE unit is not included in units involved in executing the target video service; the second indication information is set in the case that the VI unit has the capability of processing video data required to obtain the target video service.
2. The video recorder of claim 1, wherein the pre-configuration process comprises:
One or more of image size scaling, video data duplication, adjustment for specified image parameters; wherein the specified image parameter is a parameter for characterizing image quality.
3. The video recorder of claim 1, wherein the VI unit converting the video stream signal into at least one video data, comprising:
converting the video stream signal into video data, and writing the converted video data into at least one preset buffer zone respectively to obtain at least one path of video data; wherein each predetermined buffer area uniquely corresponds to a target video service;
the converting, for each target video service, one path of the video data corresponding to the target video service into target video data having video parameters required by the target video service includes:
for each target video service, converting the video data in a preset buffer zone corresponding to the target video service into target video data with video parameters required by the target video service.
4. The video recorder of claim 1, wherein,
the VI unit converts, for each target video service, one path of the video data corresponding to the target video service into target video data having video parameters required by the target video service, including:
The VI unit processes one path of video data corresponding to each target video service according to the designated image size required by the target video service to obtain target video data with video parameters required by the target video service;
or alternatively, the first and second heat exchangers may be,
and the VI unit processes one path of video data corresponding to each target video service according to the specified image size and the specified frame rate required by the target video service to obtain target video data with video parameters required by the target video service.
5. A video data processing method for use in a video recorder, the method comprising:
acquiring pre-configured service information, wherein the service information comprises video parameters required by at least one target video service to be executed;
when receiving a video stream signal to be processed, converting the video stream signal into at least one path of video data; wherein each path of video data corresponds to a target video service;
for each target video service, converting one path of video data corresponding to the target video service into target video data with video parameters required by the target video service; the target video data is written into a first storage position corresponding to the target video service;
Aiming at each target video service, acquiring data to be utilized corresponding to the target video service, and processing the data to be utilized according to a data processing mode corresponding to the target video service; wherein the data to be utilized is determined based on target video data of the target video service;
in the case that each unit involved in executing the target video service by the video recorder includes an image processing engine VPE unit, the determining method of the data to be utilized of the target video service includes: under the condition that first indication information corresponding to any target video service is detected, pre-configuring target video data of the target video service through an image processing engine (VPE) unit, and writing the processed target video data into a second storage position corresponding to the target video service;
the first indication information corresponding to each target video service comprises: information characterizing the inclusion of the VPE unit in each unit involved in executing the target video service; the first indication information is set under the condition that a VI unit of the video recorder does not have the capability of processing video data required by the target video service;
The obtaining the data to be utilized corresponding to the target video service includes:
under the condition that the first indication information corresponding to the target video service is detected, reading target video data of the target video service from a second storage position corresponding to the target video service as data to be utilized for the target video service; and/or the number of the groups of groups,
under the condition that second indication information corresponding to the target video service is detected, reading target video data of the target video service from a designated storage position corresponding to the target video service as data to be utilized for the target video service; the appointed storage position is a first storage position corresponding to the target video service;
the second indication information corresponding to each target video service comprises: information characterizing that the VPE unit is not included in units involved in executing the target video service; the second indication information is set in the case that the VI unit has the capability of processing video data required to obtain the target video service.
6. The method of claim 5, wherein the pre-configuration process comprises:
One or more of image size scaling, video data duplication, adjustment for specified image parameters; wherein the specified image parameter is a parameter for characterizing image quality.
7. The method of claim 5, wherein converting the video stream signal into at least one video stream comprises:
converting the video stream signal into video data, and writing the converted video data into at least one preset buffer zone respectively to obtain at least one path of video data; wherein each predetermined buffer area uniquely corresponds to a target video service;
the converting, for each target video service, one path of the video data corresponding to the target video service into target video data having video parameters required by the target video service includes:
for each target video service, converting the video data in a preset buffer zone corresponding to the target video service into target video data with video parameters required by the target video service.
8. The method of claim 5, wherein the step of determining the position of the probe is performed,
the converting, for each target video service, one path of the video data corresponding to the target video service into target video data having video parameters required by the target video service includes:
Aiming at each target video service, processing one path of video data corresponding to the target video service according to the designated image size required by the target video service to obtain target video data with video parameters required by the target video service;
or alternatively, the first and second heat exchangers may be,
and processing one path of video data corresponding to each target video service according to the specified image size and the specified frame rate required by the target video service to obtain target video data with video parameters required by the target video service.
9. A video data processing apparatus for use with a video recorder, said apparatus comprising:
the information acquisition module is used for acquiring pre-configured service information, wherein the service information comprises video parameters required by at least one target video service to be executed;
the first conversion module is used for converting the video stream signal into at least one path of video data when receiving the video stream signal to be processed; wherein each path of video data corresponds to a target video service;
the second conversion module is used for converting one path of video data corresponding to each target video service into target video data with video parameters required by the target video service; the target video data is written into a first storage position corresponding to the target video service;
The data processing module is used for acquiring data to be utilized corresponding to each target video service according to each target video service, and processing the data to be utilized according to a data processing mode corresponding to the target video service; wherein the data to be utilized is determined based on target video data of the target video service;
in the case that each unit involved in the video recorder executing the target video service includes an image processing engine VPE unit, the data processing module includes:
the data processing sub-module is used for performing pre-configuration processing on target video data of any target video service through the image processing engine VPE unit under the condition that first indication information corresponding to the target video service is detected, and writing the processed target video data into a second storage position corresponding to the target video service;
the first indication information corresponding to each target video service comprises: information characterizing the inclusion of the VPE unit in each unit involved in executing the target video service; the first indication information is set under the condition that a VI unit of the video recorder does not have the capability of processing video data required by the target video service;
The data processing module is specifically configured to, when detecting first indication information corresponding to the target video service, read target video data of the target video service from a second storage location corresponding to the target video service, where the target video data is to be utilized for the target video service; and/or the number of the groups of groups,
under the condition that second indication information corresponding to the target video service is detected, reading target video data of the target video service from a designated storage position corresponding to the target video service as data to be utilized for the target video service; the appointed storage position is a first storage position corresponding to the target video service;
the second indication information corresponding to each target video service comprises: information characterizing that the VPE unit is not included in units involved in executing the target video service; the second indication information is set in the case that the VI unit has the capability of processing video data required to obtain the target video service.
10. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
A memory for storing a computer program;
a processor for carrying out the method steps of any one of claims 5-8 when executing a program stored on a memory.
11. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 5-8.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311792026.XA CN117714629A (en) | 2022-01-28 | 2022-01-28 | Video recorder, video data processing method and device and electronic equipment |
| CN202210109292.7A CN114422735B (en) | 2022-01-28 | 2022-01-28 | Video recorder, video data processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210109292.7A CN114422735B (en) | 2022-01-28 | 2022-01-28 | Video recorder, video data processing method and device and electronic equipment |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311792026.XA Division CN117714629A (en) | 2022-01-28 | 2022-01-28 | Video recorder, video data processing method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114422735A CN114422735A (en) | 2022-04-29 |
| CN114422735B true CN114422735B (en) | 2023-12-01 |
Family
ID=81280230
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311792026.XA Pending CN117714629A (en) | 2022-01-28 | 2022-01-28 | Video recorder, video data processing method and device and electronic equipment |
| CN202210109292.7A Active CN114422735B (en) | 2022-01-28 | 2022-01-28 | Video recorder, video data processing method and device and electronic equipment |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311792026.XA Pending CN117714629A (en) | 2022-01-28 | 2022-01-28 | Video recorder, video data processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN117714629A (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118674611B (en) * | 2024-08-15 | 2024-12-03 | 杭州海康威视数字技术股份有限公司 | Video recorder and data processing method |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012170054A (en) * | 2011-01-26 | 2012-09-06 | Panasonic Corp | Video recording apparatus, video reproduction apparatus, and video recovery apparatus |
| CN102972011A (en) * | 2012-04-01 | 2013-03-13 | 华为技术有限公司 | Medium processing system and method for processing medium business |
| WO2018076370A1 (en) * | 2016-10-31 | 2018-05-03 | 华为技术有限公司 | Video frame processing method and device |
| CN109040664A (en) * | 2018-06-01 | 2018-12-18 | 深圳市商汤科技有限公司 | video stream processing method and device, electronic equipment and storage medium |
| CN110290421A (en) * | 2019-08-01 | 2019-09-27 | 北京达佳互联信息技术有限公司 | Frame per second method of adjustment, device, computer equipment and storage medium |
| CN110475091A (en) * | 2019-08-13 | 2019-11-19 | 国家电网有限公司 | The processing method and processing device of video conference |
| CN111324886A (en) * | 2020-03-12 | 2020-06-23 | 杭州海康威视系统技术有限公司 | Service request processing method and device and server |
| CN111432251A (en) * | 2020-04-26 | 2020-07-17 | 浙江宇视科技有限公司 | Video processing method, video processing device, electronic equipment and medium |
| CN112735141A (en) * | 2020-12-09 | 2021-04-30 | 浙江大华技术股份有限公司 | Video data processing method and device |
| CN113225619A (en) * | 2021-04-23 | 2021-08-06 | 深圳创维－Rgb电子有限公司 | Frame rate self-adaption method, device, equipment and readable storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3343561B1 (en) * | 2016-12-29 | 2020-06-24 | Axis AB | Method and system for playing back recorded video |
| US10893281B2 (en) * | 2018-10-12 | 2021-01-12 | International Business Machines Corporation | Compression of a video stream having frames with relatively heightened quality parameters on blocks on an identified point of interest (PoI) |
| CN110139113B (en) * | 2019-04-30 | 2021-05-14 | 腾讯科技（深圳）有限公司 | Transmission parameter distribution method and device for video resources |
-
2022
- 2022-01-28 CN CN202311792026.XA patent/CN117714629A/en active Pending
- 2022-01-28 CN CN202210109292.7A patent/CN114422735B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012170054A (en) * | 2011-01-26 | 2012-09-06 | Panasonic Corp | Video recording apparatus, video reproduction apparatus, and video recovery apparatus |
| CN102972011A (en) * | 2012-04-01 | 2013-03-13 | 华为技术有限公司 | Medium processing system and method for processing medium business |
| WO2018076370A1 (en) * | 2016-10-31 | 2018-05-03 | 华为技术有限公司 | Video frame processing method and device |
| CN109040664A (en) * | 2018-06-01 | 2018-12-18 | 深圳市商汤科技有限公司 | video stream processing method and device, electronic equipment and storage medium |
| CN110290421A (en) * | 2019-08-01 | 2019-09-27 | 北京达佳互联信息技术有限公司 | Frame per second method of adjustment, device, computer equipment and storage medium |
| CN110475091A (en) * | 2019-08-13 | 2019-11-19 | 国家电网有限公司 | The processing method and processing device of video conference |
| CN111324886A (en) * | 2020-03-12 | 2020-06-23 | 杭州海康威视系统技术有限公司 | Service request processing method and device and server |
| CN111432251A (en) * | 2020-04-26 | 2020-07-17 | 浙江宇视科技有限公司 | Video processing method, video processing device, electronic equipment and medium |
| CN112735141A (en) * | 2020-12-09 | 2021-04-30 | 浙江大华技术股份有限公司 | Video data processing method and device |
| CN113225619A (en) * | 2021-04-23 | 2021-08-06 | 深圳创维－Rgb电子有限公司 | Frame rate self-adaption method, device, equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114422735A (en) | 2022-04-29 |
| CN117714629A (en) | 2024-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111263208B (en) | Picture synthesis method and device, electronic equipment and storage medium | |
| CN100542214C (en) | Imaging device and image generating method of imaging device | |
| US8139081B1 (en) | Method for conversion between YUV 4:4:4 and YUV 4:2:0 | |
| US7675550B1 (en) | Camera with high-quality still capture during continuous video capture | |
| US20090290645A1 (en) | System and Method for Using Coded Data From a Video Source to Compress a Media Signal | |
| CN109479111B (en) | Image processing apparatus, reproduction apparatus, image processing method, and reproduction method | |
| KR20200081386A (en) | Method and device for generating a second image from a first image | |
| US9161030B1 (en) | Graphics overlay system for multiple displays using compressed video | |
| US10003767B2 (en) | Image processing apparatus and image processing method | |
| CN114422735B (en) | Video recorder, video data processing method and device and electronic equipment | |
| US9053752B1 (en) | Architecture for multiple graphics planes | |
| US8483389B1 (en) | Graphics overlay system for multiple displays using compressed video | |
| EP3192250B1 (en) | Image processing apparatus and image processing method | |
| JP2014229929A (en) | Imaging device, and method and system of controlling the same | |
| CN114422734B (en) | Video recorder, video data processing method and device and electronic equipment | |
| US20070222891A1 (en) | Systems and methods for video data conversion | |
| CN117812389A (en) | Video recorder, video preview method and electronic equipment | |
| CN117812390A (en) | Video recorder, remote preview method and electronic equipment | |
| CN117812391A (en) | Video recorder, preview method and electronic equipment | |
| WO2023000815A1 (en) | Code stream processing method, apparatus, terminal device, and storage medium | |
| KR20140103521A (en) | Method for recording moving-image data, and photographing apparatus adopting the method | |
| US9179046B1 (en) | Camera with high-quality still capture during continuous video capture | |
| CN118524195A (en) | Tone mapping method and device | |
| CN115442554A (en) | Video recorder, video data processing method, and device | |
| JP2008141326A (en) | Imaging system and camera unit |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |