CN106954073B

CN106954073B - Video data input and output method, device and system

Info

Publication number: CN106954073B
Application number: CN201610008428.XA
Authority: CN
Inventors: 刘卫东
Original assignee: Hisense Visual Technology Co Ltd
Current assignee: Hisense Visual Technology Co Ltd
Priority date: 2016-01-07
Filing date: 2016-01-07
Publication date: 2020-11-13
Anticipated expiration: 2036-01-07
Also published as: CN106954073A

Abstract

The embodiment of the application provides a video data input method, which is applied to a video post-processing system, wherein the video post-processing system comprises: caching a block to be compressed and caching a compressed code stream; the method comprises the following steps: receiving video frame data to be processed from at least two paths; segmenting the video frame data to be processed into a plurality of blocks to be compressed; according to the number of blocks to be compressed of two paths, inputting the block to be compressed of one path into the block cache to be compressed; sequentially compressing the blocks to be compressed in the block cache to be compressed to generate a compressed code stream; and inputting the compressed code stream into the compressed code stream cache. According to the embodiment of the application, the video frame data is firstly compressed, and then the compressed code stream obtained through compression is input into the compressed code stream cache, so that the access bandwidth and the power consumption of the compressed code stream cache are reduced.

Description

Video data input and output method, device and system

Technical Field

The present application relates to the field of video processing technologies, and in particular, to a video data input method, a video data output method, a video data input device, a video data output device, and a video data system.

Background

In recent years, the rapid development of the ultra high definition television industry has been promoted by the subjective demand for high-quality visual enjoyment and the objective conditions for rapid development of semiconductor technology. However, due to the bandwidth limitation of the current transmission system, the ultra-high definition television programs can only be transmitted at a lower frame rate. Meanwhile, the refresh rate of the large-screen display device is greatly improved, the video frame rate is lower than the screen refresh rate, the mismatch directly causes the phenomena of smear, pause, blur and the like of the image, and the display effect is poor. The video frame rate up-conversion technology is used as an important video post-processing means, the frame rate of a displayed video can be effectively improved, and the subjective quality of an image is improved on a display screen with a high refresh rate.

The input of the ultra-high definition video frame rate up-conversion kernel is a set of image sequences with fixed frame rates, and after a series of motion estimation, vector post-processing and interpolation operations, the output is a set of image sequences with higher frame rates. This results in a significant increase in the data read-write throughput of the ultra-high-definition video frame rate up-conversion kernel and the off-chip cache. Wherein, motion estimation, vector post-processing and interpolation operation need to read a large amount of pixel data of a forward reference frame and a backward reference frame from an off-chip cache; an interpolated image sequence generated by frame rate up-conversion needs to be input into an off-chip cache; the display output port needs to read the original image sequence and the image sequence generated by frame rate up-conversion from the off-chip buffer for display on the screen. However, in the current state of the art, the bandwidth of the memory access is difficult to meet such a high read-write throughput requirement, and the off-chip cache bandwidth becomes a bottleneck limiting the system performance.

Disclosure of Invention

In view of the above problems, embodiments of the present application are proposed to provide a video data input method, a video data output method, a video data input apparatus, a video data output apparatus, and a video data system that overcome or at least partially solve the above problems.

In order to solve the above problem, an embodiment of the present application discloses a video data input method, which is applied to a video post-processing system, where the video post-processing system includes: caching a block to be compressed and caching a compressed code stream; the method comprises the following steps:

receiving video frame data to be processed from at least two paths;

segmenting the video frame data to be processed into a plurality of blocks to be compressed;

according to the number of blocks to be compressed of two paths, inputting the block to be compressed of one path into the block cache to be compressed;

sequentially compressing the blocks to be compressed in the block cache to be compressed to generate a compressed code stream;

and inputting the compressed code stream into the compressed code stream cache.

Preferably, the step of inputting the block to be compressed of one of the two ways into the block buffer to be compressed according to the number of the blocks to be compressed of the two ways includes:

when the number of the blocks to be compressed of the two paths is not zero, determining the current output process according to the number of the blocks to be compressed which are input into the cache of the blocks to be compressed in the output processes of the two paths; the output process is a process of inputting a preset number of blocks to be compressed from the same path into the cache of the blocks to be compressed;

judging whether the current output process is finished or not;

when the current output process is not finished, inputting the block to be compressed of the path of the current output process into the block cache to be compressed;

and when the current output process is finished, setting the number of the blocks to be compressed of the current channel, which are input into the block cache to be compressed in the current output process, to zero, and inputting the blocks to be compressed of another channel into the block cache to be compressed.

Preferably, the step of inputting the block to be compressed of one of the two ways into the block buffer to be compressed according to the number of the blocks to be compressed of the two ways further includes:

and when the number of the blocks to be compressed of only one path is not zero, inputting the blocks to be compressed of the path with the number not zero into the block cache to be compressed.

Preferably, the step of determining the current output process according to the number of the to-be-compressed blocks that have been input into the to-be-compressed block cache in the output process of the two paths includes:

taking the output process of the path with a larger number of the blocks to be compressed, which is input into the cache of the blocks to be compressed, in the output processes of the two paths as the current output process;

and when the number of the blocks to be compressed which are input into the block cache to be compressed in the output processes of the two paths is the same, adopting the output process of one path which is specified in advance as the current output process.

Meanwhile, the application also discloses a video data output method, which is applied to a video post-processing system, wherein the video post-processing system comprises: the processing core, the compressed code stream cache and the video output module; the method comprises the following steps:

receiving a pixel block acquisition request, the pixel block acquisition request comprising: the processing core submits an acquisition request for acquiring a first-class pixel block, or the video output module submits an acquisition request for acquiring the first-class pixel block and a second-class pixel block;

after receiving the pixel block acquisition request, requesting a compressed code stream corresponding to the type of the requested pixel block in the pixel block acquisition request from the compressed code stream cache;

decoding the compressed code stream to generate a decompressed pixel block;

and outputting the decompressed pixel block to the processing kernel or the video output module which submits the pixel block acquisition request.

This application also discloses video data input device simultaneously, is applied to video post-processing system, video post-processing system includes: caching a block to be compressed and caching a compressed code stream; the device comprises:

the receiving module is used for receiving video frame data to be processed from at least two paths;

the segmentation module is used for segmenting the video frame data to be processed into a plurality of blocks to be compressed;

the device comprises a to-be-compressed block input module, a to-be-compressed block cache module and a to-be-compressed block cache module, wherein the to-be-compressed block input module is used for inputting a to-be-compressed block of one path into the to-be-compressed block cache according to the number of the to-be-compressed blocks of two paths;

the compression module is used for sequentially compressing the blocks to be compressed in the block cache to be compressed to generate a compressed code stream;

and the compressed code stream input module is used for inputting the compressed code stream into the compressed code stream cache.

Preferably, the block to be compressed input module further includes:

the current output process determining submodule is used for determining the current output process according to the number of the blocks to be compressed which are input into the block to be compressed cache in the output processes of the two paths when the number of the blocks to be compressed of the two paths is not zero; the output process is a process of inputting a preset number of blocks to be compressed from the same path into the cache of the blocks to be compressed;

the judgment submodule is used for judging whether the current output process is finished or not;

the first input submodule is used for inputting the block to be compressed of the path of the current output process into the block cache to be compressed when the current output process is not finished;

and the second input submodule is used for setting the number of the blocks to be compressed of the current channel, which are input into the block cache to be compressed in the current output process, to zero and inputting the blocks to be compressed of the other channel into the block cache to be compressed when the current output process is finished.

Preferably, the block to be compressed input module further includes:

and the third input submodule is used for inputting the blocks to be compressed of one path with the number not being zero into the block cache to be compressed when the number of the blocks to be compressed of only one path is not zero.

Preferably, the current output process determining sub-module further includes:

the first determining submodule is used for taking the output process of the path with a larger number of the blocks to be compressed which are input into the cache of the blocks to be compressed in the output processes of the two paths as the current output process;

and the second determining submodule is used for adopting the output process of one of the pre-specified paths as the current output process when the number of the blocks to be compressed which are input into the block cache to be compressed in the output processes of the two paths is the same.

This application also discloses video data output device simultaneously, is applied to video post-processing system, video post-processing system includes: the processing core, the compressed code stream cache and the video output module; the device comprises:

a request receiving module, configured to receive a pixel block acquisition request, where the pixel block acquisition request includes: the processing core submits an acquisition request for acquiring a first-class pixel block, or the video output module submits an acquisition request for acquiring the first-class pixel block and a second-class pixel block;

a compressed code stream request module, configured to, after receiving the pixel block acquisition request, request a compressed code stream corresponding to the type of the pixel block requested in the pixel block acquisition request from the compressed code stream cache;

the decoding module is used for decoding the compressed code stream to generate a decompressed pixel block;

and the output module is used for outputting the decompressed pixel block to the processing core or the video output module which submits the pixel block acquisition request.

This application also discloses video data system simultaneously, is applied to video post-processing system, video post-processing system includes: the system comprises a block to be compressed cache, a compressed code stream cache, a processing kernel and a video output module, wherein the system comprises: a video data input device and a video data output device;

the video data input device comprises:

a compressed code stream input module for inputting the compressed code stream into the compressed code stream cache;

the video data output device comprises:

The embodiment of the application has the following advantages:

according to the embodiment of the application, the video frame data is firstly compressed, and then the compressed code stream obtained by compression is input into the compressed code stream cache, so that the access bandwidth and the power consumption of the compressed code stream cache are reduced;

when the ultrahigh-definition video frame rate up-conversion processing core or the video output module requests video frame data, the compressed code stream in the compressed code stream cache is decompressed and then output to the ultrahigh-definition video frame rate up-conversion processing core or the video output module, and when the ultrahigh-definition video frame rate up-conversion processing core or the video output module can acquire the video frame data in time, the access bandwidth and the power consumption of the compressed code stream cache are reduced;

in addition, the processing time sequences are sequentially set for the original video frame data and the interpolated video frame data through the preset rule, and the data with different processing time sequences can be compressed by using only one compression kernel; the data of different processing time sequences can be decompressed by using only one decoding kernel, so that the occupation of the circuits for compressing and decompressing the kernels on the space is reduced, and the equipment cost is reduced.

Drawings

Fig. 1 is a schematic diagram of a conventional ultra high definition video frame rate up-conversion system;

FIG. 2 is a flow chart of the steps of an embodiment of a video data input method of the present application;

fig. 3 shows the results of the comparison of YCbCr 4: 2: 2, a schematic diagram of segmenting video frame data in a sampling format;

fig. 4 shows the results of the comparison of YCbCr 4: 4: 4, a schematic diagram of segmenting video frame data in a sampling format;

fig. 5 shows the results of the comparison of YCbCr 4: 2: a schematic diagram of segmenting video frame data in a 0 sampling format;

FIG. 6 is a flow chart of steps of an embodiment of a video data output method of the present application;

FIG. 7 is a block diagram of an embodiment of a video data system according to the present application;

FIG. 8 is a schematic structural diagram of a compression module in an embodiment of the present application;

FIG. 9 is a diagram illustrating an output process of an arbitration sub-module in an embodiment of the present application;

FIG. 10 is a schematic structural diagram of a compression core in an embodiment of the present application;

fig. 11 is a schematic structural diagram of a decompression module in an embodiment of the present application;

FIG. 12 is a schematic structural diagram of a decoding core in an embodiment of the present application;

FIG. 13 is a block diagram of an embodiment of a video data input device according to the present application;

fig. 14 is a block diagram of a video data output apparatus according to an embodiment of the present application;

fig. 15 is a block diagram of an embodiment of a video data system according to the present application.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present application more comprehensible, the present application is described in further detail with reference to the accompanying drawings and the detailed description.

Referring to fig. 1, a schematic diagram of an existing ultra-high-definition video frame rate up-conversion system is shown, which specifically includes: the system comprises a video input module 11, an off-chip cache 12, an ultra-high definition video frame rate up-conversion processing core 13 and a video output module 14; the video input module 11 inputs the original video frame to the off-chip cache 12; the ultra-high definition video frame rate up-conversion processing core 13 extracts an original video frame from the off-chip cache 12, performs a series of motion estimation, vector post-processing and interpolation operations on the extracted original video frame to generate an interpolated video frame, and finally sends the interpolated video frame to the off-chip cache 12; the video output module 13 extracts the original video frame data and the interpolated video frame data from the off-chip buffer 12 for output and display. However, due to the bandwidth limitation of the off-chip cache 12, the video frame data cannot be transmitted in time, so that the image is delayed in displaying.

One of the core ideas of the embodiment of the application is that video frame data is compressed, and then a compressed code stream generated by the compression processing is input into a compressed code stream cache; when the ultrahigh-definition video frame rate up-conversion processing core needs video frame data, the compressed code stream in the compressed code stream cache is decoded and then input into the ultrahigh-definition video frame rate up-conversion processing core.

Referring to fig. 2, a flow chart of steps of an embodiment of a video data input method of the present application is shown, the method is applied to video post-processing, and the video post-processing includes: caching a block to be compressed and caching a compressed code stream; the method specifically comprises the following steps:

step 101, receiving video frame data to be processed from at least two paths;

in an embodiment of the present application, the video post-processing system may include an ultra high definition video frame rate up-conversion system;

the method of the embodiment of the application aims at a scene that video frame data of different channels need to be compressed in video data compression processing, for example, an ultra-high-definition video frame rate up-conversion system.

Except for the application of the system for converting the ultra-high-definition video frame rate, the method of the embodiment of the application is also suitable for other scenes needing to compress the video frame data of different channels.

In this embodiment of the application, the ultra high definition video frame rate up-conversion system may include a video input module, an ultra high definition video frame rate up-conversion processing core, a video output module, a block to be compressed cache, and a compressed code stream cache;

the path includes: a data path for receiving original video frame data input by the video input module and a data path for receiving interpolated video frame data generated by the up-conversion processing core of the ultra-high definition video frame rate;

the video frame data to be processed comprises: raw video frame data and: interpolating video frame data;

step 102, segmenting the video frame data to be processed into a plurality of blocks to be compressed;

respectively cutting the original video frame data and the interpolated video frame data obtained from the two paths into a plurality of independent blocks to be compressed;

103, inputting the block to be compressed of one of the two paths into the block cache to be compressed according to the number of the blocks to be compressed of the two paths;

when the original video frame data and/or the interpolated video frame data are obtained from two paths, the video frame data are cut into a plurality of independent blocks to be compressed

According to the number of original video frame blocks to be compressed generated by current segmentation and the number of interpolated video frame blocks to be compressed, whether the original video frame blocks to be compressed or the interpolated video frame blocks to be compressed are input into a block cache to be compressed

In the implementation of the application, only one compression kernel is adopted to compress the block to be compressed, and the prepared time point of the block to be compressed and the available time point of the compression kernel have the problem of mismatching, so that the possible loss of the block to be compressed and the possible long-time idling of the compression kernel are caused, therefore, a cache of the block to be compressed is arranged to cache the block to be compressed, and when the compression kernel is available, the blocks to be compressed in the cache of the block to be compressed are sequentially sent to the compression kernel to be processed; the compression processing of the to-be-compressed block by the compression kernel may include: predicting, quantizing and entropy coding;

step 104, compressing the blocks to be compressed in the cache of the blocks to be compressed in sequence to generate a compressed code stream;

the compression kernel sequentially compresses the blocks to be compressed provided by the block cache to be compressed to generate a compressed code stream;

and 105, inputting the compressed code stream into the compressed code stream cache.

And according to the sequence of the generation of the compressed code stream, the compressed code stream is sequentially input into the compressed code stream cache, and the compressed code stream is generated by compressing the video frame data, so that the size of the video frame data is reduced, and the cache access bandwidth is effectively reduced.

In a preferred example of the embodiment of the present application, the step 102 may specifically be:

and segmenting the video frame data to be processed into a plurality of independent blocks to be compressed of brightness and blocks to be compressed of chroma according to the sampling mode of the video frame data.

The size of the block to be compressed may be specifically set according to a manner that the ultra high definition video frame rate up-conversion processing core reads data, and in a specific implementation, each time data is fetched by the ultra high definition video frame rate up-conversion processing core, 64 pixels in one row of video frame data are read, so that it may be considered that the block to be compressed is set into a plurality of matrices of 64 × N, and the number of rows N may be adjusted according to an actual compression effect.

According to different sampling modes of video frame data, the video frame data can be cut into a plurality of independent blocks to be compressed for brightness and blocks to be compressed for chrominance, and the common sampling of the video frame data comprises the following steps: YCbCr 4: 2: 2. YCbCr 4: 4: 4. YCbCr 4: 2: 0. YCbCr is part of the development of the world digital organization video standard as a recommendation in ITU-R BT.601, where Y refers to the luminance component, Cb to the blue chrominance component, and Cr to the red chrominance component; 4: 2: 0 means 4 luminance components per 4 pixels, 2 chrominance components (yyycbcr), sampling only odd scan lines, is the most common format for portable video devices (MPEG-4) and video conferencing (h.263); 4: 2: 2 denotes 4 luminance components per 4 pixels, 4 chrominance components (yyycbcrcbcr), the most common format for DVD, digital television, HDTV and other consumer video devices; 4: 4: 4 full pixel lattice (yyycbcrcbcrcbcrcbcr) for high quality video applications, studios and professional video production.

Referring to fig. 3, the comparison result of YCbCr 4: 2: 2, a schematic diagram of segmenting video frame data in a sampling format; the luminance components of 64 pixels in two consecutive lines in the video frame data are taken as a block to be compressed of a 64 × 2 matrix, and the 32 × 2 Cb components and the 32 × 2 Cr components corresponding to the pixel points are combined together to be taken as a block to be compressed of a 64 × 2 matrix.

Referring to fig. 4, in the embodiment of the present application, for YCbCr 4: 4: 4, a schematic diagram of segmenting video frame data in a sampling format; the luminance components of 64 pixels in two consecutive lines in the video frame data are used as a block to be compressed of a 64 × 2 matrix, 64 × 2 Cb components corresponding to the pixel points are used as a block to be compressed of a 64 × 2 matrix, and 64 × 2 Cr components corresponding to the pixel points are used as a block to be compressed of a 64 × 2 matrix.

Referring to fig. 5, the comparison result of YCbCr 4: 2: a schematic diagram of segmenting video frame data in a 0 sampling format; the luminance components of 64 pixels in four consecutive rows in the video frame data are taken as two blocks to be compressed in a 64 × 2 matrix, and the 32 × 2 Cb components and the 32 × 2 Cr components corresponding to the pixel points are combined to be taken as a block to be compressed in a 64 × 2 matrix.

Determining which path block to be compressed is input into a block to be compressed for caching according to the number of blocks to be compressed obtained by segmenting video frame data from two paths at present;

as a preferred example of the embodiment of the present application, the step 103 may specifically include the following sub-steps:

substep S11, when the number of the blocks to be compressed of the two paths is not zero, determining the current output process according to the number of the blocks to be compressed which are input into the block cache to be compressed in the output process of the two paths; the output process is a process of inputting a preset number of blocks to be compressed from the same path into the cache of the blocks to be compressed;

and determining the current output process according to the output processes of the two paths when the number of the blocks to be compressed of the original video frame generated by current segmentation and the number of the blocks to be compressed of the interpolated video frame are not zero.

The output process may include: inputting the original video frame blocks to be compressed or the video frame blocks to be compressed inserted from the same path with preset number into the block cache to be compressed.

In the embodiment of the present application, the blocks to be compressed need to be input into the block cache to be compressed one by one to determine the input and output order of the blocks to be compressed. Therefore, the process of inputting the preset number of blocks to be compressed into the block cache to be compressed by the output process of one data path cannot be completed simultaneously.

Because the blocks to be compressed can only be input into the block cache to be compressed one by one, when the number of the blocks to be compressed from the two paths is not zero, the block to be compressed of one data path needs to be selected for input, that is, the output process of one data path is selected as the current output process.

As a preferred example of the embodiment of the present application, in the sub-step S11, the step of determining the current output process according to the number of blocks to be compressed that have been input into the block buffer to be compressed in the output process of two paths may include the sub-steps of:

a substep S210, taking the output process of the path with a larger number of the blocks to be compressed, which is input into the cache of the blocks to be compressed, in the output process of the two paths as the current output process;

and when the number of the blocks to be compressed from the two paths is not zero, taking the output process of one path with a larger number of the blocks to be compressed, which is input into the cache of the blocks to be compressed, as the current output process.

And a substep S211, when the number of the blocks to be compressed which are input into the block cache to be compressed in the output process of the two paths is the same, adopting the output process of one path which is specified in advance as the current output process.

When the number of the blocks to be compressed from both paths is not zero and the number of the blocks to be compressed that have been input into the block buffer to be compressed in the output process of both paths is the same, the output process of one of the paths may be used as the output process of the priority processing. For example, the output of the block to be compressed of the data path of interpolated video frame data may be prioritized.

In the embodiment of the present application, the preset number may be specifically set according to the capacity of the compressed code stream to cache write data; in the embodiment of the present application, the compressed code stream cache may be a Synchronous Dynamic Random Access Memory (SDRAM), the number of data transmitted by the SDRAM in each burst write is 8, and 256 bits of data are transmitted in each clock beat, that is, 8 256 bits of data are output by the data channel when the address channel outputs an address once;

in this embodiment of the application, each block to be compressed includes 128 pixels, each pixel is 10 bits, the total block to be compressed is 1280 bits, and the compressed code stream after the compression processing can be set to be 512 bits (the size of the fixed code stream is equivalent to the fixed compression ratio).

The compressed code stream of the block to be compressed after the compression processing in step 104 is 512 bits, so that the compressed code stream generated by the 4 blocks to be compressed of the same type after the compression processing in step 104 can be split into 8 pieces of 256-bit data, and the data can be burst written into the SDRAM at one time.

Therefore, the process of inputting the compressed code stream generated by the step 104 into the compressed code stream cache for 4 blocks to be compressed of the same type is defined as an output process.

In the embodiment of the application, the advantage of setting the compressed code stream to 512 bits is that, on one hand, the distortion condition of the image is considered, and the requirement of a certain compression ratio is met, and on the other hand, the characteristic of reading and writing data of the SDRAM is met.

And the block cache to be compressed performs step 104 on the first block to be compressed and the second block to be compressed according to a first-in first-out principle to generate a compressed code stream, so that an output process can also be defined as a process of continuously inputting four blocks to be compressed of the same type into the block cache to be compressed. The output process of the blocks to be compressed with a large number in the blocks to be compressed which are input into the cache of the blocks to be compressed is the current output process.

The method has the advantages that the preset number of blocks to be compressed from the same path are input into the cache of the blocks to be compressed, and the compressed code stream cache is convenient to generate addresses of different areas.

For example, for the compressed code stream of the channel A, the compressed code stream cache gives the address in the same area, for the compressed code stream of the channel B, the compressed code stream cache gives the address of the other area, so that the two types of compressed code streams can be found out respectively when the code streams request.

A substep S12 of determining whether the current output process is completed;

judging whether the number of the blocks to be compressed, which are cached in the blocks to be compressed and input by the path in the current output process, reaches a preset number or not; if not, the current output process is finished; if not, the current output process is not finished;

a substep S13, when the current output process is not completed, inputting the block to be compressed of the path of the current output process into the block to be compressed cache;

and if the number of the blocks to be compressed, which are input to the block cache to be compressed by the path in the current output process, does not reach the preset number, the blocks to be compressed of the path in the current output process are input to the block cache to be compressed.

And a substep S14, when the current output process is completed, setting the number of the blocks to be compressed of the current channel, which is input into the block cache to be compressed in the current output process, to zero, and inputting the blocks to be compressed of another channel into the block cache to be compressed.

And the number of the blocks to be compressed which are input into the cache of the blocks to be compressed by the path in the current output process reaches a preset number, namely the current output process is completed. After the current output process is finished, setting the number of the blocks to be compressed, written into the block cache to be compressed, of the path of the current input process to zero, namely resetting the number of the blocks to be compressed input in the output process of the path; and inputting the block to be compressed of the other channel into the block cache to be compressed, namely performing the output process of the other path.

As a preferred example of the embodiment of the present application, the step 103 may further include the following sub-steps:

and a substep S15, when the number of the blocks to be compressed of only one path is not zero, inputting the blocks to be compressed of one path with the number not zero into the block buffer to be compressed.

When the number of the blocks to be compressed of only one path is not zero, the blocks to be compressed of the path with the number not zero are input into the block cache to be compressed, namely, the output process of the path with the number not zero is carried out.

In a preferred example of the embodiment of the present application, the compression processing includes: general compression processing and special compression processing;

the step 104 may specifically include the following sub-steps:

the substep S21, carrying out general compression processing on the compressed code stream to generate a general compressed code stream;

in a specific implementation, the general compression process may specifically include: prediction processing, quantization processing, entropy coding processing, inverse quantization and pixel reconstruction processing, and code stream packaging processing to finally generate a compressed code stream;

the substep S22, judging whether the general compressed code stream meets the preset compression rate requirement;

under the condition of meeting certain image distortion, the requirement of a compression ratio can be met by carrying out general compression processing on the image with stronger image correlation; however, for an image with weak image correlation, when a certain image distortion needs to be satisfied, the compression rate cannot meet the compression rate requirement.

And a substep S23, when the general compressed code stream does not meet the preset compression rate requirement, performing special compression processing on the compressed code stream. And special code stream processing is carried out on the pixels of the block to be compressed, and compression processing with fixed compression ratio is carried out.

Referring to fig. 6, a flowchart illustrating steps of an embodiment of a video data output method of the present application is applied to a video post-processing system, the video post-processing system comprising: the processing core, the compressed code stream cache and the video output module; the method specifically comprises the following steps:

step 201, receiving a pixel block acquisition request, where the pixel block acquisition request includes: the processing core submits an acquisition request for acquiring a first-class pixel block, or the video output module submits an acquisition request for acquiring the first-class pixel block and a second-class pixel block;

in the embodiment of the application, the video post-processing system can comprise an ultra-high-definition video frame rate up-conversion processing system, and meanwhile, the processing kernel can be an ultra-high-definition video frame rate up-conversion processing kernel;

the ultrahigh-definition video frame rate up-conversion processing kernel requests pixel blocks of original video frame data, and the video output module requests the pixel blocks of the original video frame data and pixel blocks of interpolated video frame data.

Step 202, after receiving the pixel block acquisition request, requesting a compressed code stream corresponding to the type of the requested pixel block in the pixel block acquisition request from the compressed code stream cache;

the compressed code stream cache stores a compressed code stream of original video frame data and a compressed code stream of interpolated video frame data;

step 203, decoding the compressed code stream to generate a decompressed pixel block;

the compressed code stream specifically includes: an original video frame compressed code stream and an interpolated video frame compressed code stream;

in a preferred example of the embodiment of the present application, the step 202 may include the following sub-steps:

substep S31, decoding the compressed code stream of the original video frame to generate a pixel block of the decompressed original video frame;

and a substep S32, decoding the interpolation video frame compressed code stream to generate a decompressed interpolation video frame pixel block.

In a preferred example of the embodiment of the present application, the decoding process includes: a general decoding process and a special decoding process; the compressed code stream includes: general compressed code stream and special compressed code stream;

the step of decoding the compressed code stream may include the following substeps:

the substep S41, judging whether the compressed code stream is a general compressed code stream or a special compressed code stream;

the general compressed code stream is generated by performing general compression processing on a block to be compressed, and the special code stream is generated by performing special compression processing on the block to be compressed;

substep S42, when the compressed code stream is a general compressed code stream, performing general decoding processing on the general compressed code stream;

performing general decoding processing on the general compressed code stream, where the general decoding processing specifically includes: code stream analysis processing, entropy decoding processing, inverse quantization processing, pixel formation processing and pixel block restoration processing;

and a substep S43, when the compressed code stream is a special compressed code stream, performing special decoding processing on the special compressed code stream.

And 204, outputting the decompressed pixel block to the processing core or the video output module submitting the pixel block acquisition request.

Wherein the decompressed pixel block may comprise: decompressing the original video frame pixel block and decompressing the interpolation video frame pixel block; in a preferred example of the embodiment of the present application, step 204 may specifically include the following sub-steps:

a substep S51, outputting the decompressed original video frame pixel block to the ultrahigh-definition video frame rate up-conversion processing kernel;

and a substep S52 of outputting the decompressed original video frame pixel blocks and the decompressed interpolated video frame pixel blocks to the video output module.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the embodiments are not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the embodiments. Further, those skilled in the art will also appreciate that the embodiments described in the specification are presently preferred and that no particular act is required of the embodiments of the application.

Fig. 7 is a schematic structural diagram of embodiment 2 of a video data system according to the present application. The video data system is applied to an ultra-high-definition video frame rate up-conversion system, and the ultra-high-definition video frame rate up-conversion system comprises: the video data system comprises a video input module 31, an ultra-high-definition video frame rate up-conversion processing core 32, a compressed code stream cache 33 and a video output module 34, wherein the video data system comprises:

a compression module 21, configured to receive original video frame data output by the video input module and interpolated video frame data output by the ultra-high definition video frame rate up-conversion processing core; compressing the original video frame data to generate a first compressed code stream; compressing the interpolated video frame data to generate a second compressed code stream; outputting the first compressed code stream and the second compressed code stream to the compressed code stream cache;

a decompression module 22, configured to request the first compressed code stream and the second compressed code stream from the compressed code stream cache; decompressing the first compressed code stream to generate an original frame decompressed pixel block; decompressing the second compressed code stream to generate an interpolation frame decompression pixel block; outputting the original frame decompression pixel block to the ultrahigh-definition video frame rate up-conversion processing core; outputting the original frame decompressed pixel block and the interpolated frame decompressed pixel block to the display module; the ultrahigh-definition video frame rate up-conversion processing core is used for generating interpolated video frame data by adopting the original frame decompression pixel block; the display module is configured to display the original frame decompressed pixel block and the interpolated frame decompressed pixel block.

Referring to fig. 8, a schematic structural diagram of the compression module 21 in the embodiment of the present application is shown, which may specifically include: a first block to be compressed forming submodule 2101, a first input control submodule 2102, a second block to be compressed forming submodule 2103, a second input control submodule 2104, an arbitration submodule 2105, a first cache submodule 2106, a compression core 2107, a first output control submodule 2108, and an address generating submodule 2109.

A first block to be compressed forming submodule 2101 connected to the video input module 31, the first input control submodule 2102 and the arbitration submodule 2105; the video compression module is used for receiving the original video frame data output by the video input module 31, and performing splitting processing on the original video frame data according to the sampling mode of the original video frame data to generate a plurality of first blocks to be compressed; and informs the arbitration submodule 2105 and the first input control submodule 2102 of the status of itself.

A first input control submodule 2102 connected to the arbitration submodule 2105, the video input module 31, and the first block to be compressed forming submodule 2101; for generating an input enable signal to control the data input of the video input module 31 under the control of the first block to be compressed forming submodule 2101 and the arbitration submodule 2105. In the case where the first block to be compressed formation sub-module 2101 can receive data, the data input data of the video input module 31 is controlled; when the data cannot be received temporarily, the control suspends the input of the data. The data overflow caused by too fast data input can be ensured, and the idling of the system caused by long-time data non-input can be prevented.

The first input control submodule 2102 generates an input enable signal when:

1. the first block to be compressed forming submodule 2101 has not received the original video frame data from the video input module 31;

2. the first block to be compressed forming submodule 2101 has received the original video frame data from the video input module 31, but not the entire first block to be compressed;

3. the first block to be compressed forming submodule 2101 has received the entire first block to be compressed data and the arbitration submodule 2105 has transmitted the first block to be compressed within all of the first block to be compressed forming submodule 2101 to the first cache submodule 2106.

In the case where the first block to be compressed formation submodule 2101 has received the entire first block to be compressed, and the arbitration submodule 2105 does not transfer all of the blocks to be compressed within the first block to be compressed formation submodule 2101 to the first buffer submodule 2106, the first input control submodule 2102 does not generate an input enable signal.

The second block to be compressed forming submodule 2103 is connected with the ultra-high-definition video frame rate up-conversion processing core 32, the second input control submodule 2104 and the arbitration submodule 2105; for receiving interpolated video frame data output by the ultra high definition video frame rate up-conversion processing core 32; splitting the interpolated video frame data according to the sampling mode of the interpolated video frame data to generate a plurality of second blocks to be compressed; and informs the arbitration sub-module 2105 and the second input control sub-module 2104 of their status.

The second input control sub-module 2104, together with the arbitration sub-module 2105, the ultra-high definition video frame rate up-conversion processing core 32, and the second block to be compressed forming sub-module 2103; and the module is used for generating an input enable signal and controlling the data input of the ultra high definition video frame rate up-conversion processing core 32 under the control of the second block to be compressed forming submodule 2103 and the arbitration submodule 2105. Under the condition that the second block-to-be-compressed forming submodule 213 can receive data, controlling the ultrahigh-definition video frame rate up-conversion processing kernel 32 to input data; when the data cannot be received temporarily, the control suspends the input of the data. The data overflow caused by too fast data input can be ensured, and the idling of the system caused by long-time data non-input can be prevented.

The second input control sub-module 2104 generates an input enable signal when:

1. the second block-to-be-compressed forming submodule 2103 has not received the interpolated frame data from the ultra high definition video frame rate up-conversion processing core 32;

2. the second block-to-be-compressed forming submodule 2103 has received the interpolated frame data from the ultra high definition video frame rate up-conversion processing core 32, but not the entire second block-to-be-compressed data;

3. the second block to be compressed forming submodule 2103 has received the entire second block to be compressed, and the arbitration submodule 2105 has transmitted all the second blocks to be compressed within the second block to be compressed forming submodule 2103 to the first cache submodule 2106.

In the case where the second block-to-be-compressed forming submodule 2103 has received the entire second block-to-be-compressed data, and the arbitration submodule 2105 has not transmitted all the blocks-to-be-compressed within the second block-to-be-compressed forming submodule 2103 to the first buffer submodule 2106, the second input control submodule 2104 does not generate an input enable signal.

An arbitration sub-module 2105 connected to the first block to be compressed forming sub-module 2101, the first input control sub-module 2102, the second block to be compressed forming sub-module 2103, the second input control sub-module 2104 and the first buffer sub-module 2106; a second input control sub-module 2104 for receiving a second block to be compressed output from the first input control sub-module 2102 and a second block to be compressed output from the second input control sub-module 2102; the first block to be compressed or the second block to be compressed is input to the first cache submodule 2106 after being arbitrated according to the available state of the first block to be compressed currently generated by the first block to be compressed forming submodule 2101 and the available state of the second block to be compressed currently generated by the second block to be compressed forming submodule 2102.

The available state of the first block to be compressed currently generated by the first block to be compressed forming submodule 2101 is specifically that the first block to be compressed currently generated by the first block to be compressed forming submodule 2101 meets a preset state condition, and the preset state condition may be set according to a sampling mode of video frame data: specifically, for example,

in the following step 4: 2: in the 2-sample mode, the first block to be compressed forming sub-module 2101 currently generates a first block to be compressed, which includes: a luminance block to be compressed and a chrominance block to be compressed;

in the following step 4: 4: in the 4-sample mode, the first block to be compressed forming sub-module 2101 currently generates a first block to be compressed, which includes: a luminance block to be compressed and two chrominance blocks to be compressed;

in the following step 4: 2: in the 0 sampling mode, the first block to be compressed forming sub-module 2101 currently generates a first block to be compressed including: two luminance blocks to be compressed and one chrominance block to be compressed.

Depending on the sampling pattern, only at most the above number of first blocks to be compressed is possible in the first block-forming submodule 2101 at any time; after the above number of first blocks to be compressed is generated, the first block to be compressed forming submodule 2101 must output these first blocks to be compressed to the first buffer submodule 2106 before new video frame data can be accepted.

The available state of the second block to be compressed currently generated by the second block to be compressed forming submodule 2103 is similar to the available state of the first block to be compressed currently generated by the first block to be compressed forming submodule 2101, and details are not repeated here, and reference may be made to the available state of the first block to be compressed currently generated by the first block to be compressed forming submodule 2101.

The first cache submodule 2106 is connected with the arbitration submodule 2105 and the compression core 2107; the buffer is used for caching the first block to be compressed and/or the second block to be compressed; when the compression core 2107 is available, outputting the received first block to be compressed or second block to be compressed to the compression core 2107 in sequence; the first cache submodule 2106 may specifically be an SRAM cache.

The compression core 2107 is connected with the first cache submodule 2106 and the first output control submodule 2108, and is used for compressing the first block to be compressed to generate a first compressed code stream; compressing the second block to be compressed to generate a second compressed code stream; and outputting the first compressed code stream or the second compressed code stream to the first output control sub-module 2108.

The first output control submodule 2108 is connected with the compression kernel 2107 and the compressed code stream cache 33; the first compressed code stream or the second compressed code stream is received; sending an input request to the compressed code stream cache 33 according to the number of the first compressed code stream or the second compressed code stream; and inputs the first compressed code stream or the second compressed code stream to the compressed code stream cache 33 according to the input address under the cooperation of the input address generated by the address generation submodule 2109.

The address generation submodule 2109 is connected with the compressed code stream cache 33; the cache module is used for generating an input address according to a preset cache rule; after the first output control sub-module 2108 sends the input request to the compressed code stream cache 33 and obtains the confirmation signal of the compressed code stream cache 33, the input address is sent to the compressed code stream cache 33.

FIG. 9 is a schematic diagram of an output process of an arbitration sub-module in the embodiment of the present application;

judging whether the number of first blocks to be compressed generated by the current first block forming submodule and the number of second blocks to be compressed generated by the second block forming submodule are both not zero;

if the number of the first to-be-compressed block generated by the current first to-be-compressed block forming submodule and the number of the second to-be-compressed block generated by the second to-be-compressed block forming submodule are not zero, judging whether the current output process is finished; the output process comprises the following steps: a process of inputting a preset number of first blocks to be compressed into the first cache sub-module or a process of inputting a preset number of second blocks to be compressed into the first cache sub-module. And the current output process is determined according to the number of the first blocks to be compressed which are input into the first cache submodule in the output process of the first blocks to be compressed and the number of the second blocks to be compressed which are input into the first cache submodule in the output process of the second blocks to be compressed. When the number is different, taking the output process with a larger number as the current output process; when the number is the same, the output process of the first block to be compressed or the output process of the second block to be compressed may be set in advance as the current output process.

If the current output process is finished, setting the number of the blocks to be compressed, which are input into the first cache sub-module in the current output process, to zero; and performing an output process of the other type of block to be compressed, and inputting one block to be compressed of the other type into the first cache submodule.

And if the current output process is not finished, continuing the current output process, and inputting a block to be compressed corresponding to the current output process into the first cache submodule.

If the number of the first to-be-compressed block generated by the current first to-be-compressed block forming submodule and the number of the second to-be-compressed block generated by the second to-be-compressed block forming submodule are not both zero, judging whether the number of the first to-be-compressed block generated by the current first to-be-compressed block forming submodule and the number of the second to-be-compressed block generated by the second to-be-compressed block forming submodule are only one type and are not zero;

if the number of the first to-be-compressed block generated by the current first to-be-compressed block forming submodule and the number of the second to-be-compressed block generated by the second to-be-compressed block forming submodule are only one of the numbers, the number of the first to-be-compressed block is not zero, the output process of the non-zero one of the numbers of the first to-be-compressed block is carried out, and one to-be-compressed block is output to the first cache submodule;

if the number of the first to-be-compressed block generated by the current first to-be-compressed block forming submodule and the number of the second to-be-compressed block generated by the second to-be-compressed block forming submodule are both zero, waiting for a clock;

and waiting for one clock after outputting one block to be compressed to the first cache submodule.

Fig. 10 is a schematic structural diagram of a compression kernel in the embodiment of the present application, which specifically includes:

a prediction module 3101, a quantization module 3102, an entropy coding module 3103, a code stream packing module 3104, an inverse quantization and pixel reconstruction module 3105, a compression rate special processing module 3106, a compressed code stream output module 3107;

a prediction module 3101, connected to the quantization module 3102 and the inverse quantization and pixel reconstruction module 3105, for receiving the first to-be-compressed block or the second to-be-compressed block output by the first buffer sub-module 2106 and the first reconstruction value or the second reconstruction value provided by the inverse quantization and pixel reconstruction module 3105; performing prediction processing and residual error processing on each first pixel to be compressed of the first block to be compressed according to the first reconstruction value to obtain a residual error of each first pixel to be compressed; performing prediction processing and residual error processing on each second pixel to be compressed of the second block to be compressed according to the second reconstruction value to obtain a residual error of each second pixel to be compressed; and outputs the residual of the first pixel to be compressed and the residual of the second pixel to be compressed to the quantization module 3102;

a quantization module 3102, connected to the prediction module 3101, the entropy coding module 3103, and the inverse quantization and pixel reconstruction module 3105, for receiving the residual of the first pixel to be compressed or the residual of the second pixel to be compressed, and performing quantization processing on the residual of the first pixel to be compressed to obtain a quantized residual of the first pixel to be compressed; quantizing the residual error of the second pixel to be compressed to obtain a quantized residual error of the second pixel to be compressed; and outputs the quantized residual of the first pixel to be compressed or the quantized residual of the second pixel to be compressed to the entropy coding module 3103 and the inverse quantization and pixel reconstruction module 3105.

An entropy coding module 3103, connected to the quantization module 3102 and the code stream packing module 3104, for receiving the quantized residual of the first pixel to be compressed or the quantized residual of the second pixel to be compressed, and performing entropy coding on the quantized residual of the first pixel to be compressed to obtain a coded pixel residual of the first pixel to be compressed; entropy coding is carried out on the quantized residual error of the second pixel to be compressed to obtain a coded pixel residual error of the second pixel to be compressed; and send the encoded pixel residual of the first pixel to be compressed or the encoded pixel residual of the second pixel to be compressed to the code stream packing module 3104.

A code stream packing module 3104, connected to the entropy coding module 3103 and the compressed code stream output module 3107, for receiving the encoded pixel residual of the first pixel to be compressed or the encoded pixel residual of the second pixel to be compressed; sequentially packing the received coded pixel residual of the first pixel to be compressed to generate a first packed subcode stream; sequentially packaging the received coded pixel residual errors of the second pixel to be compressed to generate a second packaged subcode stream; and sends the first packetized sub-stream and the second packetized sub-stream to the compressed stream output module 3107.

An inverse quantization and pixel reconstruction module 3105, connected to the prediction module and the quantization module, for receiving a quantized residual of the first pixel to be compressed or a quantized residual of the second pixel to be compressed; carrying out inverse quantization processing and pixel reconstruction processing on the quantized residual error of the first pixel to be compressed to generate a first reconstruction value; carrying out inverse quantization processing and pixel reconstruction processing on the quantized residual error of the second pixel to be compressed to generate a second reconstruction value; and outputs the first reconstructed value or the second reconstructed value to the prediction module 3101.

A compression rate special processing module 3106, connected to the code stream output module, configured to receive the first block to be compressed or the second block to be compressed output by the first cache sub-module 2106 when the first packed subcode stream or the second packed subcode stream output by the code stream packing processing module does not meet a preset compression rate requirement, perform special compression processing on the first block to be compressed, and generate a first special code stream; and performing preset special compression processing on the second block to be compressed to generate a second special code stream.

A compressed code stream output module 3107, connected to the first cache submodule 2106, the code stream packing module 3104 and the compression rate special processing module 3106, for receiving the first to-be-compressed block or the second to-be-compressed block output by the first cache submodule 2106, the first packed sub-code stream or the second packed sub-code stream output by the code stream packing module 3104, and the first special code stream or the second special code stream output by the compression rate special processing module 3106; when the code stream length of the first packing subcode stream is smaller than the pixel value length of the first block to be compressed and the first packing subcode stream meets the preset compression ratio requirement, outputting the first packing subcode stream; and when the code stream length of the first packing subcode stream is more than or equal to the pixel value length of the first block to be compressed and the first packing subcode stream meets the preset compression rate requirement, directly using the pixel value of the first block to be compressed. When the code stream length of the second packed sub-code stream is smaller than the code stream length of the second block to be compressed and the second packed sub-code stream meets the preset compression rate requirement, outputting the second packed sub-code stream; and when the code stream length of the second packed sub code stream is greater than or equal to the pixel value length of the second block to be compressed and the second packed sub code stream meets the preset compression rate requirement, directly using the pixel value of the second block to be compressed. And outputting the first special code stream when the first packetized subcode stream does not meet the preset compression ratio requirement. And outputting the second special code stream when the second packed sub code stream does not meet the preset compression rate requirement.

Fig. 11 is a schematic structural diagram of a decompression module in the embodiment of the present application, which specifically includes:

a code stream request submodule 4101, a code stream receiving submodule 4102, a second cache submodule 4103, a decoding kernel 4104, a second output control submodule 4105, a display format conversion submodule 4106 and a display control submodule 4107;

a code stream request submodule 4101, connected to the compressed code stream cache 33 and the second cache submodule 4103, and configured to generate, according to a preset address acquisition rule and a state of the second cache submodule, an address of the compressed code stream cache 33 where a first compressed code stream or a second compressed code stream that needs to be requested is generated; sending a code stream acquisition request for requesting a first compressed code stream or a second compressed code stream to a compressed code stream cache, wherein the first compressed code stream or the second compressed code stream which needs to be requested is at the address of a compressed code stream cache 33; the address acquisition rule is the characteristic that the video output module 34 and the ultra high definition video frame rate up-conversion processing core 32 read data, and when the video output module 34 reads data, the data are sequentially read downwards from the first line of the video frame, and the data are read from left to right in the line; the ultra high definition video frame rate up-conversion processing core 32 reads data, generally reads a rectangular block of a video frame, and sometimes needs to read randomly.

A code stream receiving submodule 4102, connected to the compressed code stream cache 33 and the second cache submodule 4103, and configured to receive a data input signal sent by the compressed code stream cache 33 after receiving a code stream acquisition request sent by the code stream request submodule 4101; after receiving the data input signal, receiving a first compressed code stream or a second compressed code stream corresponding to the code stream acquisition request sent by the compressed code stream cache 33; and sends the received first compressed code stream or second compressed code stream to the second cache sub-module 4103;

the second cache submodule 4103 is connected with the code stream request submodule 4101, the code stream receiving submodule 4102 and the decoding kernel 4104, and is configured to cache the first compressed code stream or the second compressed code stream output by the code stream receiving submodule, and sequentially send the acquired first compressed code stream or the acquired second compressed code stream to the decoding kernel 4104; informing the code stream request submodule 4101 of the state of the code stream; the second cache sub-module 4103 may be specifically an SRAM cache, and mainly solves the mismatch between the code stream output of the code stream receiving sub-module 4102 and the time when the decoding core 4104 can receive the code stream. When the decoding kernel 4104 is idle and can receive the code stream, the code stream is sequentially output to the decoding kernel 4104 for decoding processing; meanwhile, the state of the self-body is notified to the code stream request submodule 4101 for the code stream request submodule 4101 to refer to the compressed code stream cache 33, so that the second cache submodule 4103 is prevented from overflowing or being empty for a long time.

A decoding kernel 4104, connected to the second cache sub-module 4103 and the second output control sub-module 4105, and configured to receive the first compressed code stream or the second compressed code stream output by the second cache sub-module 4105, perform decoding processing on the first compressed code stream, and generate a first decoding pixel block, or perform decoding processing on the second compressed code stream, and generate a second decoding pixel block; and outputs the first decoded pixel block or the second decoded pixel block to the second output control sub-module 4105.

A second output control submodule 4105, connected to the decoding kernel 4104, the display format conversion submodule 4106 and the high-definition video frame rate up-conversion processing kernel 32, and configured to receive the first decoded pixel block or the second decoded pixel block output by the decoding kernel 4104; sending the first decoded pixel block to the high definition video frame rate up-conversion processing kernel 32; the first decoded pixel block or the second decoded pixel block is sent to the display format conversion sub-module 4106.

A display format conversion submodule 4106, connected to the second output control submodule 4105, the display control submodule 4107 and the video output module 34, for splitting the first decoded pixel block to generate first display data, or splitting the second decoded pixel block to generate second display data; sending the first display data and the second display data to the video output module 34 according to a preset data transmission protocol; and sends a data valid signal, a frame synchronization signal, and a line synchronization signal to the video output module 34.

The display control sub-module 4107 is connected to the display format conversion sub-module 4106 and the video output module 34, and is configured to control data output of the display format conversion sub-module 4106 under the control of the input enable signal of the video output module 34. The display format conversion sub-module 4106 outputs data to the video output module 34 only when the video output module 34 can receive the data, while notifying the video output module 34 that the data on the data bus is valid through a data valid signal.

Fig. 12 is a schematic structural diagram of a decoding kernel in the embodiment of the present application, which specifically includes:

a code stream analysis module 5101, an entropy decoding module 5102, an inverse quantization module 5103, a pixel forming module 5104, a compression rate special processing decoding module 5105 and a pixel block restoration module 5106;

the code stream analyzing module 5101 is connected to the entropy decoding module 5102, and is configured to receive the first compressed code stream or the second compressed code stream output by the second cache sub-module 4103; if the first compressed code stream is a first packed subcode stream, analyzing the code stream to generate a first analyzed compressed code stream; if the first compressed code stream is a first special code stream, analyzing the code stream to generate a first analyzed special code stream; if the second compressed code stream is a second packed sub-code stream, analyzing the code stream to generate a second analyzed compressed code stream; if the second compressed code stream is a second special code stream, analyzing the code stream to generate a second analyzed special code stream; and sends the first parsed compressed code stream or the second parsed compressed code stream to the entropy decoding module 5102, and sends the first parsed special code stream or the second parsed special code stream to the compression rate special processing decoding module 5105.

The entropy decoding module 5102 is connected to the code stream analyzing module 5101 and the inverse quantization module 5103, and is configured to receive the first analyzed compressed code stream or the second analyzed compressed code stream output by the code stream analyzing module 5101, perform entropy decoding processing on the first analyzed compressed code stream, and generate a first decompressed code stream; entropy decoding the second analysis compressed code stream to generate a second decompressed code stream; and send the first decompressed code stream or the second decompressed code stream to the inverse quantization module 5103.

The inverse quantization module 5103 is connected to the entropy decoding module 5102 and the pixel formation module 5104, and is configured to receive the first decompressed code stream or the second decompressed code stream; carrying out inverse quantization processing on the first decompressed code stream to generate a first inverse quantized code stream; performing inverse quantization processing on the second decompressed code stream to generate a second inverse quantized code stream; and sending the first dequantized code stream or the second dequantized code stream to the pixel formation module 5104;

the pixel forming module 5104 is connected to the inverse quantization module 5103 and the pixel block restoring module 5106, and is configured to receive the first inverse quantization code stream or the second inverse quantization code stream output by the inverse quantization module 5103; performing pixel formation processing on the first inverse quantization code stream to generate a first decoded pixel block; performing pixel formation processing on the second inverse quantization code stream to generate a second decoded pixel block; and send the first decoded pixel block or the second decoded pixel block to the pixel block restoration module 5106;

a compression rate special processing decoding module 5105, connected to the code stream analyzing module 5101 and the pixel block restoring module 5106, and configured to receive the first analyzed special code stream or the second analyzed special code stream output by the code stream analyzing module 5101; performing special decoding processing on the first analysis special code stream to generate a first special decoding pixel block; performing special decoding processing on the second analysis special code stream to generate a second special decoding pixel block; and outputting the first special decoded pixel block or the second special decoded pixel block to the pixel block restoration module 5106;

a pixel block recovery module 5106, connected to the pixel formation module 5104 and the compression rate special processing decoding module 5105, and configured to receive the first decoded pixel block or the second decoded pixel block output by the pixel formation module 5104, or the first special decoded pixel block or the second special decoded pixel block output by the compression rate special processing decoding module 5105; the first decoded pixel block or the second decoded pixel block is sent to the second output control sub-module 4105. The first decoding pixel block includes: if the first compressed code stream is a first packed subcode stream, a first decoded pixel block; and if the first compressed code stream is the first special code stream, the first special decoding pixel block. The second decoding pixel block includes: if the second compressed code stream is a second packed sub-code stream, a second decoded pixel block; and if the second compressed code stream is a second special code stream, the second special decoding pixel block.

Referring to fig. 13, a block diagram of an embodiment of a video data input device according to the present application is shown, the device is applied to a video post-processing system 60, and the video post-processing system 60 includes: a block cache to be compressed 6001 and a compressed code stream cache 6002; the video data input device 70 may include:

in an embodiment of the present application, the video post-processing system may include an ultra high definition video frame rate up-conversion system.

The device may specifically include the following modules:

a receiving module 7001, configured to receive video frame data to be processed from at least two paths;

a segmentation module 7002, configured to segment the to-be-processed video frame data into a plurality of to-be-compressed blocks;

a block to be compressed input module 7003, configured to input a block to be compressed of one of the two ways into the block to be compressed cache 6001 according to the number of blocks to be compressed of the two ways;

a compression module 7004, configured to sequentially compress the blocks to be compressed in the block cache 6001 to be compressed, and generate a compressed code stream;

and a compressed code stream input module 7005, configured to input the compressed code stream to the compressed code stream cache 6002.

In a preferred example of the embodiment of the present application, the video frame data includes: original video frame data and interpolated video frame data;

the blocks to be compressed comprise a first block to be compressed generated by segmenting the original video frame data and a second block to be compressed generated by segmenting the interpolated video frame data;

the block to be compressed input module may further include:

As a preferred example of the embodiment of the present application, the block to be compressed input module may further include:

In a preferred example of the embodiment of the present application, the current output process determining sub-module further includes:

In a preferred example of the embodiment of the present application, the compression process may include: general compression processing and special compression processing;

the compression module may further include:

the first compression submodule is used for carrying out general compression processing on the compressed code stream to generate a general compressed code stream;

the compression ratio judgment submodule is used for judging whether the general compressed code stream meets the preset compression ratio requirement or not;

and the second compression submodule is used for performing special compression processing on the compressed code stream when the general compressed code stream does not meet the preset compression rate requirement.

In a preferred example of the embodiment of the present application, the block to be compressed may include: a luminance block to be compressed and a chrominance block to be compressed; the dicing module may further include:

and the sampling and splitting submodule is used for splitting the video frame data to be processed into a plurality of independent blocks to be compressed of brightness and blocks to be compressed of chroma according to the sampling mode of the video frame data.

Referring to fig. 14, a block diagram of an embodiment of a video data output apparatus according to the present application is shown, the video data output apparatus is applied to a video post-processing system 80, and the video post-processing system 80 includes: the device comprises a processing core 8001, a compressed code stream cache 8002 and a video output module 8003;

The video data output device may specifically include:

a request receiving module 9001, configured to receive a pixel block acquisition request, where the pixel block acquisition request includes: the processing core submits an acquisition request for acquiring a first-class pixel block, or the video output module submits an acquisition request for acquiring the first-class pixel block and a second-class pixel block;

a compressed code stream request module 9002, configured to, after receiving the pixel block acquisition request, to the compressed code stream cache, a compressed code stream corresponding to the type of the requested pixel block in the pixel block acquisition request;

a decoding module 9003, configured to perform decoding processing on the compressed code stream to generate a decompressed pixel block;

an output module 9004, configured to output the decompressed pixel block to the processing core or the video output module that submitted the pixel block obtaining request.

Referring to fig. 15, a block diagram of an embodiment of a video data system of the present application is shown. Wherein, the video data system is applied to a video post-processing system, and the video post-processing system 100 includes: a block to be compressed buffer 10001, a compressed code stream buffer 10002, a processing core 10003, and a video output module 6004, where the system may include: a video data input device 110 and a video data output device 120;

The video data input device 110 may include:

a receiving module 11001, configured to receive video frame data to be processed from at least two paths;

a segmentation module 11002, configured to segment the to-be-processed video frame data into a plurality of to-be-compressed blocks;

a block to be compressed input module 11003, configured to input a block to be compressed of one of the two ways into the block to be compressed for caching according to the number of blocks to be compressed of the two ways;

the compression module 11004 is configured to sequentially compress the blocks to be compressed in the block-to-be-compressed cache to generate a compressed code stream;

a compressed code stream input module 11005, configured to input the compressed code stream into the compressed code stream cache;

the video data output device 120 may include:

a request receiving module, configured to receive a pixel block acquisition request 12001, where the pixel block acquisition request includes: the processing core submits an acquisition request for acquiring a first-class pixel block, or the video output module submits an acquisition request for acquiring the first-class pixel block and a second-class pixel block;

a compressed code stream request module 12002, configured to, after receiving the pixel block acquisition request, to the compressed code stream cache, a compressed code stream corresponding to the type of the requested pixel block in the pixel block acquisition request;

a decoding module 12003, configured to perform decoding processing on the compressed code stream to generate a decompressed pixel block;

an output module 12004, configured to output the decompressed pixel block to the processing core or the video output module that submits the pixel block obtaining request.

For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

As will be appreciated by one of skill in the art, embodiments of the present application may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present application have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the true scope of the embodiments of the application.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.

The video data input method, the video data output method, the video data input device, the video data output device and the video data system provided by the present application are described in detail above, specific examples are applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiments is only used to help understanding the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A video data input method, applied to a video post-processing system, the video post-processing system comprising: caching a block to be compressed and caching a compressed code stream; the method comprises the following steps:

receiving video frame data to be processed from at least two paths;

inputting the compressed code stream into the compressed code stream cache;

wherein, the step of inputting the block to be compressed of one of the two paths into the block cache to be compressed according to the number of the blocks to be compressed of the two paths comprises:

judging whether the current output process is finished or not;

2. The method according to claim 1, wherein the step of inputting the block to be compressed of one of the ways into the block buffer to be compressed according to the number of the blocks to be compressed of the two ways further comprises:

3. The method according to claim 2, wherein the step of determining the current output process according to the number of the blocks to be compressed that have been input into the block buffer to be compressed in the output process of the two ways comprises:

4. A video data input device, applied to a video post-processing system, the video post-processing system comprising: caching a block to be compressed and caching a compressed code stream; the device comprises:

wherein the block to be compressed input module further comprises:

5. The apparatus of claim 4, wherein the block to be compressed input module further comprises:

6. The apparatus of claim 5, wherein the current output process determination sub-module further comprises:

7. A video data system for use in a video post-processing system, the video post-processing system comprising: the system comprises a block to be compressed cache, a compressed code stream cache, a processing kernel and a video output module, wherein the system comprises: a video data input device and a video data output device;

the video data input device comprises:

the video data output device comprises: