CN114731412A - Data processing method and data processing equipment - Google Patents
Data processing method and data processing equipment Download PDFInfo
- Publication number
- CN114731412A CN114731412A CN202080077975.8A CN202080077975A CN114731412A CN 114731412 A CN114731412 A CN 114731412A CN 202080077975 A CN202080077975 A CN 202080077975A CN 114731412 A CN114731412 A CN 114731412A
- Authority
- CN
- China
- Prior art keywords
- data
- image
- image data
- reference frame
- cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 18
- 239000013598 vector Substances 0.000 claims abstract description 76
- 238000013500 data storage Methods 0.000 claims abstract description 75
- 238000000034 method Methods 0.000 claims description 34
- 230000015654 memory Effects 0.000 claims description 14
- 230000003139 buffering effect Effects 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 9
- 238000004590 computer program Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000003068 static effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
A data processing method and a data processing device are provided, wherein the data processing method comprises the following steps: according to the global motion vector between the current frame and the reference frame, acquiring first image data of the reference frame from a data storage device for caching; and searching reference image data corresponding to the image block from the cache according to the motion vector of the image block to be encoded or decoded in the current frame, and encoding or decoding the image block according to the reference image data. In this way, the efficiency of encoding or decoding can be effectively improved.
Description
The present application relates to the field of computer technologies, and in particular, to a data processing method and a data processing device.
With the continuous development of computer technology, digital multimedia technology and encoding and decoding technology, video and images are becoming more and more popular in various fields and in people's daily life. The encoding of the video or image by the encoder may include inter-frame encoding and intra-frame encoding, and when the encoder performs inter-frame encoding, it is often necessary to obtain reference data from a reference frame to predict a current frame. The reference data is cached in the Cache based on a Cache architecture, the Cache is a Cache memory, the reference data is cached in the Cache by the method, and when inter-frame encoding is performed, the encoder can acquire the reference data from the Cache to perform encoding.
At present, when a frame of image starts to be encoded, reference data is usually obtained according to a zero motion vector and is cached in Cache. When the video coding with static or small motion is carried out, the method can usually obtain the reference data required by inter-frame prediction, but when the video coding with larger motion is carried out, the reference data required by inter-frame prediction can not be obtained, so that the coding efficiency of the part of the image which just starts to be coded is lower. Moreover, the optimal reference data required by the encoder may not be in the Cache, and in this case, the encoder can only use the existing data in the Cache to perform inter-frame prediction, which also affects the encoding efficiency of inter-frame prediction of the hardware encoder.
Disclosure of Invention
The embodiment of the application discloses a data processing method and data processing equipment, which can effectively improve the coding or decoding efficiency.
A first aspect of an embodiment of the present application discloses a data processing method, where the method includes:
according to the global motion vector between the current frame and the reference frame, acquiring first image data of the reference frame from a data storage device for caching;
and searching reference image data corresponding to the image block from the cache according to the motion vector of the image block to be encoded or decoded in the current frame, and encoding or decoding the image block according to the reference image data.
A second aspect of an embodiment of the present application discloses a data processing apparatus, including: a processor and a memory, wherein:
the memory for storing a computer program comprising program instructions;
the processor is used for executing the following steps when calling the program instruction:
according to the global motion vector between the current frame and the reference frame, acquiring first image data of the reference frame from a data storage device for caching;
and searching reference image data corresponding to the image block from the cache according to the motion vector of the image block to be encoded or decoded in the current frame, and encoding or decoding the image block according to the reference image data.
A third aspect of the embodiments of the present application discloses a data processing apparatus, including:
the acquisition module is used for acquiring first image data of a reference frame from a data storage device for caching according to a global motion vector between a current frame and the reference frame;
and the searching module is used for searching reference image data corresponding to the image block from the cache according to the motion vector of the image block to be coded or decoded in the current frame, and coding or decoding the image block according to the reference image data.
A fourth aspect of the embodiments of the present application discloses a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the data processing method according to the first aspect.
According to the embodiment of the application, the first image data of the reference frame can be acquired from the data storage device for caching according to the global motion vector between the current frame and the reference frame, then the reference image data corresponding to the image block is searched from the cache according to the motion vector of the image block to be encoded or decoded in the current frame, and the image block is encoded or decoded according to the reference image data.
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without inventive labor.
FIG. 1a is a schematic diagram of interframe coding disclosed in the prior art;
FIG. 1b is a block diagram of an architecture of a data processing system according to an embodiment of the present disclosure;
FIG. 2 is a flow chart of a data processing method disclosed in an embodiment of the present application;
FIG. 3 is a diagram of inter-frame coding disclosed in an embodiment of the present application;
FIG. 4 is a schematic flow chart diagram of another data processing method disclosed in the embodiments of the present application;
FIG. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of a data processing device disclosed in an embodiment of the present application.
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the field of encoding or decoding, when inter-frame encoding or decoding is performed, reference image data needs to be acquired from a reference frame to predict a current frame. The method for caching the data of the reference frame required by inter-frame encoding or decoding based on the Cache architecture is one of methods for caching the data of the reference frame in the encoding or decoding field. Specifically, when encoding or decoding a frame of image is started, data of a reference frame is usually obtained according to a zero motion vector and cached in the Cache. The method can generally acquire the data of the reference frame required by the inter-frame prediction when the inter-frame coding or decoding of the static or small motion is carried out. However, when inter-frame coding or decoding with large motion occurs, data of a reference frame required by inter-frame prediction is often not obtained, so that coding or decoding efficiency of a part of an image which is just coded or decoded is low, and data of the reference frame required by inter-frame prediction can be gradually obtained only through updating of Cache. For example, as shown in fig. 1a, in the case of inter-frame encoding or decoding with large motion, the image marked by 11 in fig. 1a is a reference frame image, and the image marked by 12 is a current frame image. It can be seen that the current frame image is moving down as a whole relative to the reference frame image, and the blocks marked by 101 and 102 in fig. 1a are corresponding regions in the reference frame and the current frame, respectively. When a current frame is encoded or decoded, a frame marked by 103 is encoded or decoded first, if the Cache initially acquires data of a reference frame according to a zero motion vector and caches the data in the Cache, the frame marked by 104 is acquired when the encoding or decoding is performed, and it can be seen from the figure that the data of the reference frame required by the frame marked by 103, which is to be encoded or decoded by the current frame, is not in the data acquired by the Cache, that is, is not in the frame marked by 104.
In order to solve the above problem, an embodiment of the present application provides a data processing method, where a data processing device may obtain first image data of a reference frame from a data storage device according to a global motion vector between a current frame and the reference frame to perform caching, search reference image data corresponding to an image block from the caching according to a motion vector of the image block to be encoded or decoded in the current frame, and encode or decode the image block according to the reference image data. By implementing the method, the problem that the reference data required by encoding or decoding cannot be acquired due to large movement can be avoided, so that the encoding or decoding efficiency can be effectively improved.
The implementation details of the technical solution of the embodiment of the present application are set forth in detail below:
referring to fig. 1b, fig. 1b is a schematic diagram illustrating an architecture of a data processing system according to an embodiment of the present disclosure. The data processing system comprises a data storage device and a data processing device, wherein the data processing device comprises a data caching module and a data processing module.
The data storage device may be configured to store image data of the reference frame, so that when encoding or decoding, the data storage device may send, according to a received data acquisition request sent by the data processing device, the image data of the reference frame corresponding to the data acquisition request to the data processing device. Optionally, the data storage device may be a Double Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), which may be abbreviated as DDR.
The data caching module in the data processing device may cache part of the data in the image data of the reference frame, and the image data of the reference frame cached in the data caching module may be updated according to an irregular time, so that when encoding or decoding, the image data of the reference frame corresponding to the data acquisition request may be sent to the data processing module according to the data acquisition request sent by the data processing module. Optionally, the data caching module may be a Cache, and the Cache is a Cache memory. The data processing module may be a module for inter prediction in an encoder or a decoder. The data processing module in the data processing device can encode or decode an image or a video, and send a data acquisition request to the data caching module in the encoding or decoding process to acquire image data of a reference frame required for encoding or decoding.
Referring to fig. 2, fig. 2 is a schematic flowchart illustrating a data processing method according to an embodiment of the present disclosure. The data processing method described in this embodiment may include the following steps:
201. and according to the global motion vector between the current frame and the reference frame, acquiring the first image data of the reference frame from the data storage device for caching.
The global motion vector between the current frame and the reference frame may be determined according to the global motion vector utilized in the history encoding or decoding, for example, if the current frame is the 10 th frame, that is, if the 10 th frame image is currently being encoded, the global motion vector utilized by the image frame before the 10 th frame image may be obtained, and if the global motion vectors for obtaining the 5 th to 9 th frame images are D1, D2, D3, D4, and D5, respectively, the global motion vector between the current frame and the reference frame may be (D1+ D2+ D3+ D4+ D5)/5. The global motion vector may also be determined in other manners, which are not limited in this application.
In one implementation, when a frame of image starts to be encoded or decoded, the data processing apparatus may obtain image data of a reference frame from the data storage apparatus according to a global motion vector between a current frame and the reference frame, and buffer the image data of the reference frame to a data buffering module in the data processing apparatus. Specifically, the data processing module in the data processing apparatus may determine the second position information in the image data of the reference frame according to the global motion vector between the current frame and the reference frame and the first position information of the image block to be encoded or decoded in the current frame. Alternatively, the second location information may be the result of the first location information plus the global motion vector. The first position information may indicate a position of an image block to be encoded or decoded in the current frame, for example, the first position information may be coordinate information of the image block to be encoded or decoded, and may also be other manners for representing the position, which is not limited in this application. Then, after the data processing module determines the second location information, the second location information may be sent to the data caching module. After the data caching module receives the second position information, the first image data of the image area corresponding to the second position information is obtained from the image data of the reference frame stored in the data storage device and cached in the data caching module.
Optionally, the second position information may be determined by a data processing module in the data processing apparatus according to a global motion vector between the current frame and the reference frame and the first position information of the image block to be encoded or decoded in the current frame. The data processing module in the data processing device may send the global motion vector between the current frame and the reference frame and the first position information of the image block to be encoded or decoded in the current frame to the data caching module in the data processing device, and the data caching module may determine the second position information according to the global motion vector and the first position information after receiving the global motion vector and the first position information.
In one implementation, after the data caching module determines the second location information, the data caching module may obtain first image data centered on an image region corresponding to the second location information from the image data of the reference frame stored in the data storage device, and a size of the first image data may be a preset size. Specifically, the second location information may be carried in a data obtaining request sent by the data caching module to the data storage device, where the data obtaining request may also carry a preset size of data to be requested, and the size of the preset size may be M × N. Then, after the data storage device receives the data acquisition request, the data storage device may determine first image data of a preset size centered on the image area corresponding to the second position information in the stored image data of the reference frame, and send the first image data to the data cache module, and after the data cache module receives the first image data, the data cache module may cache the first image data, so that the data processing module in the subsequent data processing device may acquire the required reference image data in the data cache module.
For example, the image marked by 31 in fig. 3 is image data of a current frame, the position marked by 301 in fig. 3 is first position information of an image block to be encoded or decoded in the current frame (for example, the image block to be encoded or decoded in the first one in the current frame is not limited to the first one, but may also be the image block to be encoded or decoded most first in the first row, the second row, and so on), and the data processing module may determine second position information according to the first position information and the global motion vector, where the second position information may be, for example, the position marked by 302 in fig. 3. The data processing module may send the second position information and the preset size to the data caching module, and then the data caching module may carry the second position information and the preset size in a data acquisition request sent to the data storage device, and after the data storage device receives the data acquisition request, may send, to the data caching module, first image data of a preset size centered on an image area corresponding to the second position information in the image data of the reference frame, where the first image data may be an area marked by 303 in fig. 3.
It should be noted that, considering that the second position information may be at the edge position of the reference frame, the image data with the preset size centered on the image area corresponding to the second position information may exceed the image area of the reference frame, such as the area marked by 303 in fig. 3, i.e. exceed the image range of the reference frame. Then, in order to ensure that the first image data only includes the image data in the reference frame, when the data processing device obtains the first image data with the preset size centering on the image area corresponding to the second position information from the image data of the reference frame stored in the data storage device for buffering, it may further determine whether the image area with the preset size centering on the image area corresponding to the second position information exceeds the image area of the reference frame, and if the image area with the preset size centering on the image area corresponding to the second position information does not exceed the image area of the reference frame, the data processing device may obtain the first image data with the preset size centering on the image area corresponding to the second position information from the image data of the reference frame stored in the data storage device for buffering. If the image area with the preset size taking the image area corresponding to the second position information as the center exceeds the image area of the reference frame, the image data with the preset size taking the image area corresponding to the second position information as the center can be adjusted, so that the adjusted data only comprises the image data in the reference frame.
Alternatively, the data processing device may acquire an image region overlapping between an image region of a preset size and an image region of the reference frame from the image data of the reference frame stored in the data storage device, and may buffer the overlapping image region as the first image data. Specifically, the data caching module of the data processing device sends a data request message carrying the second location information and the preset size to the data storage device. The data storage device may determine an image area of a preset size centered on the image area corresponding to the second position information from among the stored image data of the reference frame after receiving the data acquisition request. Then, it can also be detected that the image area exceeds the image area of the reference frame, and an image area overlapping between the image area of the preset size and the image area of the reference frame, that is, the first image area, can be determined, and the overlapping image area can be the area marked by 304 in fig. 3. Then, the data storage device can send the overlapped image area to the data caching module, and the data caching module can cache after receiving the overlapped image area.
Optionally, the data processing apparatus may further adjust the position of the image area of the preset size according to the boundary of the image area of the reference frame, so that the boundary of the adjusted image area of the preset size coincides with the boundary of the image area of the reference frame, and the adjusted image area of the preset size may be the area marked by 305 in fig. 3. Then, after determining the adjusted image area of the preset size, the first image data corresponding to the adjusted image area of the preset size may be obtained from the image data of the reference frame stored in the data storage device for caching.
202. And searching reference image data corresponding to the image block from the cache according to the motion vector of the image block to be coded or decoded in the current frame, and coding or decoding the image block according to the reference image data.
The motion vector of the image block to be encoded or decoded in the current frame may be determined according to the motion vector of the adjacent encoded or decoded image block of the image block to be encoded or decoded, or may be determined by using other methods, which is not limited in the present application. For example, if the current image block to be encoded or decoded is the 5 th block, the motion vectors used by the image blocks before the 5 th block may be obtained, and assuming that the motion vectors of the 2 nd to 4 th block images obtained are d1, d2, and d3, respectively, the motion vector of the image block to be encoded or decoded may be (d1+ d2+ d 3)/3. For example, the preset condition may use the candidate motion vector with the minimum encoding residual or decoding residual as the motion vector of the image block to be encoded or decoded, or may use other preset conditions, which are not limited in this application. The encoded residual may be a difference between image data obtained by inter-frame prediction and actual image data when the image block is inter-frame encoded, and the decoded residual may be understood in the same way.
In one implementation, the data processing apparatus may determine, according to the position information of the image block to be encoded or decoded in the current frame and the motion vector, identification information of reference image data corresponding to the image block, and then determine whether the reference image data is included in the cache according to the identification information. Specifically, the data processing module in the data processing apparatus may determine the identification information of the reference image data corresponding to the image block according to the position information and the motion vector of the image block to be encoded or decoded in the current frame. Then, the data processing module may send a data obtaining request to the data caching module to request the reference image data from the data caching module for subsequent encoding or decoding, where the data obtaining request may carry identification information. Then, after the data caching module receives the data obtaining request of the data processing module, the reference image data corresponding to the image block may be searched from the image data of the cached reference frame, that is, the first image data, according to the identification information in the data obtaining request, and the reference image data is sent to the data processing module, so that after receiving the data processing module, the data processing module may encode or decode the image block according to the reference image data. The identification information may be one or more of position information, area index and size, and the position information may be coordinate information.
In the embodiment of the application, according to a global motion vector between a current frame and a reference frame, first image data of the reference frame is acquired from a data storage device for caching, reference image data corresponding to an image block is searched from the cache according to the motion vector of the image block to be encoded or decoded in the current frame, and the image block is encoded or decoded according to the reference image data. By implementing the method, the data of the reference frame is obtained and cached by utilizing the global motion vector between the current frame and the reference frame, so that the problem that the reference data required by encoding or decoding cannot be obtained due to large motion can be avoided, and the encoding or decoding efficiency can be effectively improved.
Referring to fig. 4, fig. 4 is a schematic flowchart illustrating another data processing method according to an embodiment of the present disclosure. The data processing method described in this embodiment may include the following steps:
401. and according to the global motion vector between the current frame and the reference frame, acquiring the first image data of the reference frame from the data storage device for caching.
402. And determining the identification information of the reference image data corresponding to the image block according to the position information of the image block to be coded or decoded in the current frame and the motion vector.
In one implementation, a data processing module in the data processing apparatus may determine, according to position information and a motion vector of an image block to be encoded or decoded in a current frame, identification information of reference image data corresponding to the image block. Wherein the identification information may include one or more of location information, area index, and size. For example, taking the identification information as the location information, the location information may be coordinate information of an image block to be encoded or decoded, and the data processing module may add the motion vector to the coordinate information of the image block to be encoded or decoded, so as to obtain the coordinate information of the reference image data corresponding to the image block. For another example, taking the identification information as the area index as an example, the data processing module may add the motion vector to the position information of the image block to be encoded or decoded, so as to obtain the position information of the reference image data corresponding to the image block, and the data processing module stores the corresponding relationship between the area and the area index. For another example, taking the identification information as the area index and the size as an example, after determining the area index of the reference image data corresponding to the image block, the data processing module may further determine the size of the reference image data corresponding to the image block according to the size.
For the motion vector, reference may be made to the above description, which is not repeated herein.
In one implementation, a data processing module in the data processing apparatus may send position information and a motion vector of an image block to be encoded or decoded in a current frame to a data caching module in the data processing apparatus, and then after the data caching module receives the position information and the motion vector of the image block to be encoded or decoded in the current frame, the data processing module may determine identification information of reference image data corresponding to the image block according to the position information and the motion vector.
403. And judging whether the reference image data is included in the cache or not according to the identification information.
In one implementation, the data processing apparatus may determine whether the reference image data is included in the cache according to the identification information. Specifically, a data processing module in the data processing apparatus may send a data obtaining request to the data caching module to request the data caching module to perform subsequent encoding or decoding processing on the reference image data, where the data obtaining request may carry identification information. Then, after the data caching module receives the data obtaining request of the data processing module, the data caching module may search for data corresponding to the identification information, that is, reference image data corresponding to the image block, from the cached image data of the reference frame according to the identification information in the data obtaining request, and determine whether the reference image data is included in the first image data.
404. And if so, acquiring reference image data from the cache, and encoding or decoding the image block according to the reference image data.
In one implementation, if the data processing apparatus determines that the reference image data is included in the cache according to the identification information, the reference image data may be obtained from the cache, and the image block may be encoded or decoded according to the reference image data. Specifically, if the data processing module in the data processing apparatus determines the image data of the cached reference frame according to the identification information, that is, the first image data includes the reference image data, the reference image data may be obtained from the first image data, and the reference image data is sent to the data processing module, so that the data processing module encodes or decodes the image block according to the reference image data.
405. If not, when the cache meets the preset updating condition, acquiring second image data of the reference frame from the data storage device according to the identification information for caching, acquiring reference image data from the cache, and encoding or decoding the image block according to the reference image data.
In one implementation, if the data processing apparatus determines that the reference image data is not included in the cache according to the identification information, the second image data of the reference frame may be obtained from the data storage apparatus according to the identification information for caching in a case that the cache satisfies a preset update condition, so as to obtain the reference image data from the second image data in the cache, and encode or decode the image block according to the reference image data. Specifically, if the data processing module in the data processing apparatus determines the image data of the cached reference frame according to the identification information, that is, the first image data does not include the reference image data, it is further required to determine whether the current data caching module presets an update condition, and if the preset update condition is satisfied, the second image data of the reference frame may be obtained from the data storage apparatus according to the identification information for caching, and the reference image data may be obtained from the second image data, and the image block may be encoded or decoded according to the reference image data. The specific implementation process of obtaining the second image data of the reference frame from the data storage device according to the identification information may refer to the specific implementation process of obtaining the first image data of the reference frame from the data storage device according to the second location information, which is not described herein again.
In one implementation, if the data processing apparatus determines that the cache does not include the reference image data according to the identification information and the cache does not satisfy the preset update condition, the current data processing module may obtain the existing image data in the data cache module and encode or decode the image block according to the image data. For example, for encoding, the data processing module may acquire image data existing in the data buffering module to perform inter prediction, or may not perform inter prediction, but this may greatly affect the encoding efficiency.
In one implementation, the preset updating condition may be determined according to the number of times that the image data in the cache is not accessed, or the time that the image data is not accessed, or the remaining cache space of the cache, or the like. Optionally, the preset update condition may be that the time or the number of times that the image data in the cache is not accessed exceeds a first threshold, the time that the image data is cached exceeds a second threshold, or the remaining cache space of the cache is less than or equal to a third threshold, or any combination of the foregoing, for example, the time that the image data is cached exceeds the second threshold, and the remaining cache space of the cache is less than or equal to the third threshold. The first threshold, the second threshold, and the third threshold may be preset.
For a specific implementation of step 401, reference may be made to the detailed description of step 201 in the foregoing embodiment, and details are not described here again.
In the embodiment of the application, according to a global motion vector between a current frame and a reference frame, first image data of the reference frame is acquired from a data storage device for caching, and identification information of the reference image data corresponding to an image block is determined according to position information of the image block to be encoded or decoded in the current frame and the motion vector. And judging whether the reference image data is included in the cache or not according to the identification information. And if so, acquiring reference image data from the cache, and encoding or decoding the image block according to the reference image data. If not, when the cache meets the preset updating condition, acquiring second image data of the reference frame from the data storage device according to the identification information for caching, acquiring reference image data from the cache, and encoding or decoding the image block according to the reference image data. By implementing the method, the problem that the reference data required by encoding or decoding cannot be acquired due to large movement can be avoided, so that the encoding or decoding efficiency can be effectively improved.
In one implementation, the image data of the reference frame buffered in the data buffering module in the data processing apparatus may be continuously updated, so that when the data processing module is to acquire the reference image data for encoding or decoding, the data processing module may quickly acquire the required reference image data from the data buffering module, thereby improving the encoding or decoding efficiency.
Optionally, the data caching module may determine whether the cached image data of the reference frame meets the preset updating condition when responding to the data obtaining request of the data processing module, and if the preset updating condition is met, obtain the third image data of the reference frame from the data storage device for caching according to the identification information carried in the data obtaining request, and obtain the reference image data from the cache. The specific implementation process of obtaining the third image data of the reference frame from the data storage device for caching according to the identification information carried in the data obtaining request may refer to the specific implementation process of obtaining the second image data, which is not described herein again.
Optionally, the data caching module may further determine whether the cache meets a preset updating condition, for example, the data caching module detects whether the cached image data of the reference frame meets the preset updating condition at a preset time interval. The preset time interval may be preset, and for example, may be 1 second or 3 seconds, which is not limited in this application. If the data caching module detects that the cached image data of the reference frame meets the preset updating condition, the fourth image data of the reference frame can be acquired from the data storage device for caching according to the identification information of the reference image data included in the latest received data acquisition request. The specific implementation process of obtaining the fourth image data of the reference frame from the data storage device for caching according to the identification information carried in the data obtaining request may refer to the specific implementation process of obtaining the second image data, which is not described herein again.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present disclosure. The data processing apparatus includes:
an obtaining module 501, configured to obtain first image data of a reference frame from a data storage device for caching according to a global motion vector between a current frame and the reference frame;
the searching module 502 is configured to search reference image data corresponding to the image block from the cache according to the motion vector of the image block to be encoded or decoded in the current frame, and encode or decode the image block according to the reference image data.
In an implementation manner, the obtaining module 501 is specifically configured to:
determining second position information in the image data of the reference frame according to a global motion vector between a current frame and a reference frame and the first position information in the current frame;
and acquiring first image data with a preset size by taking an image area corresponding to the second position information as a center from the image data of the reference frame stored in the data storage device for caching.
In an implementation manner, the obtaining module 501 is specifically configured to:
judging whether an image area with a preset size taking the image area corresponding to the second position information as a center exceeds the image area of the reference frame;
and if the image area of the reference frame is not exceeded, acquiring first image data with a preset size taking the image area corresponding to the second position information as the center from the image data of the reference frame stored in a data storage device for caching.
In an implementation manner, the obtaining module 501 is further configured to:
and if the image area of the reference frame exceeds the image area of the reference frame, acquiring an image area overlapped between the image area with the preset size and the image area of the reference frame from the image data of the reference frame stored in the data storage device, and caching the overlapped image area as first image data.
In an implementation manner, the obtaining module 501 is further configured to:
if the image area exceeds the image area of the reference frame, adjusting the position of the image area with the preset size according to the boundary of the image area of the reference frame, so that the boundary of the adjusted image area with the preset size is overlapped with the boundary of the image area of the reference frame;
and acquiring first image data corresponding to the adjusted image area with the preset size from the image data of the reference frame stored in the data storage device for caching.
In an implementation manner, the search module 502 is specifically configured to:
determining identification information of reference image data corresponding to the image block according to the position information and the motion vector of the image block to be coded or decoded in the current frame;
judging whether the reference image data is included in the cache or not according to the identification information;
if yes, acquiring the reference image data from the cache;
if not, when the cache meets the preset updating condition, acquiring the second image data of the reference frame from the data storage device according to the identification information for caching, and acquiring the reference image data from the cache.
In an implementation manner, the apparatus further includes an updating module 503, specifically configured to:
responding to a data acquisition request, and judging whether the cache meets a preset updating condition, wherein the data acquisition request comprises identification information of reference image data;
if so, acquiring the third image data of the reference frame from the data storage device for caching according to the identification information, and acquiring the reference image data from the cache.
In one implementation, the updating module 503 is further configured to:
judging whether the cache meets a preset updating condition or not;
and if so, acquiring fourth image data of the reference frame from the data storage device for caching according to the identification information of the reference image data included in the data acquisition request received last time.
In one implementation, the preset update condition includes one or more of a time or a number of times that the image data in the cache is not accessed exceeds a first threshold, a time that the image data is cached exceeds a second threshold, and a remaining cache space of the cache is less than or equal to a third threshold.
In one implementation, the identification information includes one or more of location information, area index, and size.
It can be understood that the functions of the functional modules of the data processing apparatus described in the embodiment of the present application may be specifically implemented according to the method in the embodiment of the method described in fig. 2 or fig. 4, and the specific implementation process may refer to the description related to the embodiment of the method in fig. 2 or fig. 4, which is not described herein again.
In this embodiment of the present application, the obtaining module 501 obtains first image data of a reference frame from a data storage device for caching according to a global motion vector between a current frame and the reference frame, and the searching module 502 searches reference image data corresponding to an image block from the cache according to a motion vector of the image block to be encoded or decoded in the current frame, and encodes or decodes the image block according to the reference image data, so that encoding or decoding efficiency can be effectively improved.
Referring to fig. 6, fig. 6 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present disclosure. The data processing apparatus described in this embodiment includes: a processor 601 and a memory 602. The processor 601 and the memory 602 are connected by a bus.
The Processor 601 may be a Central Processing Unit (CPU), and may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 602 may include both read-only memory and random access memory, and provides program instructions and data to the processor 601. A portion of the memory 602 may also include non-volatile random access memory. Wherein, the processor 601, when calling the program instruction, is configured to perform:
according to a global motion vector between a current frame and a reference frame, acquiring first image data of the reference frame from a data storage device for caching;
and searching reference image data corresponding to the image block from the cache according to the motion vector of the image block to be encoded or decoded in the current frame, and encoding or decoding the image block according to the reference image data.
In one implementation, the processor 601 is specifically configured to:
determining second position information in the image data of the reference frame according to a global motion vector between the current frame and the reference frame and the first position information in the current frame;
and acquiring first image data with a preset size by taking an image area corresponding to the second position information as a center from the image data of the reference frame stored in the data storage device for caching.
In one implementation, the processor 601 is specifically configured to:
judging whether an image area with a preset size taking the image area corresponding to the second position information as a center exceeds the image area of the reference frame;
and if the image area of the reference frame is not exceeded, acquiring first image data with a preset size by taking the image area corresponding to the second position information as a center from the image data of the reference frame stored in a data storage device for caching.
In one implementation, the processor 601 is further configured to:
and if the image area exceeds the image area of the reference frame, acquiring an image area overlapped between the image area with the preset size and the image area of the reference frame from the image data of the reference frame stored in the data storage device, and caching the overlapped image area as first image data.
In one implementation, the processor 601 is further configured to:
if the image area exceeds the image area of the reference frame, adjusting the position of the image area with the preset size according to the boundary of the image area of the reference frame, so that the boundary of the adjusted image area with the preset size is superposed with the boundary of the image area of the reference frame;
and acquiring first image data corresponding to the adjusted image area with the preset size from the image data of the reference frame stored in the data storage device for caching.
In one implementation, the processor 601 is specifically configured to:
determining identification information of reference image data corresponding to the image block according to the position information and the motion vector of the image block to be coded or decoded in the current frame;
judging whether the reference image data is included in the cache or not according to the identification information;
if yes, acquiring the reference image data from the cache;
if not, when the cache meets the preset updating condition, acquiring the second image data of the reference frame from the data storage device according to the identification information for caching, and acquiring the reference image data from the cache.
In one implementation, the processor 601 is further configured to:
responding to a data acquisition request, and judging whether the cache meets a preset updating condition, wherein the data acquisition request comprises identification information of reference image data;
if so, acquiring the third image data of the reference frame from the data storage device for caching according to the identification information, and acquiring the reference image data from the cache.
In one implementation, the processor 601 is further configured to:
judging whether the cache meets a preset updating condition or not;
and if so, acquiring fourth image data of the reference frame from the data storage device for caching according to the identification information of the reference image data included in the data acquisition request received last time.
In one implementation, the preset update condition includes one or more of a time or a number of times that the image data in the cache is not accessed exceeds a first threshold, a time that the image data is cached exceeds a second threshold, and a remaining cache space of the cache is less than or equal to a third threshold.
In one implementation, the identification information includes one or more of location information, a region index, and a size.
In a specific implementation, the processor 601 and the memory 602 described in this embodiment of the present application may execute the implementation described in the data processing method provided in fig. 2 or fig. 4 in this embodiment of the present application, and may also execute the implementation of the data processing apparatus described in fig. 5 in this embodiment of the present application, which is not described herein again.
In the embodiment of the present application, the processor 601 obtains the first image data of the reference frame from the data storage device for caching according to the global motion vector between the current frame and the reference frame, searches the reference image data corresponding to the image block from the cache according to the motion vector of the image block to be encoded or decoded in the current frame, and encodes or decodes the image block according to the reference image data, thereby effectively improving the encoding or decoding efficiency.
The embodiment of the present application also provides a computer storage medium, in which program instructions are stored, and when the program is executed, some or all of the steps of the data processing method in the embodiment corresponding to fig. 2 or fig. 4 may be included.
It should be noted that, for simplicity of description, the above-mentioned embodiments of the method are described as a series of acts or combinations, but those skilled in the art should understand that the present application is not limited by the order of acts described, as some steps may be performed in other orders or simultaneously according to the present application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the application.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
The foregoing detailed description is directed to a data processing method and a data processing apparatus provided in the embodiments of the present application, and specific examples are applied in the present application to explain the principles and implementations of the present application, and the descriptions of the foregoing embodiments are only used to help understand the methods and core ideas of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (20)
- A method of data processing, comprising:according to a global motion vector between a current frame and a reference frame, acquiring first image data of the reference frame from a data storage device for caching;and searching reference image data corresponding to the image block from the cache according to the motion vector of the image block to be encoded or decoded in the current frame, and encoding or decoding the image block according to the reference image data.
- The method according to claim 1, wherein the obtaining the first image data of the reference frame from a data storage device for buffering according to the global motion vector between the current frame and the reference frame comprises:determining second position information in the image data of the reference frame according to a global motion vector between the current frame and the reference frame and the first position information in the current frame;and acquiring first image data with a preset size by taking an image area corresponding to the second position information as a center from the image data of the reference frame stored in the data storage device for caching.
- The method according to claim 2, wherein the obtaining, from the image data of the reference frame stored in the data storage device, first image data of a preset size centered on the image area corresponding to the second position information for buffering comprises:judging whether an image area with a preset size taking the image area corresponding to the second position information as a center exceeds the image area of the reference frame;and if the image area of the reference frame is not exceeded, acquiring first image data with a preset size taking the image area corresponding to the second position information as the center from the image data of the reference frame stored in a data storage device for caching.
- The method of claim 3, further comprising:and if the image area exceeds the image area of the reference frame, acquiring an image area overlapped between the image area with the preset size and the image area of the reference frame from the image data of the reference frame stored in the data storage device, and caching the overlapped image area as first image data.
- The method of claim 3, further comprising:if the image area exceeds the image area of the reference frame, adjusting the position of the image area with the preset size according to the boundary of the image area of the reference frame, so that the boundary of the adjusted image area with the preset size is superposed with the boundary of the image area of the reference frame;and acquiring first image data corresponding to the adjusted image area with the preset size from the image data of the reference frame stored in the data storage device for caching.
- The method according to any one of claims 1 to 5, wherein the searching for the reference image data corresponding to the image block from the buffer according to the motion vector of the image block to be encoded or decoded in the current frame comprises:determining identification information of reference image data corresponding to the image block according to the position information and the motion vector of the image block to be coded or decoded in the current frame;judging whether the reference image data is included in the cache or not according to the identification information;if yes, acquiring the reference image data from the cache;if not, when the cache meets the preset updating condition, acquiring the second image data of the reference frame from the data storage device according to the identification information for caching, and acquiring the reference image data from the cache.
- The method according to any one of claims 1 to 5, further comprising:responding to a data acquisition request, and judging whether the cache meets a preset updating condition, wherein the data acquisition request comprises identification information of reference image data;if so, acquiring the third image data of the reference frame from the data storage device for caching according to the identification information, and acquiring the reference image data from the cache.
- The method according to any one of claims 1 to 5, further comprising:judging whether the cache meets a preset updating condition or not;and if so, acquiring fourth image data of the reference frame from the data storage device for caching according to the identification information of the reference image data included in the data acquisition request received last time.
- The method according to any one of claims 6 to 8, wherein the preset updating condition comprises one or more of that the time or the number of times that the image data in the cache is not accessed exceeds a first threshold, that the time that the image data is cached exceeds a second threshold, and that the remaining cache space of the cache is less than or equal to a third threshold.
- The method of any one of claims 6 to 8, wherein the identification information comprises one or more of location information, area index and size.
- A data processing apparatus, characterized by comprising: a processor and a memory, wherein:the memory to store program instructions;the processor is used for executing the following steps when calling the program instruction:according to the global motion vector between the current frame and the reference frame, acquiring first image data of the reference frame from a data storage device for caching;and searching reference image data corresponding to the image block from the cache according to the motion vector of the image block to be encoded or decoded in the current frame, and encoding or decoding the image block according to the reference image data.
- The data processing device of claim 11, wherein the processor is specifically configured to:determining second position information in the image data of the reference frame according to a global motion vector between the current frame and the reference frame and the first position information in the current frame;and acquiring first image data with a preset size by taking an image area corresponding to the second position information as a center from the image data of the reference frame stored in the data storage device for caching.
- The data processing device of claim 12, wherein the processor is specifically configured to:judging whether an image area with a preset size taking the image area corresponding to the second position information as a center exceeds the image area of the reference frame;and if the image area of the reference frame is not exceeded, acquiring first image data with a preset size taking the image area corresponding to the second position information as the center from the image data of the reference frame stored in a data storage device for caching.
- The data processing device of claim 13, wherein the processor is further configured to:and if the image area exceeds the image area of the reference frame, acquiring an image area overlapped between the image area with the preset size and the image area of the reference frame from the image data of the reference frame stored in the data storage device, and caching the overlapped image area as first image data.
- The data processing apparatus of claim 13, wherein the processor is further configured to:if the image area exceeds the image area of the reference frame, adjusting the position of the image area with the preset size according to the boundary of the image area of the reference frame, so that the boundary of the adjusted image area with the preset size is overlapped with the boundary of the image area of the reference frame;and acquiring first image data corresponding to the adjusted image area with the preset size from the image data of the reference frame stored in the data storage device for caching.
- The data processing device of any one of claims 11 to 15, wherein the processor is specifically configured to:determining identification information of reference image data corresponding to the image block according to the position information and the motion vector of the image block to be coded or decoded in the current frame;judging whether the reference image data is included in the cache or not according to the identification information;if yes, acquiring the reference image data from the cache;if not, when the cache meets the preset updating condition, acquiring the second image data of the reference frame from the data storage device according to the identification information for caching, and acquiring the reference image data from the cache.
- The data processing apparatus of any of claims 11 to 15, wherein the processor is further configured to:responding to a data acquisition request, and judging whether the cache meets a preset updating condition, wherein the data acquisition request comprises identification information of reference image data;if so, acquiring the third image data of the reference frame from the data storage device for caching according to the identification information, and acquiring the reference image data from the cache.
- The data processing apparatus of any of claims 11 to 15, wherein the processor is further configured to:judging whether the cache meets a preset updating condition or not;and if so, acquiring fourth image data of the reference frame from the data storage device for caching according to the identification information of the reference image data included in the data acquisition request received last time.
- The data processing device according to any one of claims 16 to 18, wherein the preset update condition includes one or more of a time or a number of times that the image data in the cache is not accessed exceeds a first threshold, a time that the image data is cached exceeds a second threshold, and a remaining cache space of the cache is less than or equal to a third threshold.
- The data processing apparatus of any of claims 16 to 18, wherein the identification information comprises one or more of location information, area index and size.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2020/133127 WO2022115999A1 (en) | 2020-12-01 | 2020-12-01 | Data processing method and data processing device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114731412A true CN114731412A (en) | 2022-07-08 |
Family
ID=81852854
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202080077975.8A Pending CN114731412A (en) | 2020-12-01 | 2020-12-01 | Data processing method and data processing equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114731412A (en) |
| WO (1) | WO2022115999A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116667884B (en) * | 2023-06-08 | 2024-06-28 | 广东视安通智慧显控股份有限公司 | Indoor comprehensive two-wire control system of non-polarized IP |
| CN117499663B (en) * | 2023-12-29 | 2024-03-15 | 摩尔线程智能科技(北京)有限责任公司 | A video decoding system and method, electronic device and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030043912A1 (en) * | 2001-08-23 | 2003-03-06 | Sharp Laboratories Of America, Inc. | Method and apparatus for motion vector coding with global motion parameters |
| US20030072373A1 (en) * | 2001-10-04 | 2003-04-17 | Sharp Laboratories Of America, Inc | Method and apparatus for global motion estimation |
| CN101163244A (en) * | 2007-11-22 | 2008-04-16 | 上海交通大学 | Picture element caching method and system in movement compensation process of video decoder |
| CN103428482A (en) * | 2012-05-15 | 2013-12-04 | 中兴通讯股份有限公司 | Processing method and device of image data |
| US20140010303A1 (en) * | 2012-07-03 | 2014-01-09 | Mstar Semiconductor, Inc. | Motion compensation image processing method and associated apparatus |
| US20170272773A1 (en) * | 2016-03-18 | 2017-09-21 | Google Inc. | Motion Vector Reference Selection Through Reference Frame Buffer Tracking |
| US20180139462A1 (en) * | 2015-05-26 | 2018-05-17 | Allwinner Technology Co., Ltd. | Method of Storing Decoded Video Data, Method of Computing Motion Vector Data and Computer Device |
| CN108702512A (en) * | 2017-10-31 | 2018-10-23 | 深圳市大疆创新科技有限公司 | Method for estimating and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8593469B2 (en) * | 2006-03-29 | 2013-11-26 | Nvidia Corporation | Method and circuit for efficient caching of reference video data |
-
2020
- 2020-12-01 WO PCT/CN2020/133127 patent/WO2022115999A1/en active Application Filing
- 2020-12-01 CN CN202080077975.8A patent/CN114731412A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030043912A1 (en) * | 2001-08-23 | 2003-03-06 | Sharp Laboratories Of America, Inc. | Method and apparatus for motion vector coding with global motion parameters |
| US20030072373A1 (en) * | 2001-10-04 | 2003-04-17 | Sharp Laboratories Of America, Inc | Method and apparatus for global motion estimation |
| CN101163244A (en) * | 2007-11-22 | 2008-04-16 | 上海交通大学 | Picture element caching method and system in movement compensation process of video decoder |
| CN103428482A (en) * | 2012-05-15 | 2013-12-04 | 中兴通讯股份有限公司 | Processing method and device of image data |
| US20140010303A1 (en) * | 2012-07-03 | 2014-01-09 | Mstar Semiconductor, Inc. | Motion compensation image processing method and associated apparatus |
| US20180139462A1 (en) * | 2015-05-26 | 2018-05-17 | Allwinner Technology Co., Ltd. | Method of Storing Decoded Video Data, Method of Computing Motion Vector Data and Computer Device |
| US20170272773A1 (en) * | 2016-03-18 | 2017-09-21 | Google Inc. | Motion Vector Reference Selection Through Reference Frame Buffer Tracking |
| CN108702512A (en) * | 2017-10-31 | 2018-10-23 | 深圳市大疆创新科技有限公司 | Method for estimating and device |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022115999A1 (en) | 2022-06-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8619862B2 (en) | Method and device for generating an image data stream, method and device for reconstructing a current image from an image data stream, image data stream and storage medium carrying an image data stream | |
| US10390040B2 (en) | Method, apparatus, and system for deep feature coding and decoding | |
| US10097826B2 (en) | Method and device for generating a predicted value of an image using interpolation and motion vectors | |
| US9959506B1 (en) | Predictive content retrieval using device movements | |
| CN114731412A (en) | Data processing method and data processing equipment | |
| CN101605256A (en) | Method and device for video encoding and decoding | |
| WO2020140915A1 (en) | Video processing method and apparatus | |
| WO2022160744A1 (en) | Gpu-based video synthesis system and method | |
| JP2024102325A5 (en) | ||
| JPWO2020180685A5 (en) | ||
| US7408989B2 (en) | Method of video encoding using windows and system thereof | |
| US20140010303A1 (en) | Motion compensation image processing method and associated apparatus | |
| US7881367B2 (en) | Method of video coding for handheld apparatus | |
| CN111405358A (en) | Cache-based video frame extraction method, apparatus, medium, and system | |
| WO2022116574A1 (en) | Motion vector prediction method and system based on quadratic polynomial and computer medium | |
| CN112702607B (en) | An intelligent video compression method and device based on optical flow decision-making | |
| CN109493304B (en) | Image data processing method and device | |
| CN114040209B (en) | Motion estimation method, device, electronic device and storage medium | |
| CN110800301A (en) | Control method and device of coding equipment and storage medium | |
| US20150055707A1 (en) | Method and Apparatus for Motion Compensation Reference Data Caching | |
| CN117896537A (en) | Ultra-high definition video coding acceleration algorithm based on SVT-AVS3 | |
| CN113535606B (en) | A data processing method and device | |
| JP2002305749A (en) | Moving detection method, video compression method, and moving prediction device | |
| CN113438476B (en) | Video encoding method, system and storage medium | |
| CN114501017A (en) | Video coding adaptation method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |