CN108804508A - Method and system for storing input image - Google Patents

Abstract

The present invention provides a method and system for storing an input image. The present invention describes allocating one or more frame buffers in a memory. The present invention also describes segmenting the input image into multiple access units corresponding to multiple subsets of the input image, and allocating a main part and a sub-part to each of the multiple access units in the frame buffer, wherein at least one sub-part is non-sequentially located after its respective main part in the frame buffer. The present invention also describes compressing the access units into compressed access units, storing each compressed access unit in a respective main part, and if the size of the compressed access unit exceeds the size of the main part, storing the remaining part of the compressed access unit in a respective sub-part. The present invention enables the compressed access units stored in the memory to be efficiently accessed.

Description

A method and system for storing input images

priority statement

This application claims priority to U.S. Provisional Patent Application No. 62/489,588, filed April 25, 2017, entitled "Memory AccessEfficiency Optimization for Frame Buffer Compression," and filed October 18, 2017 US Provisional Patent Application No. 15/786,908 for "Distributed Access Unit for FrameBuffer Compression," which is hereby incorporated by reference in its entirety.

technical field

The disclosed embodiments of the invention relate to storage techniques, and more particularly, to a method and system of storing input images.

Background technique

The background description provided herein is for the purpose of generally presenting the context of the disclosure. The content of the work of the presently named inventors, including both the content of the work described in this Background section, and aspects that were not considered to be prior art at the time of filing, are neither expressly nor impliedly considered to be part of the present invention. Invention prior art.

Electronic devices, such as computer systems, may include one or more memories. In one example, an electronic device includes a component, such as a central processing unit (CPU), on a separate integrated circuit chip than the memory, which accesses the memory through a memory controller. Memory accessed by the CPU creates heavy data traffic between the CPU and the memory.

Contents of the invention

In view of this, the present invention provides a method and system for storing an input image, so as to efficiently access the compressed access units stored in the memory.

Aspects of the invention provide a method of storing an input image in a memory. The method may include: allocating one or more frame buffers in memory; dividing the input image into a plurality of access units corresponding to a plurality of subsets of the input image, and assigning each of the plurality of access units in the frame buffer allocating a main part and a sub part for access units, wherein at least one sub part is not sequentially located after its respective main part in the frame buffer; compressing the plurality of access units into a plurality of compressed access units; and compressing each of the The compressed access units are stored into their respective main sections, and if the size of the compressed access units exceeds the size of the main section, the remainder of the compressed access units are stored into their respective secondary sections.

Aspects of the invention also provide a system for storing input images. The system includes memory, memory allocation device and memory controller. The memory has one or more frame buffers; the memory allocating means for receiving the input image, allocating the frame buffer in the memory to store the input image, dividing the input image into a plurality of accesses corresponding to the plurality of subsets of the input image unit, and allocates a main part and a sub part to each access unit in the frame buffer, wherein at least one sub part is located non-sequentially after its respective main part in the frame buffer; and a memory controller for responding to the memory allocation Instructions for means to store each compressed access unit into its respective main section, and if the size of the compressed access unit exceeds the size of the main section, store the remainder of the compressed access unit into its respective secondary section middle.

An optional aspect of the present invention may provide a non-transitory computer-readable medium storing computer-readable instructions. When the processing circuit executes the computer-readable instructions, the processing circuit performs a method comprising: allocating a or multiple frame buffers;

dividing an input image into a plurality of access units corresponding to a plurality of subsets of the input image, and assigning a main part and a sub part to each access unit in the plurality of access units in a frame buffer, wherein at least one sub part is in the frame non-sequentially located in the cache after its respective main part; compress multiple access units into multiple compressed access units; store each compressed access unit into its respective main part, and if the compressed access unit's size exceeds the size of the main part, the remainder of the compressed access unit is stored in the respective secondary part.

The present invention allocates a main part and a sub part in memory for each access unit, after the access unit is compressed, stores the compressed access unit into the main part, and also stores the compressed access unit in the main part when the size of the main part is smaller than the compressed access unit. Storing the remainder of the compressed access unit to the secondary section allows the compressed access unit to be efficiently accessed.

Description of drawings

Various embodiments of the invention, provided as examples, will be described in detail with reference to the following drawings, in which like symbols refer to like elements, and in which:

1 is an exemplary block diagram of a memory system according to an embodiment of the present invention;

Fig. 2 is an exemplary data structure according to an embodiment of the present invention;

FIG. 3 is three exemplary super blocks in three frame buffers according to an embodiment of the present invention;

FIG. 4 is three exemplary super blocks in three frame buffers according to an embodiment of the present invention;

FIG. 5 is two exemplary super blocks in two frame buffers according to an embodiment of the present invention;

FIG. 6 is an example of an optional frame buffer according to an embodiment of the present invention;

FIG. 7 is a flowchart describing an exemplary process according to an embodiment of the present invention.

Detailed ways

FIG. 1 shows an exemplary block diagram of a memory system 100 according to an embodiment of the present invention. As shown, the memory system 100 may include a memory allocation device 110 , a memory controller 120 and a memory 130 . The memory 130 may include a frame buffer 131 . The memory system 100 is used to divide the input image into one or more access units and store each compressed access unit (compressed access unit) into the main portion of the frame buffer 131 allocated for the respective access unit (main portion) and sub-part (secondary portion).

Memory system 100 may be any suitable system for storing data. In one embodiment, the memory system 100 is an electronic device, such as a desktop computer, a tablet computer, a smart phone, a wearable device, a smart TV, a video camera, a camcorder, a media player, and the like. In an example embodiment, memory system 100 may also include other components that access data stored in memory 130 . For example, other components may include a CPU 141 , a graphics processing unit (GPU) 142 , a multimedia engine 143 , a display circuit 144 , an image processor 145 , a video codec 146 and the like.

In one embodiment, memory 130 may have a sequence of memory blocks separated by memory boundaries based on page size or channel partitioning, for example, at intervals of 32 bytes, 64 bytes, 128 bytes, 256 bytes, 512 bytes 1K bytes, 2K bytes or 4K bytes. Accessing a certain amount of data stored in a memory block between two adjacent boundaries is more efficient than accessing the same amount of data stored separately in two memory blocks that straddle the memory boundary. Therefore, data in the memory 130 can be efficiently accessed by another component of the memory system 100 when the starting address of the data is aligned with a memory boundary. Memory boundaries are formed at addresses that are multiples of the memory block size. In one embodiment, the memory block size can hold a certain amount of data that can be transferred between the memory 130 and other components of the memory system 100 in a burst of read/write commands with a single or a few precharge commands and activate commands. Sequential forms are transmitted quickly. The memory block size may be selected based on characteristics of the memory 130 and other components of the memory system 100 that access the memory 130, such as the page size and channel partitioning of the memory 130, and the architecture of the memory 130 and other components of the memory system 100 that access the memory 130 and operating modes.

The memory allocator 110 is used to receive an input image and divide it into one or more access units. The memory allocating device 110 is also used for allocating part of the memory 130 to the input image, such as the frame buffer 131, and assigning two memory parts to each access unit in the frame buffer 131, namely a main part and a secondary part. In one example, the start address of the frame buffer 131 may be aligned with a memory boundary, eg, 0 bytes. The memory allocator 110 is used to compress each access unit and store each compressed access unit into its respective main part, and store the remainder of the compressed access unit if the size of the compressed access unit exceeds the size of the main part to their respective subsections. In one embodiment, the memory allocator 100 may be integrated in any component that accesses data stored in the memory 130, such as one or more components of the memory system 100, including the CPU 141, the GPU 142, the multimedia engine 143, the display circuit 144, image processor 145, video codec 146, and the like.

In one embodiment, the main portion may have a start address aligned with a memory boundary and a size that is one or more times the size of the memory block. Therefore, data stored in the main section can be efficiently accessed. Alternatively, when the size of the main parts is smaller than the size of the memory block, each main part can be located in a respective memory block, and the start address of one or more main parts can be aligned with one or more memory boundaries.

The size of the secondary portion may be a fraction of the size of the memory block. Thus, two or more subparts can be grouped together and stored independently of their respective main parts. When the size of the compressed access unit is smaller than or equal to the size of the main part, the compressed access unit can be completely stored inside the main part without using the sub part. In this way, accessing the compressed access unit can be done efficiently because the main part can be efficiently accessed.

In another embodiment, at least one of the subsections is not located consecutively after its respective main section in the frame buffer, which includes a reverse order in which the main section has a greater address than its respective subsection.

The memory controller 120 is used to manage memory access from the memory allocator 110 to the memory 130 . The memory controller 120 may be configured to receive requests from the memory allocator 110 to store the compressed access units into respective primary and secondary portions of the frame buffer 131 of the memory 130 . Based on these requests, memory controller 120 may send commands to memory 130 with instructions to store compressed access units into frame buffer 131 in respective primary and secondary portions. Memory controller 120 may also be used to schedule and cache these requests, among other things.

Memory 130 may be any suitable device for storing data. In one embodiment, the memory 130 includes a dynamic random access memory (dynamic random access memory, DRAM) type memory module, for example, double data rate synchronous DRAM (double data rate synchronous DRAM, DDR SDRAM), double data rate double synchronous DRAM ( double data rate two synchronous DRAM, DDR2 SDRAM), double data rate three synchronous DRAM (double data rate three synchronous DRAM, DDR3SDRAM), double data rate four synchronous DRAM (double data rate four synchronous DRAM, DDR4SDRAM), low power DDR SDRAM ( low powerDDR SDRAM, LPDDR SDRAM), etc.

In one embodiment, the memory system 100 may be a system-on-chip (SOC) in which all components are located on a single integrated circuit (IC) chip. In addition, other components such as CPU 141, GPU 142, multimedia engine 143, display circuit 144, image processor 145, and video codec 146 may also be contained on the same single IC chip. Alternatively, components in memory system 100 may be distributed across several ICs. For example, memory allocator 110, memory controller 120, memory 130, and other components of memory system 100 may be located on multiple IC chips. In addition, the memory allocator 110 can be integrated in any component that accesses the data stored in the memory 130, such as one or more components of the memory system 100, including a CPU 141, a GPU 142, a multimedia engine 143, a display circuit 144, An image processor 145, a video codec 146, and the like.

During operation, an input image may be received by memory allocation device 110 . The memory allocator 110 may divide the input image into one or more access units. In addition, the memory allocating means 110 may allocate a portion of the memory 130, such as the frame buffer 131, to the input image. Two memory sections, a main section and a sub section, are allocated to each access unit in the frame buffer 131 . Under the instruction of the memory allocator 110, the memory controller 120 may store the compressed access units into their respective main parts, and depending on the size, the secondary parts. The main portion may have a start address aligned with a memory boundary and a size that is one or more times the size of the memory block. The size of the secondary portion may be a fraction of the memory block size. Thus, two or more subparts can be grouped together and stored independently of their respective main parts. When the size of the compressed access unit is smaller than or equal to the size of the main part, the compressed access unit can be completely stored inside the main part without using the sub part. In this way, accessing the compressed access unit can be done efficiently.

FIG. 2 is an exemplary data structure 200 illustrating an input image 210 partitioned into access units, a frame buffer 231A, and a frame buffer 231B, according to an embodiment of the present invention. As shown, the input image 210 may be partitioned into an NxM array of access units. Inside the array, the size of an access unit depends on the number of pixels and the pixel bit-depth in this access unit. The pixel bit depth is the number of bits for a pixel of a specified color, for example, 10 bits or 12 bits, which correspond to 1024 colors or 4049 colors, respectively. In one example, the number of pixels in an access unit may depend on the compression method used by the memory allocator 110, for example, the size of the compression unit depends on which compression method is used to operate. For example, the size of the compression unit may be 4×4 pixels, 8×8 pixels, 16×4 pixels, 16×8 pixels, 16×16 pixels and so on. An access unit can have one or more compression units.

Frame buffer 231A shows an exemplary frame buffer structure for storing input images. A frame buffer may be a memory that has accessible locations for storing data. Accessible locations within memory may be grouped into memory blocks having a memory block size. As noted above, the memory block size may be selected based on characteristics of the memory 130 and other components of the memory system 100 accessing the memory 130, such as the page size and channel partitioning of the memory 130, and the characteristics of the memory 130 and the memory system 100 accessing the memory 130. Architecture and mode of operation of other components. In one example, the memory block size may be selected as 32 bytes, 64 bytes, 128 bytes, 256 bytes, 512 bytes, 1K bytes, 2K bytes, 4K bytes, etc. For example, memory 130 may be a DDR3 SDRAM device, and data is retrieved from memory 130 . A memory block size is the amount of data that can be retrieved from memory 130 in a single read cycle. Specifically, when the data bus width and the burst length (burst length) are 64 bits (ie, 8 bytes) and 4 respectively, the memory block size may be 32 bytes. In another example, the memory block size may be determined by the CPU 141 or GPU 142 cache line accessing the memory as 64 bytes or 128 bytes, etc. In the example of FIG. 2, the memory block size is 64 bytes, and thus in frame buffer 231A and frame buffer 231B, memory boundaries 250(1)-250(n) are located at 0 byte, 64 byte, 128 bytes, 192 bytes, etc.

A main part and a sub part can be assigned to each access unit. In one embodiment, the size of the main part may depend on the compressibility of the input image, compression method, memory block size, etc. Furthermore, in one embodiment, the sum of the size of the main part and the size of the sub part may be equal to the size of the access unit. Likewise, the ratio of the size of the main part to the size of the secondary part may depend on the compressibility of the input image, the compression method, the memory block size, and the like. For example, when an access unit can be compressed to a smaller size, a smaller main section is sufficient to store the compressed access unit, and the respective subsection can be left empty such that the ratio of the size of the main section to the size of the subsection smaller. For example, the ratio of the size of the main portion to the size of the secondary portion may be 2, 4 or 8, etc.

Additionally, the main portion may have a start address aligned with a memory boundary and a size that is a multiple of the memory block size so that data stored in the main portion can be efficiently accessed. In the example of FIG. 2, the size of main parts 221(1)-221(3) may be selected to have a memory block size of 64 bytes, and the starting addresses of main parts 221(1)-221(3) are respectively Aligns with memory demarcation lines 250(1)-250(3). The size of the respective subparts 241(1)-241(3) may be chosen to be less than 64 bytes, eg 32 bytes.

Frame buffer 231B shows an example when compressed access units 261-263 of various sizes are stored. Compressed access units 261-263 may be stored in their respective primary sections 221(1)-221(3) and secondary sections 241(1)-241(3). In the example of FIG. 2, the size of the compressed access unit 261 is smaller than the memory block size of 64 bytes. Thus, compressed access unit 261 may be completely stored within primary portion 221(1), while secondary portion 241(1) remains empty. The size of the compressed access unit 262 is equal to a memory block size of 64 bytes. Thus, compressed access unit 262 may be completely stored within primary portion 221(2), while secondary portion 241(2) remains empty. However, the size of the compressed access unit 263 is larger than the memory block size of 64 bytes. Thus, a first portion of compressed access unit 263 may fill primary portion 221(3), and a second or remaining portion of compressed access unit 263 may be stored in secondary portion 241(3).

In various embodiments, the size of the access unit, the size of the main part and the size of the sub part may be selected and kept constant for the input image. Alternatively, a plurality of input images, such as a sequence of frames of video, may be stored by the memory system 100 . The size of the access unit, the size of the main part and the size of the sub part can be selected for each individual input image, so it can vary dynamically from one input image to another.

The main part and the sub part may be arranged in the frame buffer 131 according to various layouts. Figures 3-5 show example layouts that include repeating patterns of main and sub-parts in a cyclical manner, where the smallest repeating unit is a superblock. Accordingly, the main section and the subsection in the frame buffer 131 may be arranged, for example, by sequentially placing superblocks adjacent to each other. In one embodiment, the size of a superblock may be a multiple of the memory block size.

FIG. 3 is a diagram of three exemplary super blocks 341A-341C in three frame buffers 331A-331C according to an embodiment of the present invention. Superblocks 341A-341C share the same property, where all subparts in a superblock are combined into a subpart group, which is located in the middle of the respective superblock. The start address of the main section is aligned with the memory boundary. The size of the sub-group is one or more times the size of the memory block, and the first sub-part of the sub-group is aligned with the boundary of the memory.

Referring to the super block 341A, the size of the access unit is set to 160 bytes, the size of the memory block is set to 128 bytes, and the sizes of the main part and the sub part are set to 128 bytes and 32 bytes, respectively. As shown, the superblock model has a subsection group consisting of four subsections (ie, S ₀ -S ₃ ), which are inserted between a first main section section (ie, M ₀ -M ₁ ) and a second main section section (ie between M ₂ -M ₃ ). The size of the superblock 341A is 5 times the size of the memory block (ie, 640 bytes). The memory boundary of the super block 341A is located at 0 byte, 128 byte, 256 byte, 384 byte, 512 byte and 640 byte, and the starting addresses of all main parts M ₀ -M ₃ are respectively located at word 0 section, 128 bytes, 384 bytes, and 512 bytes are aligned on memory boundaries. The sub-part group has a size of 128 bytes, with the first sub-part S ₀ aligned to a memory boundary at 256 bytes.

Referring to the super block 341B, the size of the access unit is set to 192 bytes, the size of the memory block is set to 128 bytes, and the sizes of the main part and the sub part are set to 128 bytes and 64 bytes, respectively. As shown, the superblock model has a subsection group consisting of four subsections (ie, S ₀ -S ₃ ), which are inserted between a first main section section (ie, M ₀ -M ₁ ) and a second main section section (ie between M ₂ -M ₃ ). The size of the superblock 341B is 6 times the size of the memory block (ie, 768 bytes). The memory boundary of the super block 341B is located at 0 byte, 128 byte, 256 byte, 384 byte, 512 byte, 640 byte and 768 byte, and the starting addresses of all main parts M ₀ -M ₃ are respectively Aligned to memory boundaries at 0 bytes, 128 bytes, 512 bytes, and 640 bytes. The sub-part group has a size of 256 bytes, and the first sub-part S ₀ is aligned with a memory boundary at 256 bytes.

Referring to the super block 341C, the size of the access unit is set to 384 bytes, the memory block size is set to 256 bytes, and the sizes of the main part and the sub part are set to 256 bytes and 128 bytes, respectively. As shown, the superblock model has a subsection group comprising two subsections (ie S ₀ -S ₁ ) inserted between a first main section M ₀ and a second main section M ₁ . The size of the superblock 341C is three times the size of the memory block (ie, 768 bytes). The memory boundary of super block 341C is positioned at 0 byte, 256 bytes, 512 bytes and 768 bytes, and the starting address of main part M ₀ -M ₁ is respectively at the memory boundary of 0 byte and at 512 word Section memory boundaries are aligned. The sub-part group has a size of 256 bytes, and the first sub-part S ₀ is aligned with a memory boundary at 256 bytes.

FIG. 4 illustrates three exemplary superblocks 441A-441C in three frame buffers 431A-431C according to an embodiment of the present invention. Superblocks 441A-441C share the same property, where all secondary parts in a superblock are combined into a secondary part group, which is followed by a main part group containing the main part. The start address of the main section is aligned with the memory boundary. The size of the sub-group is one or more times the size of the memory block, and the first sub-part of the sub-group is aligned with the boundary of the memory.

Referring to the super block 441A, the size of the access unit is set to 192 bytes, the size of the memory block is set to 128 bytes, and the sizes of the main part and the sub part are set to 128 bytes and 64 bytes, respectively. As shown, the superblock model 441A has a subsection group consisting of two subsections (ie, S ₀ -S ₁ ), followed by a main section group (ie, M ₀ -M ₁ ). The size of the superblock 441A is three times the size of the memory block (ie, 384 bytes). The memory boundaries of the super block 441A are located at 0 bytes, 128 bytes, 256 bytes and 384 bytes, and the starting addresses of all main parts M ₀ -M ₁ are respectively connected to the memory boundaries at 0 bytes and at 128 bytes. Bytes are aligned on memory boundaries. The sub-part group has a size of 128 bytes, with the first sub-part S ₀ aligned to a memory boundary at 256 bytes.

Referring to the super block 441B, the size of the access unit is set to 160 bytes, the memory block size is set to 128 bytes, and the sizes of the main part and the sub part are set to 128 bytes and 32 bytes, respectively. As shown, the superblock model has a subsection group consisting of four subsections (ie, S ₀ -S ₃ ), followed by a main section group (ie, M ₀ -M ₃ ). The size of the superblock 441B is 5 times the size of the memory block (ie, 640 bytes). The memory boundary of the super block 441B is located at 0 byte, 128 byte, 256 byte, 384 byte, 512 byte and 640 byte, and the starting addresses of all main parts M ₀ -M ₃ are respectively located at word 0 memory boundaries at 128 bytes, 256 bytes, and 384 bytes. The sub-part group has a size of 256 bytes, with the first sub-part S ₀ aligned to the memory boundary at 512 bytes.

Referring to the super block 441C, the size of the access unit is set to 320 bytes, the memory block size is set to 128 bytes, and the sizes of the main part and the sub part are set to 256 bytes and 64 bytes, respectively. As shown, the superblock model has a subsection group consisting of two subsections (ie, S ₀ -S ₁ ), followed by a main section group (ie, M ₀ -M ₁ ). The size of the superblock 441C is 5 times the size of the memory block (ie, 640 bytes). The memory boundary of the super block 441C is located at 0 byte, 128 byte, 256 byte, 512 byte and 640 byte, and the starting addresses of all main parts M ₀ -M ₁ are respectively located at 0 byte and 256 byte Alignment on memory boundaries at sections. The sub-part group has a size of 256 bytes, with the first sub-part S ₀ aligned to a memory boundary at 512 bytes.

FIG. 5 is a diagram of two exemplary superblocks 541A-541B in two frame buffers 531A-531B according to an embodiment of the present invention. Superblocks 541A-541C share the same characteristic, where the size of the main portion (ie, 128 bytes) is smaller than the memory block size (ie, 256 bytes). In addition, some compressed access units may need to be stored in both the main part and the secondary part, while some compressed access units may be completely stored in the main part. In order to allow efficient access to the compressed access units stored in the primary and secondary parts, as many secondary parts as possible corresponding to the primary part can be contained in the same memory block, and preferably immediately following the respective main part. For example, in superblock 541A, main portion M ₀ is followed by its secondary portion S ₀ , and main portion M ₃ is followed by its secondary portion S ₃ in their respective memory blocks.

Referring to the super block 541A, the size of the access unit is set to 192 bytes, the size of the memory block is set to 256 bytes, and the sizes of the main part and the sub part are set to 128 bytes and 64 bytes, respectively. As shown, the superblock model has a subsection S ₀ that follows a respective main section M ₀ , and a subsection S ₃ that follows a respective main section M ₃ . The size of the superblock 541A is three times the size of the memory block (ie, 768 bytes). The memory boundary of the super block 541A is located at 0 byte, 256 bytes, 512 bytes and 768 bytes, and the starting addresses of the three main parts M ₀ , M ₁ and M ₃ are respectively located at 0 byte, 256 bytes Sections and memory boundaries at 512 bytes are aligned. _The main portion _M2 is not aligned with memory boundaries, but is located inside a single block of memory between 256 bytes and 512 bytes.

Referring to the super block 541B, the size of the access unit is set to 192 bytes, the memory block size is set to 256 bytes, and the sizes of the main part and the sub part are set to 128 bytes and 64 bytes, respectively. As shown, the superblock model has _a subsection _S0 following its main section _M0 , and _a subsection S1 following its main section M1. The size of the superblock 541B is three times the size of the memory block (ie, 768 bytes). The memory boundary of the super block 541A is located at 0 byte, 256 bytes, 512 bytes and 768 bytes, and the starting addresses of the three main parts M ₀ , M ₁ and M ₂ are respectively located at 0 byte, 256 bytes Sections and memory boundaries at 512 bytes are aligned. _The main portion M3 is not aligned with memory boundaries, but is located within a single block _of memory between 512 bytes and 768 bytes.

Figure 6 shows an example of an optional frame buffer according to an embodiment of the present invention. The main and sub-sections of frame buffer 631A and frame buffer 631B can be arranged by having two sets, a main sub-set comprising all of the main sub-sections placed sequentially adjacent to each other, and sub-sections A section group consists of all subsections placed next to each other in sequence. In one embodiment, the size of the main part is one or more times the size of the memory block, and the start address of the main part may be aligned with the boundary of the memory. The main subgroup may be placed adjacent to the subgroup, or may be separated from the subgroup.

As shown in the frame buffer 631A, the size of the access unit is set to 80 bytes, the size of the memory block is set to 64 bytes, and the sizes of the main part and the sub part are set to 64 bytes and 16 bytes, respectively. The main section group includes all main sections. As shown, the main sections are placed adjacent to each other and have start addresses that correspond to consecutive memory boundaries at 0 bytes, 64 bytes, 128 bytes, 192 bytes, 256 bytes, etc. align. The subsection group includes all subsections. The first secondary portion S ₀ may have a start address aligned with a memory boundary, for example aligned with a memory boundary at 512 bytes.

As shown in the frame buffer 631B, the size of the access unit is set to 160 bytes, the size of the memory block is set to 128 bytes, and the sizes of the main part and the sub part are set to 128 bytes and 32 bytes, respectively. The main section group includes all main sections. As shown, the main sections are placed adjacent to each other with start addresses that correspond to consecutive memory boundaries at 0 bytes, 128 bytes, 256 bytes, 384 bytes, 512 bytes, etc. align. The subsection group includes all subsections. The first secondary portion S ₀ may have a start address aligned with a memory boundary, for example aligned with a memory boundary at 4096 bytes.

In the superblocks and framebuffers as shown in FIGS. 3-6, at least one secondary section is not sequentially located after its respective main section.

In one embodiment, the start addresses of the super block and the frame buffer may be aligned with the memory boundary of the memory 130 , for example, at byte 0 as shown in FIGS. 3-6 .

Although exemplary superblocks and frame buffers are shown in FIGS. 3-6, it should be understood that variations such as variations of the superblock model, locations of superblocks in frame memory, etc., are possible in order to accommodate different memory usage scenarios. of.

During operation, when the primary and secondary portions are placed in the frame buffer 131 according to a layout, such as the layouts shown in FIGS. 3-6 , compressed access units may be stored in the respective primary portions. For example, compressed access units may be stored in respective main sections and, if desired, in respective secondary sections. When the size of the compressed access unit is equal to or smaller than the size of the respective main part, the compressed access unit may be completely stored in the respective main part, and the corresponding sub part may remain empty.

FIG. 7 shows a flowchart describing an exemplary process 700 according to an embodiment of the invention. In one example, the process 700 is performed by the memory system 100 in FIG. 1 . The process starts at step S701 and continues to step S710.

In step S710, the input image is divided into one or more access units, for example, an N×M array of access units as shown in FIG. 2 . In one example, the memory allocator 110 is used to partition the input image into access units of the array. The input image may be a video frame, a photographic image, a graphic image, an animated image, or the like. For example, a video frame may be a reference frame used by the video codec 146 . Then the flow continues to step S720.

In step S720, a frame buffer is allocated in the memory. In one example, the memory allocator 110 is used to allocate the frame buffer 131 in the memory 130 . The size of the frame buffer is equal to or larger than the size of the input image. In one embodiment, the start address of the frame buffer may be aligned with a memory boundary, for example, the memory boundary at byte 0.

In step S730, each access unit is allocated two memory sections in the frame buffer, namely a main section and a secondary section. In one example, the memory allocator is used to allocate a main part and a secondary part to each access unit in the frame buffer 131 . In one embodiment, the sum of the size of the primary part and the size of the secondary part may be equal to the size of the access unit, eg, may be the size of an uncompressed access unit. In one embodiment, the size of the main part may depend on the compressibility of the input image, compression method, memory block size, etc. Furthermore, in one embodiment, the ratio of the size of the main part to the size of the sub part may depend on the compressibility of the input image, the compression method, and the like. For example, when an access unit can be compressed to a smaller size, a smaller main section is sufficient to store the compressed access unit, and the respective subsections can be left empty, so that the ratio of the size of the main section to the size of the subsection is better. Small.

Further, in one embodiment, the main portion may have a start address aligned with a memory boundary and a size that is one or more times the size of a memory block, so that data stored in the main portion can be efficiently accessed.

Alternatively, when the size of the main sections is smaller than the memory block size, each main section may be located within a respective memory block, and one or more main sections may have a start address aligned with one or more memory boundaries.

In one embodiment, the size of the secondary portion may be a fraction of the size of the memory block. Thus, two or more sub-parts may be grouped together as one or more sub-part groups and stored separately from their respective main parts. Further, in one embodiment, the first sub-part in each respective sub-part group may have a start address aligned with a memory boundary.

In another embodiment, at least one secondary section is not sequentially located after its respective main section in the frame buffer.

The main part and the secondary part can be arranged in the frame buffer, such as the frame buffer 131, according to different layouts. In one embodiment, the layout may include a repeating model of a superblock, where a superblock is the smallest repeating unit in a frame buffer. Thus, the primary and secondary sections in the frame buffer can be arranged, for example, by sequentially placing superblocks next to each other. The size of a superblock can be set as a multiple of the memory block size.

In one embodiment, the start address of the main portion in the superblock is aligned with a memory boundary. Subparts in a superblock may be grouped into one or more subpart groups whose size is a multiple of the memory block size. A first sub-part in each sub-part group may be aligned with a memory boundary. Some exemplary superblocks with the above characteristics are shown in FIGS. 3 and 4 .

In another embodiment, a superblock may have one or more main parts whose size is smaller than the memory block size. Some exemplary superblocks are shown in FIG. 5 . For example, as many secondary parts as possible immediately follow the respective main part (eg, S ₀ follows M ₀ and S ₃ follows M ₃ in superblock 541A of FIG. 5 ). As another example, each main portion is located entirely in the same memory block.

In one embodiment, the layout does not include a repeating model of the superblock. Conversely, by having main and sub-section groups, the main and sub-sections in the frame buffer can be arranged, such as the example shown in FIG. 6 . For example, a main part group includes a main part with a start address aligned with a contiguous memory boundary. A subpart group includes subparts adjacent to each other. A first sub-part of the sub-part group may be aligned with a memory boundary.

In step S740, the access unit may be compressed into a compressed access unit, so as to reduce bandwidth requirements for data transmission between the memory and another device accessing the memory. For example, in memory system 100 , memory 130 may be located on a different chip than memory allocator 110 , and memory allocator 110 is used to compress access units to reduce bandwidth requirements for data transfer between memory 130 and memory allocator 110 . Both lossless and lossy compression methods can be used to compress the access unit. Lossless compression methods preserve the quality of the original data, while lossy compression methods achieve more compression. The compression method may be a general compression method, an image compression method, or a video compression method. For example, compression methods may include run-length encoding, dictionary-based algorithms, Hoffman encoding, deflation, chroma subsampling, discrete cosine transform, and the like.

In step S750, the size of the compressed access unit is compared with the size of the main part. In one example, the memory allocator 110 is configured to compare the size of the compressed access unit with the size of the main part. If the size of the compressed access unit is larger than the size of the main part, the flow continues to step S770. Otherwise, the process continues to step S760.

In step S760, the compressed access units can be completely stored in the respective main parts because the size of the compressed access units is smaller than or equal to the size of the main part. In one example, memory controller 120 is configured to store compressed access units to their respective main portions in response to instructions from memory allocator 110 .

When the size of the compressed access unit is larger than the size of the main part, the flow proceeds to step S770. In step S770, the first part of the compressed access unit may be stored to the respective main part. The first part of the compressed access unit may be the same size as the main part and fills the respective main part. In one example, the memory controller 120 is configured to store the first portion of the compressed access unit to its respective main portion in response to an instruction of the memory allocator 110 .

In step S780, the second or remaining part of the compressed access unit may be stored to the respective secondary part. Therefore, when the size of the compressed access unit is larger than the size of the main part, the compressed access unit can be stored separately into the main part and the sub part. In one example, the memory controller 120 is configured to store the remaining portions of the compressed access units to respective secondary portions in response to instructions from the memory allocator 110 .

Before the flow continues to step S799 where the flow ends, steps S740-S780 may be repeatedly executed for all access units. In one example, the memory allocation device 110 and the memory controller 120 are used to repeatedly execute step S740-step S780 for all access units in the input image.

In various embodiments, the size of the access unit, the size of the main part and the size of the sub part may be selected and kept constant for the input image. Alternatively, a plurality of input images, such as a sequence of frames of a video, may be stored by the memory. The size of the access unit, the size of the main part and the size of the sub part can be selected for each individual input image, so it can vary dynamically from one input image to another.

In various examples, the memory allocating device 110 or the functions of the memory allocating device 110 may be implemented by hardware, software or a combination thereof. In one example, the memory allocating device 110 is implemented in hardware, such as a processing circuit, and the hardware may include one or more of discrete components, integrated circuits, application-specific integrated circuits (ASICs), and the like. In another example, the functions of the memory distribution may be implemented by software or firmware including instructions stored in a computer-readable non-transitory storage medium, which when executed by the processing circuit, cause the processing circuit to perform the respective functions.

Since various aspects of the invention have been described in conjunction with specific embodiments of the invention presented as examples, substitutions, modifications and variations of these examples may be made. Accordingly, the embodiments described herein are intended to be illustrative and not limiting. Changes may be made without departing from the scope of the claims.

Claims (20)

1. A method for storing an input image, characterized in that, the input image is stored in a memory, comprising: allocating one or more frame buffers in said memory; partitioning the input image into a plurality of access units corresponding to a plurality of subsets of the input image, and assigning a main part and a sub part to each access unit of the plurality of access units in the frame buffer, wherein at least one of said secondary sections is non-sequentially located after its respective main section in said frame buffer; compressing the plurality of access units into a plurality of compressed access units; and storing each compressed access unit into a respective main part, and if the size of the compressed access unit exceeds the size of the main part, storing the remainder of the compressed access unit into a respective secondary section. 2. A method of storing an input image as claimed in claim 1, wherein the memory has a series of memory boundaries separated by memory boundaries at addresses that are multiples of the memory block size A block of memory, and the size of the block of memory is determined based on characteristics of the memory and characteristics of a plurality of devices accessing the memory. 3. The method for storing an input image as claimed in claim 2, wherein the size of each main part is one or more times the size of the memory block, and the start address of each main part is respectively the same as Alignment on memory boundaries. 4. A method of storing an input image as claimed in claim 2, wherein the size of each main part is a fraction of the size of the memory block and each main part is located inside a respective memory block. 5. A method of storing an input image as claimed in claim 2, wherein the size of each sub-part is a fraction of the size of the memory block, and a plurality of sub-parts are combined into one or more sub-part groups . 6. The method for storing an input image as claimed in claim 5, wherein the size of the one or more sub-part groups is one or more times the size of the memory block, and each of the sub-parts The start address of the first subsection of the bank is aligned to a memory boundary. 7. The method of storing an input image as claimed in claim 2, wherein, a plurality of main parts and sub parts are arranged in a preset pattern to form a super block whose size is one or more times the size of the memory block; and the plurality of main parts and the A plurality of subparts are arranged by sequentially placing a plurality of super blocks adjacent to each other. 8. The method for storing an input image as claimed in claim 2, wherein the memory block size is selected as 32 bytes, 64 bytes, 128 bytes, 256 bytes, 512 bytes, 1K words section, 2K bytes, or 4K bytes. 9. The method for storing an input image as claimed in claim 1, wherein the input image is a still image or a video frame. 10. A system for storing input images, comprising: a memory having one or more frame buffers; memory allocating means for receiving said input image, allocating a frame buffer in said memory to store said input image, dividing said input image into a plurality of corresponding to the plurality of subsets of said input image access units, and assigning each access unit a main part and a sub part in the frame buffer, wherein at least one of the sub parts is not sequentially located after its respective main part in the frame buffer; and a memory controller for storing each compressed access unit into a respective main part in response to a plurality of instructions of the memory allocating means, and if the size of the compressed access unit exceeds the size of the main part , then store the remainder of the compressed access unit into the respective sub-parts. 11. The system for storing input images as claimed in claim 10, wherein said memory has a series of memory blocks divided by a plurality of memory blocks located at addresses that are multiples of the memory block size. boundaries, and the memory block size is determined based on characteristics of the memory and characteristics of devices accessing the memory. 12. The system for storing input images as claimed in claim 11, wherein said memory allocating means is used to select the size of each main part as one or more times the size of said memory block, and divide each The start address of the main section is aligned with the memory boundary. 13. A system for storing input images as claimed in claim 11 , wherein said memory allocating means is adapted to select the size of each main part as a fraction of said memory block size and to place each main part in inside the respective memory block. 14. A system for storing input images as claimed in claim 11 , wherein said memory allocating means is adapted to select the size of each subpart as a fraction of said memory block size and to combine a plurality of subparts into One or more subgroups. 15. The system for storing input images as claimed in claim 14, wherein said memory allocating means is used to select one or more subgroups whose size is one or more times the size of said memory block, and The start address of the first sub-part of each of the sub-part groups is aligned with a memory boundary. 16. The system for storing input images as claimed in claim 11, wherein said memory allocating means is configured to arrange a plurality of main parts and sub parts in a preset pattern to form a size that is twice the size of said memory block Or multiply super blocks, and place multiple super blocks adjacent to each other in the frame buffer. 17. The system for storing input images as claimed in claim 11, wherein said memory allocation device is used to determine that said memory block size is 32 bytes, 64 bytes, 128 bytes, 256 bytes, 512 words section, 1K byte, 2K byte, or 4K byte. 18. The system for storing input images of claim 10, wherein said memory is located on a different integrated circuit chip than said memory allocating means. 19. The system for storing input images as claimed in claim 10, wherein said memory allocation means is integrated in a video codec. 20. A non-transitory computer-readable medium, wherein computer-readable instructions are stored, and when the processing circuit executes the computer-readable instructions, the processing circuit executes a method, the method comprising: allocating one or more frame buffers in said memory; partitioning the input image into a plurality of access units corresponding to a plurality of subsets of the input image, and assigning a main part and a sub part to each access unit of the plurality of access units in the frame buffer, wherein at least one of said secondary sections is non-sequentially located after its respective main section in said frame buffer; compressing the plurality of access units into a plurality of compressed access units; storing each compressed access unit into a respective main section, and if the size of the compressed access unit exceeds the size of the main section, storing the remainder of the compressed access unit into a respective secondary section middle.