CN112734868B - Lossless real-time compression method for 2D OpenGL texture based on tile segmentation principle - Google Patents

Abstract

A lossless real-time compression method for 2D OpenGL textures based on the principle of tile segmentation. Based on the characteristics of large-area blank areas and large-area monochrome in artificially drawn images, the invention implements real-time compression of OpenGL textures in a tile-aligned manner. The compression process is shown in the attached figure. It achieves a significant reduction in memory usage, fast compression time, real-time decompression, and the compressed image is directly rendered and displayed without decompression. The decompression and display of the compressed image is exactly the same as the original image without quality loss, and the image that needs to be edited repeatedly can be decompressed and compressed multiple times to ensure the content of the original image. This compression technology is suitable for all kinds of devices based on OpenGL display technology, such as personal computers, workstations, mobile devices, and embedded devices. Supports application environments with different programming interfaces such as OpenGL, OpenGL ES, and WebGL. All calculations involved in the algorithm can be done in the GPU through multiple renderings, and there is no data transfer between the GPU and the CPU.

Description

Lossless real-time compression method for 2D OpenGL texture based on tile segmentation principle

technical field

The invention relates to the field of lossless real-time compression, and particularly designs a lossless real-time compression method for 2D OpenGL textures based on the principle of tile segmentation.

Background technique

When OpenGL displays complex surfaces, images are often used to express the details of instant noodles, or when the multi-layer support of planar drawing software is used, the uncompressed textures used will occupy a large amount of display memory. Take a 4096x4096 32-bit texture as an example, which occupies 64M of memory. Considering other overheads, a display device with 2G display memory supports up to 32 layers when using uncompressed textures, and the graphics card manufacturer supports some texture compression. Methods, such as ECT1, PVRTC, ATITC, and S3TC, currently have the following problems: 1) Due to the lack of unified specifications and poor compatibility, applications cannot be deployed on computers and mobile devices, or between mobile devices of different brands; 2) The real-time performance is poor, and it cannot support real-time conversion between the original image and the compressed texture. There are insurmountable barriers to the use of the drawing system that requires flexible editing of the texture content; 3) Due to different platforms and different interfaces for different hardware devices Developing code and achieving compatibility at the application layer is costly.

Contents of the invention

In order to solve the above problems. The present invention proposes a lossless real-time compression method for 2D OpenGL textures based on the principle of tile segmentation. This technology is applicable to all computers, workstations, and mobile terminals based on OpenGL display, and is applicable to application environments with different programming interfaces such as OpenGL, OpenGL ES, and WebGL. For this purpose:

The present invention proposes a lossless real-time compression method for 2D OpenGL textures based on the principle of tile segmentation. The specific steps are as follows, and it is characterized in that:

1) To tile the image content and perform 2x2 full convolution on the RGBA image to reduce the resolution;

2) Compare the difference between the value after convolution and the pixel value before convolution, if there is a change, repeat step 1);

3) All values remain unchanged or the resolution has been reduced to 1x1, and the convolution process ends;

4) According to the convolution result, the next step of data encoding can be performed, and according to the optimal principle, the best splicing combination of different 2 ⁿ sizes is encoded into a 2D texture;

After the best splicing, it is encoded as a 2D texture, which is directly used for display or decoding to restore the original image or overall editing in the compressed state;

5) Do non-repetitive encoding for single-value pixels of different colors into 1D texture;

6) 2 ⁿ size position and index information are coded into a structure array;

7) Design shaders that support display and decoding;

Program the shader, accept texture data as 2D texture, non-repetitive color coding as 1D texture, position and size index as drawing coordinates, and realize drawing the original image in one rendering;

8) According to the coded 2D, 1D texture and position index information, use a dedicated shader to display all or part of the image in real time, and realize the display process of the original image effect by drawing the original image to the display buffer;

9) According to the coded 2D, 1D texture and position index information, use a dedicated shader to restore all or part of the image, draw the original image to the off-screen texture, and realize the decoding process of restoring the original image;

10) Compress and save 2D and 1D textures, positions and indexes to files, and provide a compressed texture format that can be stored for a long time.

As a further improvement of the present invention, in the step 2), the RGBA image is fully convoluted to reduce resolution by 2x2, and the formula is expressed as follows:

F is the convolution kernel, m, n is the size of the convolution kernel, generally m=n=2 ⁿ

G is the original image;

H is the image after convolution;

i, j are pixel coordinates.

As a further improvement of the present invention, the lossless real-time compression method is implemented on a GPU or implemented by chip solidified algorithm hardware.

The present invention implements a lossless real-time compression method for 2D OpenGL textures based on the principle of tile segmentation, and the design points are as follows:

Platform compatibility: can support computers, workstations and mobile terminals in a unified manner;

Language compatibility: C++, Java, JavaScript can be supported uniformly;

Editable: Textures can be edited while being compressed, or edited after being expanded in real time;

Real-time: compression, decompression and display are realized in milliseconds;

And there is no quality loss, and the compression rate of artificially drawn images is high.

Description of drawings

Figure 1 is a flow chart of the application system;

Figure 2 is a schematic diagram of the tile segmentation process of this application;

Figure 3 is a schematic diagram of monochrome image conversion in this application;

Figure 4 This application contains a schematic diagram of complex content image conversion.

Detailed ways

The present invention proposes a lossless real-time compression method for 2D OpenGL textures based on the principle of tile segmentation. This technology is applicable to all computers, workstations, and mobile terminals based on OpenGL display, and is applicable to application environments with different programming interfaces such as OpenGL, OpenGL ES, and WebGL.

Below in conjunction with accompanying drawing and specific embodiment, the present invention is further described, and concrete system flowchart is as shown in Figure 1:

1) Coding;

Image content tile detection;

Perform 2x2 full convolution on the RGBA image to reduce the resolution. The formula is expressed as follows:

F is the convolution kernel, m, n is the size of the convolution kernel, generally m=n=2 ⁿ

G is the original image;

H is the image after convolution;

i, j are pixel coordinates.

Compare the difference between the value after convolution and the pixel value before convolution, if there is no change in the content of all tiles, repeat step 1)

All values have changed or the resolution has been reduced to 1x1, and the convolution process is over;

The tile segmentation process of this application is shown in Figure 2:

The principle is to find the largest possible square area for the same pixel image features (blank, monochrome, complex pixels)

The segmentation effect is shown in Figures 3 and 4, wherein Figure 3 is a schematic diagram of monochrome image conversion, and Figure 4 is a schematic diagram of image conversion with complex content;

(The left side is the original image that needs to be compressed, and the right side is the tile mark image after convolution detection):

The tile marker map contains the sizes and positions of three types of tiles: blank tiles (blank areas), monochrome tiles, and complex tiles. According to the convolution result, the next step of data encoding can be performed.

2 optimal mosaic of tiled images of size ⁿ (2D texture);

The content is complex content tiles of different 2 ⁿ sizes, which are coded into a two-dimensional texture after optimal splicing, which can be directly used for display (GL) or decoded to restore the original image or edited as a whole in a compressed state;

No repeating color coding (1D texture);

2 ⁿ size, position and index encoding (struct array);

2) Decoding:

data:

2 The whole image after splicing tiled images of size ⁿ ;

non-repetitive color coding;

location and index codes;

shader:

A special programming shader that accepts texture data as a two-dimensional texture, non-repeating color codes as a one-dimensional texture, position and size indexes as drawing coordinates, and one rendering can realize drawing the original image;

image decoding;

Draw the original image to the off-screen texture to realize the decoding process of restoring the original image;

Image display;

Draw the original image to the display buffer to realize the display process of the original image effect;

3) editing;

simple editing;

decoding;

Edit (modify image content);

Coding after all editing is completed;

overall editing;

The overall editing supports linear operations such as rotation, translation, mirroring, etc. that do not change the pixel content of the image;

Overall editing can be realized directly in the case of compressed encoding;

4) storage;

2D image compression;

Color-coded compression;

2 ⁿ size, position and index compression.

This method involves all or part of the following applications:

1) Complete image compression and decompression process;

2) The process of obtaining each data encoding by convolution dimensionality reduction in the image compression process;

3) The encoded data is saved to files in various compression methods;

4) The tile size is 2 ⁿ , which can improve the speed of compression, decompression and display, but the method itself can support tiles of any size;

5) When the size of the encoding process is reduced from 2 to ²ⁿ , the image dimension can be reduced. This method can achieve the best resolution, but in practical applications, any gradient and series from small to large can be selected;

6) The algorithms involved in the present invention are all implemented on the GPU, and can also be implemented through chip solidified algorithm hardware.

The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any other form, and any modification or equivalent change made according to the technical essence of the present invention still belongs to the scope of protection required by the present invention .

Claims (2)

1. The lossless real-time compression method for the 2D OpenGL texture based on the tile segmentation principle comprises the following specific steps of:

1) Performing tiled detection on image content, and performing 2x2 full convolution on an RGBA image to reduce resolution;

2) Comparing the difference between the value after convolution and the pixel value before convolution, and repeating the step 1) if the change exists;

in the step 1), the RGBA image is subjected to 2x2 full convolution to reduce resolution, and the formula is expressed as follows:

f is the convolution kernel, m, n is the size of the convolution kernel, typically m=n=2 ⁿ

G is an original image;

h is the convolved image;

i, j is the pixel coordinates;

3) All values have no change or the resolution has been reduced to 1x1, and the convolution process ends;

4) According to the convolution result, the next step of data coding is carried out, and according to the optimal principle, different data of 2 ⁿ The size is optimally spliced, combined and encoded into a 2D texture;

encoding into a 2D texture after optimal splicing, wherein the texture is directly used for displaying or decoding the restored original image or editing the original image in a compressed state;

5) The single-value pixels with different colors are coded into 1D textures without repetition;

6)2 ⁿ the size, position and index information are encoded into a structure array;

7) Designing a shader supporting display and decoding;

programming a shader, namely receiving texture data as a 2D texture, taking non-repeated color codes as a 1D texture, taking position and size indexes as drawing coordinates, and rendering for one time to realize drawing of an original image;

8) Displaying all or a part of the images in real time by adopting a special shader according to the encoded 2D,1D textures and position index information, and realizing the display process of the original image effect by drawing the original image into a display buffer zone;

9) According to the coded 2D,1D texture and position index information, adopting a special shader to restore all or part of the image, drawing the original image to the off-screen texture, and realizing the decoding process of restoring the original image;

10 2D and 1D textures, locations and indices are saved to a file in compression, providing a compressed texture format that can be saved for a long period of time.

2. The method for lossless real-time compression of 2D OpenGL texture based on tile segmentation principle according to claim 1, wherein:

the lossless real-time compression method is realized in a GPU or realized through chip curing algorithm hardware.

Images