JP2010288267A - Image processing method and image processing apparatus - Google Patents

Abstract

Data relating to rendering of a three-dimensional model is efficiently managed.
A computer 20 pastes a texture of a predetermined pattern created in advance on a three-dimensional model, renders it, and analyzes a rendered image obtained as a bitmap image, thereby rendering the coordinates (x, y of the rendered image). ) And texture coordinates (Xt (x, y), Yt (x, y)), gain Gc, t (x, y), and bias Bc, t (x, y) are set. Then, the texture region is extracted from the bias Bc, t (x, y), the texture region bias Ec, t (x, y) is compressed by lossless compression, and the non-texture region bias Fc, t (x, y ) Is compressed by JPEG compression after complementing blank pixels in the texture area so that the gradation value gradually changes near the boundary.
[Selection] Figure 1

Description

  The present invention relates to an image processing method and an image processing apparatus for processing image data used when rendering by applying a texture.

  Conventionally, as this type of image processing method, a three-dimensional model is rendered in real time and displayed on a display (see, for example, Patent Document 1), and a three-dimensional model is rendered in advance to create a bitmap image. It has been proposed to save and display the display by reading a bitmap image.

Japanese Patent Application Laid-Open No. 07-152925

  In the former method, since it is necessary to perform the rendering process at a cycle shorter than the display cycle of the screen, a high calculation capability is required. Therefore, depending on the computer to be used, there is a shortage in computing ability, and high quality rendering such as ray tracing cannot be performed. On the other hand, since the latter method only displays a bitmap image, it is possible to display a high-quality image by creating a bitmap image by performing high-quality rendering in advance. Then you can't replace it with a different texture later. Further, such image data is large in size and is required to be managed efficiently depending on the capacity of the mounted memory.

  An image processing method and an image processing apparatus according to the present invention are mainly intended to efficiently manage a rendering image of a three-dimensional model.

  The image processing method and the image processing apparatus of the present invention employ the following means in order to achieve the main object described above.

The image processing method of the present invention includes:
An image processing method for processing image data used when rendering by pasting a texture,
Extracting the texture area where the texture is pasted from the image data,
The extracted texture area is compressed by a reversible compression method, and a blank pixel in the area after extracting the texture area in the image data is subjected to a predetermined complementing process so that the pixel value gradually changes at the boundary of the area. The image data after complementation is compressed by the JPEG compression method.

  In the image processing method of the present invention, a texture region to which a texture is pasted is extracted from image data, the extracted texture region is compressed by a reversible compression method, and blank pixels in the region after extracting the texture region are extracted from the image data. The pixel data is complemented by a predetermined complementing process so that the pixel value gradually changes at the boundary of the region, and the complemented image data is compressed by the JPEG compression method. Thereby, it is possible to improve the image quality while increasing the compression rate of the entire image data.

  In the image processing method of the present invention, the predetermined complementing process may be a process of complementing the blank area by filling in the average pixel value of adjacent pixels.

  In the image processing method of the present invention, the predetermined complement processing includes the image data configured by a plurality of unit blocks of a predetermined size, and includes a part of the non-texture area among the plurality of unit blocks. The block is complemented by filling the blank area with an average pixel value of adjacent pixels, and for a block not including the non-texture area, the blank pixel is replaced with the average pixel value of each pixel in the adjacent block. It can also be a process that complements by filling. In this way, the compression rate can be increased by simple processing.

  Furthermore, in the image processing method of the present invention, a predetermined pattern in which different gradation values are set for each coordinate is pasted and rendered as a texture on a three-dimensional model, and a rendered image obtained as a bitmap image by the rendering is rendered. By analyzing, the correspondence between the coordinates of the rendered image and the coordinates of the predetermined pattern is set and stored together with the image data as image drawing information, and when the desired texture is displayed as an image, the stored The desired texture may be arranged and displayed in the rendered image based on image drawing information. By doing so, it is possible to display an image obtained by rendering the three-dimensional model by replacing a desired texture, and it is possible to reduce the processing burden as compared with the case where the three-dimensional model is rendered and displayed in real time. Here, the display of the image includes an image that is drawn in units of frames and displayed as a moving image. In this aspect of the image processing method of the present invention, the correspondence relationship may be derived by specifying the corresponding coordinates of the predetermined pattern from the gradation value of each coordinate of the rendered image. In this aspect of the image processing method of the present invention, the predetermined pattern is a plurality of patterns corresponding to the number of bits when coordinates are expressed in binary numbers, and gradation values corresponding to the values of bits corresponding to the respective coordinates. Can also be set. In this way, the correspondence can be set more accurately. In this case, the binary number may be a gray code (alternate binary number). In this way, since only 1-bit change always occurs when moving to an adjacent coordinate, it is possible to suppress erroneous data from being acquired due to an error in the gradation value of the image. In the image processing method of the present invention according to these aspects, in addition to the correspondence setting pattern for setting the correspondence as the predetermined pattern, the first solid pattern formed by solid painting with a minimum gradation value is further described. A bias value that is a gradation value of the first solid pattern in the rendered image is stored as the image drawing information, and is rendered based on the stored bias value. It is also possible to display the image after converting it into the gradation value of the rendered image by offsetting the gradation value. By doing this, it is possible to reflect the effect of rendering the three-dimensional model that does not depend on the original texture. In this case, the bias value of the texture area extracted from the bias value of each pixel as the image data is compressed by the lossless compression method, and blank pixels in the area after extraction of the texture area are complemented by a predetermined complement process. In addition, the bias value after the complement can be compressed by the JPEG compression method. Furthermore, in the image processing method of the present invention of these aspects, in addition to the correspondence setting pattern for setting the correspondence as the predetermined pattern, a first solid pattern that is further solid-coated with a minimum gradation value And the second solid pattern formed by applying a solid color with the maximum gradation value are pasted on the three-dimensional model and rendered, respectively, and the gradation value of the second solid pattern and the first solid pattern in the rendered image are rendered. A gain that is a deviation from the gradation value of the paint pattern is calculated and stored as the image drawing information, and the gradation value of the desired texture is converted into the gradation value of the rendered image based on the stored gain. It can also be displayed. In this way, it is possible to reflect the effect of the original texture gradation value among the effects of rendering the three-dimensional model. In this case, when a plurality of desired textures are arranged and displayed on the rendered image, the second solid coating is a set group provided by the number of the desired textures to be arranged. A first solid coating pattern obtained by subtracting the value 1 from the number of textures and the number of textures to be arranged, and the place where the second solid coating pattern is applied to the three-dimensional model is different for each set. A set group and one second set including the same number of the first solid coating patterns as the number of the desired textures to be arranged are pasted and rendered on the three-dimensional model for each set, and the first set is rendered. Render the tone values of each rendered image and the second set obtained by rendering a set group for each set. A texture region which is a region where a texture is pasted on the three-dimensional model by comparing the gradation value of the rendered image obtained by each of the sets of the first set group, The gain may be calculated for the identified texture region. In this way, the texture area can be specified more easily.

The image processing apparatus of the present invention
An image processing apparatus that processes image data used when rendering by pasting a texture,
Preprocessing means for extracting a texture region to which the texture is pasted from the image data;
First compression means for compressing the extracted texture region by a reversible compression method;
A second compression unit that complements the blank pixels of the region after extracting the texture region of the image data by a predetermined complementing process and compresses the complemented image data by a JPEG compression method. Is the gist.

  In the image processing apparatus of the present invention, a texture region to which a texture is pasted is extracted from image data, the extracted texture region is compressed by a lossless compression method, and blank pixels in the region after extracting the texture region from the image data are extracted. The image data is complemented by a predetermined complementing process and the complemented image data is compressed by the JPEG compression method. Thereby, it is possible to improve the image quality while increasing the compression rate of the entire image data.

The block diagram which shows the outline of a structure of the computer 20 used for an image processing method. The flowchart which shows an example of a special texture production | generation process. Explanatory drawing which shows an example of a special texture. Explanatory drawing which shows a mode that a special texture is rendered for every set. 10 is a flowchart illustrating an example of rendered image analysis processing. Explanatory drawing explaining bias Bc, t (x, y) and gain Gc, t (x, y). The flowchart which shows an example of a compression process. The flowchart which shows an example of a complementation process. Explanatory drawing which shows the mode of a blank pixel complement. Explanatory drawing which shows the texture for three replacements. Explanatory drawing which shows an example of the slide show display of the texture for replacement. Explanatory drawing which shows the special texture of a modification. Explanatory drawing which shows a mode that it renders using the special texture of a modification. Explanatory drawing which shows the special texture of a modification.

  Next, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a configuration diagram showing an outline of configurations of a computer 20 and a viewer 40 used in an image processing method according to an embodiment of the present invention. The computer 20 of this embodiment is a general-purpose computer comprising a central processing unit (CPU), a ROM for storing processing programs, a RAM for temporarily storing data, a graphic processor (GPU), a hard disk (HDD), a display 22 and the like. It is configured as a computer, and as its functional blocks, a storage unit 31 for storing 3D modeling data (hereinafter referred to as a 3D model), texture data (hereinafter referred to as texture) to be pasted thereon, and the 3D model A special texture generation processing unit 32 that generates a special texture for preprocessing to be pasted on a rendering unit, a rendering processing unit 34 that generates a bitmap image by rendering a three-dimensional model, and a bitmap image obtained by rendering Render Includes a rendered image analysis processing section 36 for analyzing the grayed completed image, the compression processing unit 38 compresses the various kinds of data obtained by analyzing the rendered image and the rendered image analysis processing unit 36 which generated.

  The special texture generation processing unit 32 is a processing unit that generates a texture of a predetermined pattern to be pasted on the three-dimensional model rendered by the rendering processing unit 34. Specifically, as the predetermined pattern, values 0.0 to 1.. A solid white pattern with a gradation value of 1.0 within a gradation value range of 0, a solid black pattern with a gradation value of 0.0, and a gradation value of 0.0 and 1.0 A vertical stripe pattern that alternately appears in the horizontal direction, and a horizontal stripe pattern that has gradation values of 0.0 and 1.0 appear alternately in the vertical direction. The role of each pattern will be described later.

  The rendering processing unit 34 is a processing unit that functions when software for 3D rendering is installed in the computer 20. The rendering processing unit 34 performs rendering by pasting the texture generated by the special texture generation processing unit 32 on the three-dimensional model. Thus, a bitmap image is reproduced frame by frame at a predetermined frame rate (for example, 30 times or 60 times per second) to display a moving image. In the present embodiment, the rendering process is performed using a ray tracing method that calculates and renders reflection of an object surface, light refraction, and the like while tracing light from a light source.

  The rendered image analysis processing unit 36 analyzes the bitmap image (rendered image) generated by the rendering processing unit 34 to freely replace desired image data such as a photograph instead of a predetermined pattern texture. Image drawing information for enabling the rendered image to be displayed on the viewer 40 side is generated.

  The compression processing unit 38 is a processing unit for compressing the image drawing information generated along with the analysis by the rendered image analysis processing unit 36. In the present embodiment, the overall compression rate is suppressed while suppressing deterioration in image quality. In order to improve the data, the data is compressed by using different types of compression methods. Details of the compression method will be described later.

  The viewer 40 according to the present embodiment stores a rendered image obtained by the rendering processing unit 34 of the computer 20, image rendering information analyzed by the rendered image analysis processing unit 36 and compressed by the compression processing unit 38. A storage unit 41, an input processing unit 42 that controls input of image data such as photographs stored in the memory card MC, image data input by the input processing unit 42, and rendered images stored in the storage unit 41 , A development processing unit 44 that decodes (decompresses) image drawing information, and a drawing processing unit 46 that combines and renders image data input to the rendered image as a texture. The viewer 40 sequentially reads a plurality of image data stored in the memory card MC according to an instruction from a user, and pastes the read image data on a rendered image of a three-dimensional model using image drawing information, and sequentially reproduces the image data. The slide show display can be performed.

  Next, the operations of the special texture generation processing unit 32, the rendering processing unit 34, the rendered image analysis processing unit 36, and the compression processing unit 38 of the computer 20 of the present embodiment configured as described above, the decompression processing unit 44 of the viewer 40, The operation of the drawing processing unit 46 will be described. First, the processing of the special texture generation processing unit 32 will be described. FIG. 2 is a flowchart illustrating an example of the special texture generation process.

  In the special texture generation process, first, the target set number i is initialized to a value 1 (step S100), and n special textures are generated for each RGB color component for the target set number i (step S110). The target set number i is incremented by 1 (step S120), the target set number i is compared with the value n (step S130), and when the target set number i is less than or equal to the value n, the process returns to step S110 and the next target The process of generating n special textures for the set number i is repeated. When the target set number i exceeds the value n, the process proceeds to the next process. Here, the generation of the special texture with the target set number i from the value 1 to the value n shifts the target texture number j from the first to the nth by the value 1 from the first as shown in the following equation (1). While comparing the target texture number j and the target set number i, the target texture number j that matches the target texture number j is in the gradation value range from the minimum value 0.0 (black) to the maximum value 1.0 (white). A special white solid texture is generated by setting a gradation value of 1.0 to all coordinates (x, y), and the target texture number j that does not match is set to all coordinates (x, y). This is done by generating a solid black special texture by setting a gradation value of 0.0. Here, “c” in the formula (1) indicates a value corresponding to each color of the RGB values of the image data, “n” indicates the number of textures arranged on one screen, and “b” indicates the texture coordinates. Indicates the number of bits when expressed as a binary number, and “Tc, i, j (x, y)” indicates the coordinates (x, y) of the special texture in the color component c, the target set number i, and the target texture number j. Indicates a gradation value (hereinafter the same).

  When the special texture having the target set number i having the value 1 to the value n is generated, n special textures for each color component having the target set number i having the value (n + 1) are generated (step S140). i is incremented by 1 (step S150). Here, the generation of the special texture having the target set number i of the value (n + 1) is performed with respect to all the target texture numbers j from 1 to n as shown in the following equation (2). This is done by generating a solid black special texture by setting a gradation value of 0.0 to y).

  When the special texture having the target set number i of the value (n + 1) is generated, the {i− (n + 2)] when the texture coordinates are expressed as an alternating binary number (gray code) with respect to the target set number i. N special textures for each color component of the vertical stripe pattern corresponding to the bit are generated by the following equation (3) (step S160), the target set number i is incremented by 1 (step S170), and the target set number i Is compared with the value (n + b + 1) (step S180), and when the target set number i is equal to or smaller than the value (n + b + 1), the process returns to step S160 to generate n special textures for the next target set number i. Repeatedly, when the target set number i exceeds the value (n + b + 1), the process proceeds to the next process. Here, “gray (a)” in Equation (3) is a gray code (alternate binary code) representation of the numerical value a, and “and (a, b)” is the logical product of a and b for each bit. Shown (same below). The target set numbers i from (n + 2) to (n + b + 1) are the 0th bit (most significant bit) to the (b-1) th bit (least significant bit) when the texture coordinates are expressed in binary numbers, respectively. The gradation value of 1.0 (white) is set when the value of the bit corresponding to the target set number i is 1, and the value of the corresponding bit is 0. A special texture having a vertical stripe pattern is generated by setting a gradation value of 0.0 (black). In this embodiment, the texture coordinates are expressed in alternating binary numbers. For example, if the texture number n is 3 and the coordinates are 3 bits (b = 3), the target set number i is the first number. As a special texture with a value of 5 indicating 0 bit (most significant bit), a black gradation value is set for an x coordinate value of 1 to 4, a white gradation value is set for a value of 5 to 8, and a target set is set. As for the special texture having the value 6 in which the number i indicates the first bit, the black coordinate value is set for the x coordinate values 1 and 2, the white tone value is set for the values 3 to 6, and the values 7 and 8 are set. Is a black gradation value, and the target set number i is a special texture with a value 7 indicating the second bit (least significant bit). For 3 the white tone value is set and for values 4 and 5 So that the black gradation value is set for the gradation value set value 8 white for black gradation value set value 6,7.

  When the special texture having the target set number i of the value (n + 2) to the value (n + b + 1) is generated, the {i− (n + b + 2) when the y coordinate of the texture is expressed as an alternating binary number with respect to the target set number i. )] N special textures for each color component of the horizontal stripe pattern corresponding to the bit are generated by the following equation (4) (step S185), the target set number i is incremented by 1 (step S190), and the target set is set. The number i is compared with the value (n + 2b + 1) (step S195). When the target set number i is equal to or smaller than the value (n + 2b + 1), the process returns to step S185 to generate n special textures for the next target set number i. When the process is repeated and the target set number i exceeds the value (n + 2b + 1), the generation of all the special textures is completed. This routine is terminated. The target set numbers i from (n + b + 2) to (n + 2b + 1) are the 0th bit (most significant bit) to the (b-1) th bit (least significant bit) when the texture coordinates are expressed in binary numbers, respectively. The gradation value of 1.0 (white) is set when the value of the bit corresponding to the target set number i is 1, and the value of the corresponding bit is 0. A special texture with a horizontal stripe pattern is generated by setting a gradation value of 0.0 (black). In this embodiment, the texture coordinates are expressed by alternating binary numbers. For example, if the texture number n is 3 and the y coordinate is 3 bits (b = 3), the target set number i is As a special texture having a value of 8 indicating the 0th bit (most significant bit), a black gradation value is set for a y coordinate of 1 to 4, a white gradation value is set for a value of 5 to 8, and As a special texture having a set number i with a value 9 indicating the first bit, a black gradation value is set for values 1 and 2, and a white gradation value is set for values 3 to 6, and a value of 7, A black tone value is set for 8 and a special texture with a target set number i indicating the second bit (least significant bit) of value 10 is set to a y tone value of 1 and a black tone value is set. For 2 and 3, the white gradation value is set to values 4 and 5. There will be a black tone value is set for the gradation value set value 8 white for black gradation value set value 6,7 is. FIG. 3 shows a list of special textures generated when the number of textures n is 3 and the number of coordinate bits b is 3.

  For each set, the rendering processing unit 34 performs rendering processing by pasting the corresponding n special textures onto the three-dimensional model. FIG. 4 shows the rendering process. In this embodiment, since the three-dimensional model is rendered as a moving image, the number of textures n is 3 and the number of bits b is 3, the rendering process for a total of 10 sets is performed and 10 sets of moving images are generated. Will be. This moving image is composed of bitmap images (rendered images) generated for each frame from frames 1 to T.

  Next, processing for analyzing a rendered image generated by the rendering processing unit 34 will be described. FIG. 5 is a flowchart illustrating an example of a rendered image analysis process executed by the rendered image analysis processing unit 36.

  In the rendered image analysis process, first, as shown in the following equation (5), the variable It (x, y) of the coordinates (x, y) of the rendered image at each frame number t (= 1 to T) is set to the value 0. (Step S200), a white solid area (coordinates) in the rendered image of the set numbers 1 to n in the target frame t is specified, and the texture corresponding to the variable It (x, y) of this white solid area A number (= target set number i) is set (step S210). As shown in the following equation (6), this processing is performed by sequentially shifting the target set number i from No. 1 to n, and the gradation value of the rendered image of the target set number i (the gradation value for each color component). This can be done by comparing the sum of the tone values of the rendered image with the set number (n + 1) (the sum of the tone values for each color component). Here, “w” in equation (5) indicates the size in the width direction of the rendered image, and “h” indicates the size in the height direction of the rendered image. In addition, “Ac, i, t (x, y)” in the equation (6) is a rank of the coordinate (x, y) of the rendered image at the color component c, the set number i (1 to n), and the frame number t. Indicates the key value (hereinafter the same).

  Subsequently, the gradation value of the rendered image with the set number (n + 1) is set as the bias Bc, t (x, y) by the following equation (7) (step S220), and the variable It (x, y) is a value. The gain Gc, t (x, y) is calculated by the following equation (8) for the coordinates (x, y) of the rendered image that is not 0, that is, the white solid region (step S230). Here, “Ac, It (x, y), t (x, y)” in the equation (8) is the color component c and the set number i and the frame number t stored in the variable It (x, y). Indicates the gradation value of the coordinates (x, y) of the rendered image. FIG. 6 shows the relationship between the bias Bc, t (x, y) and the gain Gc, t (x, y). When rendering with a texture attached to a 3D model, as shown in the figure, the offset that does not depend on the gradation value of the original texture corresponds to the bias Bc, t (x, y), and the gradation of the original texture The gradient of the change in the gradation value of the rendered image with respect to the change in value corresponds to the gain Gc, t (x, y).

  Then, the coordinates (X't (x, y), Y't (x, y)) of the texture gray code expression are initialized to the value 0 by the following equation (9) (step S240), and the set number (n + 2) Correspondences between the coordinates (x, y) of the rendered image (n + 2b + 1) and the coordinates (X′t (x, y), Y′t (x, y)) of the texture are set (step S250). Here, the coordinate correspondence is performed by the following equation (10). Specifically, the gradation value of the rendered image of the set number (i + n + 1) while sequentially shifting the set number i from 1 to n. Ac, i + n + 1, t (x, y) minus bias Bc, t (x, y) (sum of each color component) is the gain Gc, t (x , y) greater than 2 divided by the value 2 (total for each color component), that is, whether the coordinates (x, y) of the white and black vertical stripe patterns in the set number (i + n + 1) are white If it is white, the value (1) is set to the value of the corresponding (i−1) -th bit of the coordinate X′t (x, y) in the alternating binary number expression, and the set number i is set from 1 to n Subtracting the bias Bc, t (x, y) from the gradation value Ac, i + b + n + 1, i (x, y) of the rendered image of the set number (i + b + n + 1) while sequentially shifting (each color Whether or not (sum per minute) is greater than the gain Gc, t (x, y) of the rendered image with set number i divided by the value 2 (sum for each color component), ie at set number (i + b + n + 1) It is determined whether or not the coordinate (x, y) is white in the white and black horizontal stripe pattern, and if it is white, the value of the corresponding (i−1) -th bit of the coordinate Y′t (x, y) is the value. This is done by setting it to 1. Here, “or (a, b)” in the expression (10) indicates a logical sum of bits a and b.

  When the correspondence relationship of coordinates is set, the coordinates (X't (x, y), Y't (x, y)) of the texture of the gray code expression are decoded using the following equation (11) to obtain the decoded coordinates (Xt (x, y), Yt (x, y)) is calculated (step S260), and it is determined whether or not the processing has been completed for all the frames having values 1 to T (step S270). When the process is not completed, the next frame is set as the target frame t, and the process returns to step S210 to repeat the process. When the process is completed for all frames, the process is terminated. Here, “gray-1 (a)” in Expression (11) indicates a value obtained by decoding the Gray code a, and “Xt (x, y)” indicates the coordinates (x, y) represents the x coordinate of the texture, and “Yt (x, y)” represents the y coordinate of the texture corresponding to the coordinate (x, y) of the rendered image of frame number t. In this embodiment, since the origin of the coordinates (X′t (x, y), Y′t (x, y)) is (1,1), the value 1 is set to the value obtained by decoding the Gray code. It is adding. Image rendering information includes variable It (x, y), bias Bc, t (x, y), gain Gc, t (x, y), and coordinates (Xt (x, y), Yt (x, y)) And are included.

  Next, processing for compressing image drawing information obtained by analysis by the rendered image analysis processing unit 36, particularly processing for compressing the bias Bc, t (x, y) will be described. FIG. 7 is a flowchart illustrating an example of the compression process executed by the compression processing unit 38.

  In the compression process, first, the texture area is extracted from the bias Bc, t (x, y) of the entire area, and the texture area bias Ec, t (x, y) is set (step S300). This process sets the texture region bias Ec, t (x, y) to the bias Bc, t (x, y) for the coordinates (x, y) where the variable It (x, y) is not 0, and the variable The coordinate (x, y) where It (x, y) is 0 is performed by setting the value 0 to the texture region bias Ec, t (x, y). Subsequently, the non-texture area bias Bc, t (x, y) is extracted to set the non-texture area bias Fc, t (x, y) (step S310), and the set non-texture area bias Fc, Blank pixels in the texture area are complemented from t (x, y) (step S320), and deflate compression, which is one of the lossless compression methods, is performed on the texture area bias Ec, t (x, y) (step S320). In step S330, JPEG compression is performed for the non-textured region (step S340), the compressed data is saved (step S350), and the process ends. In this embodiment, It (x, y), Gc, t (x, y), Xt (x, y), and Yt (x, y) as image drawing information are also compressed by lossless compression such as deflate compression. It was supposed to be.

  FIG. 8 is a flowchart illustrating an example of a complement process for complementing blank pixels in step S320 of FIG. In this complementing process, first, the target MCU is set in the order of the width direction (x coordinate direction) for each column (y coordinate) in the MCU (Minimum Coded Unit) which is the basic unit of JPEG processing (step S400). It is determined whether or not there is a blank pixel in the target MCU (step S410). When there is no blank pixel in the target MCU, it is not necessary to complement the blank pixel, so it is determined whether or not there is a next MCU (step S450). Set and repeat the process. On the other hand, if there is a blank pixel in the target MCU, it is determined whether or not a part in the target MCU is a blank pixel (step S420). If a part in the target MCU is a blank pixel, the blank pixel is determined as an adjacent pixel. Is filled with the average gradation value of the adjacent MCU (step S430). When all of the target MCUs are blank pixels, a process of filling with the average gradation value of the MCU adjacent to the left side is performed (step S440). move on. If it is determined in step S450 that there is a next MCU, the process returns to step S400, the next MCU is set as the target MCU, and the process is repeated. If it is determined that there is no next MCU, this process ends. To do. FIG. 9 shows how blank pixels are complemented. This process is such that the gradation value gradually changes near the boundary between the texture area and the non-texture area, and is filled with the average gradation value of the MCU near the boundary at a part away from the boundary. By compressing the non-texture region bias Fc, t (x, y) supplemented with blank pixels in this way by JPEG compression, the compression efficiency during JPEG compression can be increased.

  Image drawing information (variable It (x, y), bias Ec, (x, y), Fc, t (x, y), gain Gc, t (x, y)) compressed by the compression processing unit 38 of the computer 20 , Coordinates (Xt (x, y), Yt (x, y)) are stored in the storage unit 41 of the viewer 40, and the variable It (x, y) and the bias Ec, For t (x, y), gain Gc, t (x, y), and coordinates (Xt (x, y), Yt (x, y)), the bias Fc, t (x, y) compressed by JPEG by deflate decompression processing ) Is developed by JPEG decompression processing and used for drawing processing by the drawing processing unit 46. The drawing processing unit 46 reads a plurality of image data such as photographs stored in the memory card MC as replacement textures. In addition, the texture can be replaced by combining the rendered image using the following equation (12) and drawing sequentially. Slide show playback that displays rendered images of the three-dimensional model, where “Uc, i (x, y)” in equation (12) is the texture for replacement in the color component c and texture number i. Represents the gradation value (0.0 to 1.0) of the coordinates (x, y) of the image, and “Pc, t (x, y)” represents the display image (rendered image) in the color component c and frame number t. Indicates the gradation value (0.0 to 1.0) of the coordinates (x, y) As shown in the equation (12), the setting of the gradation value Pc, t (x, y) of the display image is a variable. For texture placement areas where It (x, y) is not 0, the replacement texture coordinates (Xt (x, y), Yt (x, y)) corresponding to the display image coordinates (x, y) Set the gradation value multiplied by the gain Gc, t (x, y) and add the bias Ec, t (x, y), and set the variable It (x, y) to a value other than the texture placement area Set bias Fc, t (x, y) for the region. Is carried out by. Texture number in FIG. 10 shows three for replacement texture 1-3 illustrates how to draw by placing a replacement for texture 10 in the rendered image in FIG. 11.

  According to the image processing method of the embodiment described above, bias Bc, t (x as one of image drawing information obtained by pasting and rendering a special texture on a three-dimensional model and analyzing the rendered image , y), extract the texture area, set the texture area bias Ec, t (x, y), and complement the blank pixels so that the gradation value gradually changes near the area boundary, then the non-texture area Bias Fc, t (x, y) for texture, bias Ec, t (x, y) for texture area is compressed by deflate compression, and bias Fc, t (x, y) for non-texture area Since compression is performed by JPEG compression, data can be compressed at a high compression rate while suppressing deterioration of the overall image quality. On the computer 20 side, a vertical stripe pattern for x coordinates and a horizontal stripe pattern for y coordinates corresponding to the value of each bit when coordinates (x, y) are expressed in binary are used as a texture in three dimensions. Rendering by pasting to the model and analyzing the rendered image obtained as a bitmap image by rendering, the coordinates (x, y) of the rendered image and the coordinates of the texture (Xt (x, y), Yt (x , y)) are set and stored as image drawing information, and when the rendered image is displayed on the viewer 40 side, the coordinates of the texture (Xt (x, Since rendering is performed at the coordinates (x, y) of the display image based on the gradation values of y), Yt (x, y)), the rendered image of the three-dimensional model can be freely replaced with the texture and played. In addition, the processing load can be reduced as compared with a case where a three-dimensional model is rendered and displayed in real time. In addition, since the tone value of the texture is converted by using the gain Gc, t (x, y) and the bias Bc, t (x, y) and the tone value of the display image is set, the three-dimensional model is rendered. It is also possible to reflect the effects of refracted light, specular reflection, and shadows. Furthermore, since a vertical striped pattern and a horizontal striped pattern corresponding to alternating binary numbers are formed as a special texture for specifying the correspondence of coordinates, a change of 1 bit always occurs when moving to adjacent coordinates. Therefore, it is possible to suppress erroneous data from being acquired due to an error in the gradation value of the image.

  In the present embodiment, as a complementary process, when a part of the target MCU is a blank pixel, the blank pixel is filled with the average gradation value of the adjacent pixel, and when all of the target MCU is a blank pixel, the average of the MCUs adjacent to the left side is averaged. However, the present invention is not limited to this, and it is only necessary to complement the gradation value so that it gradually changes in the vicinity of the region boundary. It is good.

  In the present embodiment, the texture region bias Ec, t (x, y) is compressed by deflate compression, but may be compressed by any lossless compression process other than deflate compression, It is good also as what to do.

  In the present embodiment, the texture region bias Ec, t (x, y) and the non-texture region bias Fc, t (x, y) are set from the bias Bc, t (x, y), and the compression methods are different from each other. However, the data may be applied to data other than the bias Bc, t (x, y) as long as the data is used when rendering with the texture attached to the three-dimensional model.

  In the present embodiment, a three-dimensional pattern is obtained by using a vertical stripe pattern for x coordinates and a horizontal stripe pattern for y coordinates corresponding to the value of each bit when coordinates (x, y) are expressed in binary notation as a texture. Image rendering information is generated by pasting and rendering the model and analyzing the rendering result. However, the pattern used is not limited to this, and the gradation (gradation value) gradually increases in the x-coordinate direction (lateral direction). It is also possible to use a pattern that changes to a pattern and a pattern in which shading gradually changes in the y-coordinate direction (vertical direction). In this case, a single pattern of the set number (n + 2) obtained by the following equation (13) is used instead of the vertical stripe pattern of the set numbers (n + 2) to (n + b + 1) obtained by the equation (3) described above and the equation Instead of the horizontal stripe pattern of the set numbers (n + b + 2) to (n + 2b + 1) obtained by (4), one pattern of the set number (n + 3) obtained by the following equation (13) may be used.

  When the pattern of Expression (12) and the pattern of Expression (13) are used, the setting of the coordinate correspondence can be obtained by the following Expression (15). FIG. 12 shows an example of the special texture, and FIG. 13 shows a state in which the special texture of FIG. 12 is pasted on the three-dimensional model and rendered. Thereby, the number of special textures to be generated can be reduced.

  In the present embodiment, the special texture of the vertical stripe pattern whose target set number i is a value (n + 2) to a value (n + b + 1) corresponds to the value of each bit when the coordinates are expressed in alternating binary numbers, and the target set number The special texture of the horizontal stripe pattern with i being the value (n + b + 2) to the value (n + 2b + 1) is assumed to correspond to the value of each bit when the coordinates are expressed in an alternating binary number. It may be generated as a value corresponding to the value of each bit when expressed in decimal. An example of the special texture in this case is shown in FIG.

  In the present embodiment, an image is reproduced by the viewer 40, but any device such as a mobile phone with a liquid crystal screen or a printer may be used as long as the device can reproduce an image.

  It should be noted that the present invention is not limited to the above-described embodiment, and it goes without saying that the present invention can be implemented in various modes as long as it belongs to the technical scope of the present invention.

  20 computers, 22 displays, 31 storage units, 32 special texture generation processing units, 34 rendering processing units, 36 rendered image analysis processing units, 38 compression processing units, 40 viewers, 41 storage units, 42 input processing units, 44 decompression processings Part, 46 drawing processing part, MC memory card.

Claims (13)

An image processing method for processing image data used when rendering by pasting a texture,
Extracting the texture area where the texture is pasted from the image data,
The extracted texture area is compressed by a reversible compression method, and a blank pixel in the area after extracting the texture area in the image data is subjected to a predetermined complementing process so that the pixel value gradually changes at the boundary of the area. An image processing method comprising: complementing and compressing the complemented image data by a JPEG compression method. The image processing method according to claim 1,
The predetermined complementing process is an image processing method in which the blank area is complemented by filling an average pixel value of adjacent pixels. The image processing method according to claim 2,
In the predetermined complement processing, the image data is configured by a plurality of unit blocks of a predetermined size, and the blank area is adjacent to a block including a part of the non-texture area among the plurality of unit blocks. Image processing is a process of complementing by filling with an average pixel value of pixels, and for a block that does not include the non-textured region, complementing by filling the blank pixels with an average pixel value of each pixel in an adjacent block Method. The image processing method according to any one of claims 1 to 3,
Render by pasting a predetermined pattern with different gradation values for each coordinate as a texture to a 3D model,
By analyzing the rendered image obtained as a bitmap image by the rendering, the correspondence between the coordinates of the rendered image and the coordinates of the predetermined pattern is set and stored as image drawing information together with the image data,
An image processing method, wherein when displaying a desired texture as an image, the desired texture is arranged and displayed in the rendered image based on the stored image drawing information.   The image processing method according to claim 4, wherein the correspondence relationship is derived by specifying the corresponding coordinates of the predetermined pattern from the gradation value of each coordinate of the rendered image.   6. The predetermined pattern is formed by setting a gradation value corresponding to a bit value corresponding to each coordinate in a plurality of patterns corresponding to the number of bits when coordinates are expressed in binary numbers. Image processing method.   The image processing method according to claim 6, wherein the binary number is a gray code (alternate binary number). An image processing method according to any one of claims 4 to 7,
In addition to the correspondence setting pattern for setting the correspondence as the predetermined pattern, a first solid pattern that is further solid-coated with a minimum gradation value is pasted on the three-dimensional model and rendered,
A bias value that is a gradation value of the first solid pattern in the rendered image is stored as the image drawing information;
An image processing method for converting and displaying the gradation value of the rendered image by offsetting the gradation value of the desired texture based on the stored bias value.   The bias value of the texture area extracted from the bias value of each pixel as the image data is compressed by the lossless compression method, and blank pixels in the area after extracting the texture area are complemented by a predetermined complementing process and The image processing method according to claim 8, wherein the bias value after complementing is compressed by a JPEG compression method. An image processing method according to any one of claims 4 to 9,
In addition to the correspondence setting pattern for setting the correspondence as the predetermined pattern, a first solid pattern that is solid-coated with a minimum gradation value and a second solid that is solid-coated with a maximum gradation value A paint pattern is pasted on the 3D model and rendered.
Calculating a gain that is a deviation between the gradation value of the second solid pattern and the gradation value of the first solid pattern in the rendered image, and storing the gain as the image drawing information;
An image processing method for converting a gradation value of the desired texture into a gradation value of the rendered image based on the stored gain and displaying the converted gradation value. The image processing method according to claim 10, comprising:
In the case where a plurality of desired textures are arranged and displayed on the rendered image, a set group provided by the number of the desired textures to be arranged, each set being one second solid pattern and the A first set group consisting of the number of the first solid coating patterns obtained by subtracting the value 1 from the number of textures to be arranged, and the place where the second solid coating pattern is applied to the three-dimensional model for each set; , One second set consisting of the same number of the first solid pattern as the number of the desired textures to be arranged is pasted to the three-dimensional model for each set, and rendered.
The gradation value of each rendered image obtained by rendering the first set group for each set and the gradation value of the rendered image obtained by rendering the second set are the first values. An image processing method for identifying a texture region, which is a region where a texture is pasted on the three-dimensional model, by comparing each set of the set group, and calculating the gain for the identified texture region.   The image processing method according to claim 1, wherein the image is drawn in units of frames and displayed as a moving image. An image processing apparatus that processes image data used when rendering by pasting a texture,
Preprocessing means for extracting a texture region to which the texture is pasted from the image data;
First compression means for compressing the extracted texture region by a reversible compression method;
An image comprising: a second compression unit that complements the blank pixels in the area after extracting the texture area from the image data by a predetermined complementing process and compresses the complemented image data by a JPEG compression method. Processing equipment.