JP3990055B2 - Image encoding / decoding system, image encoding / decoding method, image encoding device, image decoding device, image encoding method, image decoding method, and image encoding / decoding device - Google Patents

Description

[0001]
(table of contents)
TECHNICAL FIELD OF THE INVENTION
Conventional technology (FIGS. 62 to 68)
Problems to be solved by the invention
Means for solving the problem
BEST MODE FOR CARRYING OUT THE INVENTION
(A) Description of one embodiment (FIGS. 1 to 38)
(B) Description of the first modification (FIGS. 39 to 41)
(C) Description of the second modification (FIGS. 42 to 48)
(D) Description of the third modification (FIGS. 49 to 58)
(E) Others (FIGS. 59 to 61)
The invention's effect
[0002]
BACKGROUND OF THE INVENTION
The present invention relates to a technique for efficiently compressing (encoding) and restoring (decoding) image information (image data) such as a still image and a moving image, and in particular, a communication (transmission) path or accumulation in which an error exists. Image encoding / decoding system, image encoding apparatus, image decoding apparatus, image encoding method, image decoding method, and image encoding / decoding suitable for use when an error occurs in encoded information through a medium or the like Relates to the device.
[0003]
[Prior art]
FIG. 62 is a block diagram showing a basic configuration of a conventional image communication system (image encoding / decoding system). The system 1 a shown in FIG. 62 includes an encoding unit 101 and a multiplexing 102 as an image encoding device 100. In addition, the image decoding apparatus 200 includes a multiplexing decoding unit 201 and a decoding unit 202, and these apparatuses 100 and 200 are connected to each other via a predetermined transmission path (communication path) 300. Yes.
[0004]
Here, in the image encoding device 100, the encoding unit 101 uses the input image signal F to generate one or more pieces of image data by a predetermined encoding method (algorithm) such as predictive encoding, transform encoding, or vector quantization. Is encoded in units of blocks of pixels, and predetermined encoding parameters e1, e2, ..., ex (x is a natural number) such as quantization step size and DCT (discrete cosine transform) coefficient are calculated. The encoding parameters e1, e2,..., Ex are output as encoding information.
[0005]
Note that the encoding parameters e1, e2,..., Ex are not limited to scalar quantities, but are, for example, two-dimensional motion vectors or conversion counts in the case of block transform encoding (the number of dimensions is one or more dimensions). There is. Further, when encoding is performed with reference to the decoding result in the encoding unit 101, the configuration is as shown in FIG. 63, for example. That is, the encoding parameters e1, e2,..., Ex output from the encoding unit 101 are decoded by the local decoding unit 103 having the same configuration (decoding algorithm) as the decoding unit 202 in the image decoding device 200, The decoding result F ′ is input to the encoding unit 101 and referenced.
[0006]
Further, the multiplexing unit 102 multiplexes the encoding parameters e1, e2,..., Ex obtained by the encoding unit 101. Specifically, the encoding parameters e1, e2,. .., ex, variable-length code or fixed-length code according to a predetermined syntax [transmission format: a grammar that allows information to be arranged in a bit string and that the receiving side converts the bit string into encoded information (multiplex decoding)] And the bit string is output to the transmission line 300.
[0007]
For example, as illustrated in FIG. 64, the multiplexing unit 102 performs unique recovery of the unique word 400 (synchronized recovery) in the middle of the beginning of a bit string or the beginning of a block line, variable length code, or fixed length code in the case of block coding. (Synchronization signal) is added (the packet from unique word to unique word that can be recovered synchronously may be called a packet), and encoding parameters e1, e2, ..., variable length coding or fixed length coding, A bit string is created by adding information about ex and other images.
[0008]
On the other hand, in the image decoding device 200 (receiving side), the multiplexing decoding unit 201 multiplex-decodes the encoding parameters e1, e2,..., Ex from the bit string input from the transmission path 300. Usually, since an error is added in the transmission line 300 (a code error occurs), the multiplexed and decoded encoding parameters are e1 ′, e2 ′,..., Ex ′, for example, and are not necessarily the original encoding parameters. It is not the same as e1, ..., ex.
[0009]
The decoding unit 202 decodes the image data based on the encoding parameters e1 ′, e2 ′,..., Ex ′ multiplexed and decoded by the multiplexing decoding unit 201, and the decoding result is reproduced. It is output as an image signal F ″.
Here, as a specific example of a currently well-known image encoding / decoding technique, there are techniques for internationally standardized moving pictures as shown in (1) and (2) below.
[0010]
(1) ISO / IEC JTC1 / SC29 / WG11 N2202 (MPEG4) Simple Profile
(2) ITU-T Recommendation H.263
Next, these techniques will be described as typical conventional techniques.
FIG. 65 is a block diagram showing a configuration example of a main part of the image encoding / decoding system relating to (2) above. In the system 1b shown in FIG. 65, the image transmission apparatus 500 and the image transmission apparatus 600 are connected to the transmission path 300. Are respectively connected to each other as transmission systems (video coder) 500A and 600A, encoding units (source coder) 501, 601 and multiplexing units (video multiplex coder) 502, 602, respectively. , Transmission buffers 503 and 603, coding controllers 504 and 604, and receiving systems (video decoders) 500B and 600B, receiving buffers 505 and 605, respectively. , Multiplex decoding sections (video multiplex decoder) 506, 606, decoding sections (source decoder) 507, 607.
[0011]
Here, in the image coding apparatus 500A (600A), the coding unit 501 (601) corresponds to the coding unit 101, and converts the input image signal (video signal: frame) F into a predetermined coding unit (1). Coding parameters e1, e2,..., Ex are obtained by performing predetermined coding processing such as motion compensation, discrete cosine transform of prediction error signals, quantization, etc. in units of blocks having pixels or more, for example, The configuration is as shown in FIG.
[0012]
In the encoding unit 501 (601), the difference (prediction error) between the input image signal F and the input image signal F and the output (motion compensation or prediction value between frames) variably delayed in the variable delay frame memory 710. ) The value calculated by the prediction error calculation unit 701 is adaptively selected by the control of the switch 703 of the switch unit 702 from the encoding control unit 504 (604), and the transform unit 705 performs orthogonal transform such as DCT. Then, the quantization unit 706 quantizes the coefficients (DCT coefficients and the like).
[0013]
At this time, encoding section 501 (601) performs local decoding for encoding the next encoding unit as described above with reference to FIG. That is, the quantized value (prediction error) q is decoded through the inverse quantization unit 707 and the inverse transform unit 708, and the decoded result is supplied to the predicted value (the one frame supplied via the switch 704 of the switch unit 702) by the addition unit 709. The result is added to the previous addition unit 709 and accumulated in the variable delay frame memory 710.
[0014]
Through the encoding process as described above, the prediction type p, the motion vector v, the quantization parameter qz, the quantization value q, the quantization step size qz, and the like are used as the encoding parameters e1, e2,. Obtained and output to multiplexing section 502 (602).
Next, the above-described multiplexing unit 502 (602) corresponds to the above-described multiplexing unit 102, and according to a predetermined syntax (transmission format), for example, as shown in FIG. .., ex including v, quantization parameter qz, quantization value q, quantization step size qz, etc. are converted into binary variable length codes, and a predetermined synchronization signal (a header to be described later) is converted. ) And the resolution of the input image and the like are added to create a coded bit string, thereby multiplexing the coding parameters e1, e2,.
[0015]
Specifically, as described above, the encoding parameters e1, e2,..., Ex are converted into binary bit strings, and then the encoding parameters ei of the adjacent blocks in the same frame in the line scan order (where i = In the encoding / decoding technique (1), a packet 800 called a video packet having a format as shown in FIG. 68, for example, is created by the multiplexing unit 502 (602).
[0016]
Here, a unique word 801 called “Resync-Marker” always exists at the boundary between the packets 800 (synchronization signal: a code that can always be found as a unique code even if a bit string by variable length coding is mixed). Even if an error occurs and the packet 800 is discarded, resynchronization can be performed by the unique word 801 of the next packet 800.
[0017]
The packet 800 includes, for example, information (macro block number) 802 indicating a position of a macro block (a collection of a plurality of blocks: a group of four blocks is called a macro block in standardization) and a quantization parameter (quant scale) 803, extension code (HEC) 804, information such as motion information and DC component (Motion & Header Information) 805, information (Motion Marker) 806 that separates this information 805 from the following information (Texture Information) ) 807 is appropriately mounted. Note that the order (output order) of information 801 to 807 other than the unique word 801 may be different from that shown in FIG.
[0018]
Next, the transmission buffer 503 (603) temporarily stores the encoded bit string obtained by the multiplexing unit 502 (602), and the encoding control unit 104 transmits the bit of the transmission path 300. When the rate is constant or the amount of transmission information (code amount) needs to be controlled, the remaining amount of the transmission buffer 503 (603) is used as a parameter for quantization in the encoding unit 501 (601). The roughness is controlled, and the multiplexing unit 502 (602) multiplexes the control results.
[0019]
On the other hand, in the image decoding apparatus 600A (600B), the reception buffer 505 (605) receives and holds the encoded bit string transmitted through the transmission path 300, and the multiplexing decoding unit 506 (606) In contrast to the multiplexing unit 502 (602) described above with reference to FIG. 65, it corresponds to the multiplexing decoding unit 201, and the type of prediction of the encoded bit string held in the reception buffer 505 (605) according to the syntax described above. p, motion vector v, quantization parameter qz, quantization value q, quantization step size qz, etc., encoding parameters e1, e2,..., ex (e1 ′, e2 ′,..., ex ′) It is to convert to.
[0020]
The decoding unit 507 (607) corresponds to the decoding unit 202, and the encoding parameters e1, e2,..., Ex (e1 ′, e2 ′,) obtained by the multiplexing decoding unit 506 (606). .., ex ′) to decode the image data and output a reproduced image signal.
In the image encoding / decoding system 1b configured as described above, the encoding unit 501 (601) encodes an input image signal by a predetermined encoding method such as predictive encoding, transform encoding, or vector quantization. As a result, the temporal and spatial redundancy of the image signal (image data) is reduced to greatly reduce the amount of image information (image compression), and then transmitted through the transmission path 300, and the opposite decoding unit 607 In (507), since the transmitted image data is restored and reproduced, the band of the transmission line 300 can be used effectively.
[0021]
Note that when image data is encoded and decoded in one apparatus (for example, a personal computer) (when image data is not transmitted), the encoded image data (encoding parameters e1, e2,. .., ex) is stored in a storage medium such as a hard disk, and the image data stored in the storage medium is read and decoded / reproduced as necessary. However, in this case as well, errors may be added to the encoding parameters e1, ..., ex through the storage medium.
[0022]
[Problems to be solved by the invention]
By the way, in the image encoding / decoding systems 1a and 1b as described above, a predetermined compression rate can be obtained by encoding. However, the higher the efficiency of the encoding method, the higher the efficiency is received from the transmission line 300 (or storage medium). The effect of errors is large. That is, since the image data is encoded using temporal and spatial redundancy of the image signal as described above, the encoded image data (encoding parameters e1, e2,. In contrast to ex), if an error occurs even partially, the error propagates temporally and spatially starting from the incorrect part, and a correct decoding result (reproduced image) cannot be obtained.
[0023]
Therefore, conventionally, as an error countermeasure, error correction by channel coding (error correction code) is performed, or when predictive coding is performed, error propagation is suppressed by reducing the prediction coefficient value. However, none of them are effective measures at present, and sufficient resistance against errors cannot be secured.
[0024]
The present invention was devised in view of such problems, and by passing information on blocks having the same coding information from the coding side to the decoding side, errors occur in the coding information of some blocks. Even so, the correct value can be obtained from the encoding information of other blocks, and the tolerance to errors is improved compared to the conventional one, minimizing the deterioration of the quality of the reproduced image, and the high quality reproduced image It aims to be able to obtain.
[0025]
[Means for Solving the Problems]
In order to achieve the above object, an image encoding / decoding system according to the present invention (Claim 1) includes an image encoding device for encoding image data, and image data encoded by the image encoding device. And an image decoding device that decodes and reproduces the image, and the image encoding device described above encodes image data in units of one block or more to obtain predetermined encoding information for the block. And a common information generation unit that generates common information indicating which block has the same encoded information among the encoded information about the plurality of blocks obtained by the encoding unit, and the above The image decoding device includes an error correction unit that corrects an error in the encoded information based on the common information generated by the common information generation unit of the image encoding device, and an error correction unit that corrects the error by the error correction unit. It is characterized in that it includes a decoding unit for reproducing the image data by decoding the encoded information.
[0026]
Here, in the present image encoding / decoding system, the above-described image decoding apparatus issues a re-transmission request for information necessary for re-error correction of encoded information that cannot be error-corrected by the above-described error correction unit. A re-transmission request unit is provided for the encoding device, and the image encoding device re-sends only information corresponding to the re-transmission request from the re-transmission request unit of the image decoding device to the image decoding device. A re-sending unit may be provided (claim 2).
[0027]
The image encoding / decoding method of the present invention (claim 3) includes an encoding step of encoding image data in units of one block or more and obtaining predetermined encoding information for the block. And a common information generating step for generating common information indicating which block has the same encoded information among the encoded information for the plurality of blocks obtained by the encoding step, and generating the common information An error correction step for correcting an error in the encoded information based on the common information generated in the step, and a decoding step for decoding the encoded information corrected in the error correction step and reproducing the image data It is characterized by that.
[0028]
Next, an image encoding device of the present invention (Claim 4) encodes image data in units of blocks composed of one or more pixels, thereby obtaining predetermined encoding information for the block; A common information generating unit that generates common information indicating which blocks have the same encoded information among the encoded information for a plurality of blocks obtained by the encoding unit, and generated by at least the common information generating unit And a sending unit that sends the common information sent to the image decoding apparatus.
[0029]
Here, the common information generation unit creates a histogram about the occurrence frequency of the encoded information in a plurality of blocks, and coarsely quantizes the encoded information whose occurrence frequency is a predetermined value or less in this histogram, or In the histogram, the coding information is corrected to the neighboring coding information, or the coding information is shared by modifying the coding information of the neighboring block to generate the common information. (Claims 5 to 7).
[0030]
Further, the sending unit may be configured to send a block having the same encoded information distributed in a plurality of packet data (claim 8), or send the common information a plurality of times. (Claim 9).
Further, when the image encoding apparatus receives a re-transmission request for information necessary for re-error correction of encoded information that cannot be corrected by the image decoding apparatus, only the information corresponding to the re-transmission request is sent to the image encoding apparatus. A re-sending unit for re-sending to the image decoding device may be provided.
[0031]
Next, the image decoding apparatus according to the present invention (claim 11) encodes image data in units of blocks composed of one or more pixels in the image encoding apparatus, thereby obtaining predetermined encoding information for the block. In addition, when common information representing which block has the same coding information among the coding information for a plurality of blocks is obtained, at least a receiving unit that receives the common information from the image coding device, An error correction unit that corrects an error in encoded information based on common information received by the reception unit, and a decoding unit that decodes the encoded information error-corrected by the error correction unit and reproduces image data are provided. It is characterized by having.
[0032]
Here, when it is known in advance that there is an error in the encoded information, the error correction unit described above selects another block having the same encoded information as the erroneous encoded information based on the common information. It may be configured to correct the error of the encoded information by specifying and outputting the encoded information of the block as the encoded information of the erroneous block (claim 12).
[0033]
In addition, when it is not known in advance that there is an error in the encoded information, the error correction unit specifies a plurality of blocks that should have the same encoded information based on the common information, and An error may be detected by comparing each piece of encoded information, and the error may be corrected (claim 13).
Further, the error correction unit determines the reliability of the common information and / or the block identification information, and estimates the correct information from information other than the information with low reliability for information with low reliability. It may be configured (claim 14).
[0034]
In addition, the present image decoding apparatus includes a re-transmission request unit that makes a re-transmission request to the image encoding apparatus for information necessary for re-error correction of encoded information that cannot be corrected by the error correction unit. (Claim 15).
Next, an image encoding method of the present invention (Claim 16) encodes image data in units of one block or more, thereby obtaining predetermined encoding information for the block, and A common information generating step for generating common information indicating which blocks have the same encoded information among the encoded information for a plurality of blocks obtained by this encoding step, and at least generated by this common information generating step And a sending step for sending the common information sent to the decoding side.
[0035]
According to the image decoding method of the present invention (claim 17), predetermined encoding information is obtained for a block by encoding the image data in units of one or more pixels, and codes for a plurality of blocks are obtained. When common information indicating which blocks of the encoded information have the same encoded information is obtained on the encoding side, at least a reception step for receiving the common information from the encoding side, and a reception step An error correction step for correcting an error in the encoded information based on the received common information, and a decoding step for decoding the encoded information corrected by the error correction step and reproducing the image data. It is a feature.
[0036]
Furthermore, an image encoding / decoding device according to the present invention (claim 18) is encoded by an image encoding device that encodes image data, and an image encoding / decoding device provided opposite to the image encoding device. In an apparatus comprising an image decoding apparatus that decodes and reproduces the converted image data, the image encoding apparatus encodes the image data in units of blocks each composed of one or more pixels. A common unit for generating common information indicating which block has the same encoded information among the encoded information for a plurality of blocks obtained by the encoding unit And an information generation unit, and the image decoding device is generated by the opposite image encoding / decoding device, and the image data is encoded by the image encoding / decoding device. An error correction unit for correcting an error of the encoded information based on common information indicating which blocks have the same encoded information among the encoded information of the plurality of blocks, and error correction by the error correction unit And a decoding unit that decodes the encoded information and reproduces the image data.
[0037]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
(A) Description of an embodiment of the present invention
FIG. 1 is a block diagram showing the configuration of an image communication system (image encoding / decoding system) as an embodiment of the present invention. The system 1 shown in FIG. 1 also has an image encoding device 2 for encoding image data. And an image decoding device 3 that decodes and reproduces the image data encoded by the image encoding device 2, and the image encoding device 2 and the image decoding device 3 communicate with each other by image communication. Are connected to each other via a desired transmission path 4 (which may be wired or wireless).
[0038]
Further, as shown in FIG. 1, the above-described image encoding device 2 is configured to include an encoding unit 21, a commoning unit 22, a multiplexing unit 23, and the like, when attention is paid to the function of the main part. The image decoding device 3 is configured to include a multiplexing decoding unit 31, an error detection / correction unit 32, a decoding unit 33, and the like, focusing on the function of the main part.
[0039]
Here, first, in the image encoding device 2, the encoding unit 21 divides the input image signal (image data) F into predetermined blocks (a collection of one or more pixels: rectangular or non-rectangular), By encoding the input image signal F in a block unit by a predetermined encoding method (algorithm) such as predictive encoding, transform encoding, or vector quantization, a quantization parameter (quantization step size) or a quantization value is obtained. , DCT coefficients, textures, colors, in the case of moving images, predetermined encoding parameters e1, e2,..., Ex (x is a natural number) such as a motion compensation vector are calculated and the encoding parameters e1, e2,. .., ex is output as encoded information. Also in this case, the encoding parameters e1,..., Ex are not limited to scalar quantities, but are, for example, two-dimensional motion vectors or transform coefficients in the case of block transform coding (the number of dimensions is one or more dimensions). )
[0040]
Further, the common unit (common information generation unit) 22 basically sets the coding parameters e1, e2,..., Ex obtained by the coding unit 21 to each coding parameter in a predetermined common unit. Common to e1, e2, ..., ex (may not be common), output encoding parameters ek1, ek2, ..., ekx as a result of the commonization, and process of this commonization The common information (common information) ecom indicating which blocks have common parameters (same coding information) is calculated (generated) and output.
[0041]
Here, commonality refers to, for example, certain coding parameters ek1, ek2,..., Ekx over a plurality of blocks. To recalculate. The above-mentioned common unit is, for example, a partial block (a neighboring block or a non-neighboring block of a plurality of frames) of each of a plurality of frames, as shown by a shaded portion in each of FIGS. 2 (a) to 2 (d). Alternatively, only one frame, a plurality of frames, or only a part of one frame may be used.
[0042]
Further, for example, the encoding parameters ek1, ek2,..., Ekx after commonization must be calculated by the encoding unit 21 using the common encoding parameter ek1. In such a case (when the encoding parameters ek1, ek2,..., Ekx are shared by the sharing unit 22), as shown by a broken line in FIG. .
[0043]
Further, the multiplexing unit 23 multiplexes the commonized encoding parameters ek1, ek2,..., Ekx obtained by the commoning unit 22 and the common information ecom. Specifically, as will be described later, the encoding parameters ek1, ek2,..., Ekx and the common information ecom are respectively converted into variable length codes (or fixed length codes) according to a predetermined syntax (transmission format). By performing the conversion, an encoded bit string is generated. The obtained encoded bit string is packetized as will be described later and sent to the image decoding device 3 via the transmission path 4. That is, the multiplexing unit 23 functions as a sending unit that sends at least the common information ecom generated by the commoning unit 22 to the image decoding device 3.
[0044]
On the other hand, in the image decoding device 3, the multiplex decoding unit 31 transmits the encoded bit string (encoding parameters ek1, ek2,..., Ekx + common information ecom) transmitted from the image encoding device 2 through the transmission path 4. ), And the encoded bit string received from the transmission path 4 is multiplexed and decoded by subjecting the encoded bit string to a process reverse to the multiplexing process by the multiplexing unit 23 described above. The previous values (encoding parameters ek1 ′, ek2 ′,..., Ekx ′ and common information ecom ′) are obtained. That is, the multiplex decoding unit 31 functions as a receiving unit that receives at least the common information ecom obtained by the commoning unit 22 of the image coding device 2 from the image coding device 2.
[0045]
Here, also in this case, since the encoding parameters ek1 ′, ek2 ′,..., Ekx ′ and the common information ecom ′ have errors added by the transmission path 4, they are not necessarily the original encoding parameters ek1. , ek2,..., ekx and the common information ecom, but in this embodiment, at least the common information ecom is received by the error detection / correction unit as described later. Can be restored (estimated).
[0046]
Further, the error detection / correction unit 32 restores (estimates) the original common information ecom from the common information ecom ′ obtained by the multiplex decoding unit 32 and performs error correction using a normal error correction code. Without correcting the errors of the above encoding parameters ek1 ′, ek2 ′, ..., ekx ′ based on the common information ecom, the encoded parameters ek1 ″, ek2 ″,. ., ekx ″. The obtained encoding parameters ek1 ″, ek2 ″, ..., ekx ″ are the encoding parameters ek1 ′, ek2 ′, ..., ekx before error correction. Although the number of errors is smaller than ′, it may not always completely match the original common encoding parameters ek1, ek2,.
[0047]
Then, the decoding unit 33 reproduces the image data F by decoding the encoding parameters ek1 ″, ek2 ″,..., Ekx ″ corrected by the error detection / correction unit 32 (reproduced image signal Fk ′). ).
Hereinafter, out of each of the above-described units 21 to 23 and 31 to 33, paying attention to the common unit 22, the multiplexing unit 23, and the error detection / correction unit 32 which are the main parts of the present embodiment, each operation is classified by item. Detailed description.
[0048]
(1) Explanation of operation of the common unit 22
The sharing unit 22 basically operates according to a flowchart (algorithm) as shown in FIG. 3 to share the encoding parameters e1, e2,. Generate ecom.
That is, first, all blocks or a part (divided part) of blocks in one frame or a plurality of frames, which are common units as described above with reference to FIG. Such block collection may be performed by a known segmentation method.
[0049]
The encoding parameter ei (where i = 1 to x) of each block (block number n: n is a natural number) obtained in such a unit may be expressed as e (n) (simply “e”). ), If the encoding parameter after sharing is expressed as ek (n) (may be simply expressed as “ek”), and the sharing information is expressed as ecom, the possible value of the encoding parameter e before sharing is m There are types (m is a natural number satisfying m ≦ n) (step A1).
[0050]
Next, the communicator 22 performs communalization on the encoding parameter e, and sets m types of values to u types (where u is a natural number satisfying u <m) (step A2). That is, the sharing unit 22 reduces the amount of the encoding parameter e to be transmitted by sharing the encoding parameter e, and increases the number of blocks having the same information (encoding parameter e). There are always a plurality of blocks with the.
[0051]
As a specific common method at this time, for example,
(1) A method in which a statistic about the frequency of occurrence (appearance) of an encoding parameter e in a plurality of blocks is examined to perform sharing
(2) A common clustering method
(3) A method for checking whether or not the encoding parameter e of a block around a certain block is similar and sharing the similar blocks
Etc.
[0052]
Then, the commoning unit 22 calculates the encoding parameter ek (n) obtained by the above-mentioned commonization (hereinafter sometimes simply referred to as “encoding parameter ek”) and the above-mentioned commonization process. The (generated) common information ecom is output (step A3: common information generating step). For example, as a result of sharing, the value of a certain encoding parameter ek of each block is in a state as shown in FIG. 4A, and the block number (identification information) of each block is as shown in FIG. If it is assigned as shown in b), the commoning unit 22 outputs the respective encoding parameters ek shown in FIG. 4A, and also shows the common information ecom as shown in FIG. 5A. A table (ecomtable) 5 is generated in which the encoding parameter value is associated with the block number of the block having the encoding parameter value (that is, indicating which block has the same encoding parameter value). And output.
[0053]
Next, the common methods (1) to (3) will be described in more detail.
(1.1) Explanation of common method (1)
The above common method (1) is realized by the common unit 22 operating according to the flowchart (algorithm) shown in FIG. 6, for example. That is, the encoding parameter of the n blocks to be shared is e, the encoding parameter (output) after sharing is ek, the sharing information is ecom, and the occurrence frequency histogram for the encoding parameter e is H ( m) When the threshold for the number of occurrence frequencies in the occurrence frequency histogram H (m) is set to th1 (step A4), the common unit 22 first determines the occurrence frequency of the value of the encoding parameter e (n). Statistics are taken and a histogram H (m) is created (step A5).
[0054]
For example, if the value on one screen of a quantization parameter, which is a kind of encoding parameter e, is in a state as shown in FIG. The quantization parameter value of the block is examined to determine the occurrence frequency thereof, and an occurrence frequency histogram H (m) (also simply referred to as “histogram H”) as shown in FIG.
[0055]
Next, the sharing unit 22 checks the encoding parameter e for which the frequency of the histogram H (m) is equal to or less than the threshold value th1 as the number of blocks is insufficient for sharing (for example, when the threshold value th1 = 3). In FIG. 12, the quantization parameter value “4” having an occurrence frequency of “2” is checked.) On the other hand, a common encoding parameter e is generated as common information ecom (described in Table 5: step A6).
[0056]
Then, the common unit 22 determines whether or not the checked encoding parameter e exists (step A7), and if not, outputs the encoding parameter e and the common information ecom (table 5). (From step A7 NO route to step A8) On the other hand, if it exists, only the checked encoding parameter e is roughly quantized to the encoding parameter ek (step A9), and the histogram H is created again and checked ( Until the encoding parameter e (the occurrence frequency is equal to or less than the threshold th1) no longer exists (until NO is determined in step A7), the processing from step A6 onward is repeated.
[0057]
Here, the quantization in step A9 is performed, for example, as shown in FIG. That is, if the quantization parameter is q, the coding parameter before quantization (encoding parameter whose occurrence frequency is equal to or less than the threshold th1) is e, and the coding parameter after quantization is ek (step A9-1), the common First, the quantization unit 22 divides the encoding parameter e by the quantization parameter q and adds 0.5 to a rounded integer, and the obtained value (temp) is multiplied by the quantization parameter q again. This is set as the encoding parameter ek (step A9-2).
[0058]
For example, assuming that the value of the parameter q is “2” and the encoding parameter value “4” whose occurrence frequency is equal to or less than the threshold th1 as described above with reference to FIG. 12 is considered, 4/2 + 0.5 = 2. Since it is 5 and rounded off to “3”, the encoding parameter ek is 3 × 2 = 6. In this manner, in the histogram H, the encoding parameter e having a frequency of occurrence below the threshold th1 is roughly quantized so that the encoding parameter e below the threshold th1 is encoded more frequently than the other threshold th1. The parameter e is standardized.
[0059]
That is, the commoning unit 22 creates a histogram H regarding the occurrence frequency of the encoding parameter e in a plurality of blocks, and coarsely quantizes the encoding parameter e whose occurrence frequency is equal to or less than the predetermined value th1 in the histogram H. Thus, the encoding parameter e is shared and the common information ecom is generated. Therefore, it is possible to share the encoding parameter e with a very simple calculation.
[0060]
The above example is a case where the encoding parameter e is a scalar quantity, but even if the encoding quantity e is a vector quantity of two or more dimensions, it is possible to share it by executing similar quantization for each element of the vector. .
By the way, instead of the above step A9, for example, the step A9 ′ shown in FIG. 7 or the step A9 ″ shown in FIG.
[0061]
That is, in FIG. 7, the checked encoding parameter e is corrected to a value near the histogram. For example, as shown in FIG. 10, if a histogram H, a threshold th2, an encoding parameter (value) e before quantization, and a quantized encoding parameter (value) ek (step A9'-1) are common, The encoding unit 22 refers to the histogram H, and a value k having a frequency equal to or greater than the threshold th2 in the vicinity of the encoding parameter e. ₁ (However, k ₁ = 1 to m), the value closest to the encoding parameter e (e−k ₁ Of the smallest absolute value) ₁ Is the encoding parameter ek (step A9'-2).
[0062]
That is, the common unit 22 in this case creates a histogram H for the frequency of occurrence of the encoding parameter e in a plurality of blocks, and in the histogram H, the encoding parameter e having an occurrence frequency equal to or less than a predetermined value th1 By correcting to the neighboring encoding parameter e, the encoding parameter e is made common to generate common information ecom. Therefore, in this case, the encoding parameter e can be shared more reliably than in the case of coarse quantization as described above.
[0063]
On the other hand, in FIG. 8, the checked encoding parameter e is changed to the value of the neighboring block. For example, encoding parameters er (u) (may be simply expressed as “er”) of u blocks existing around in time or space, encoding parameters (values) before quantization e, Assuming that the quantized encoding parameter (value) ek is (step A9 ″ −1), the communicator 22 sets the value [er (k ₂ ) -E having the smallest absolute value: k ₂ = 1 to u] encoding parameter er (k ₂ ) The value is set as an encoding parameter ek (step A9 ″ -2).
[0064]
That is, the common unit 22 in this case creates a histogram H regarding the occurrence frequency of the encoding parameter e in a plurality of blocks, and the encoding parameter having the occurrence frequency equal to or less than the predetermined value th1 in the histogram H By changing to the encoding parameter er, the encoding parameter e is shared, and common information ecom is generated. Therefore, also in this case, the encoding parameter e can be surely shared.
[0065]
The common unit 22 may operate according to the flowchart (algorithm: steps B1 to B3) shown in FIG. 13 instead of the flowchart (algorithm) shown in FIG. That is, if the encoding parameter of the unit n block to be shared is e, the encoding parameter histogram is H, the encoding parameter (output) after the sharing is ek, and the sharing information is ecom (step B1), The sharing unit 22 first creates a histogram H of the encoding parameter e (step B2).
[0066]
Next, the sharing unit 22 checks the encoding parameter e for which the frequency of the histogram H is equal to or less than the threshold th1 as the number of blocks is insufficient for sharing, and uses the encoding parameter e and the block number as the sharing information. Write to ecom, and output the encoding parameter e together with the common information ecom as the encoding parameter ek (step B3). Note that the encoding parameter e exceeding the threshold th1 is also written in the common information ecom as described above. The common information ecom in this case is as shown in FIG. 14A, for example, and the encoding parameter e with insufficient blocks (one block number) is also written in the common information ecom (table 5). Yes.
[0067]
That is, the commoning unit 22 in this case does not perform the above-described commonization even if there is a coding parameter e whose frequency of the histogram H is equal to or less than the threshold th1, and sets the coding parameter e that cannot be shared. The information about the block is written in the common information ecom and output. Therefore, in this case, it is only necessary to add the common unit 22 to the existing image encoding device 2 (encoding unit 21), which contributes to improvement in versatility.
[0068]
In this case, as will be described later, a block having the encoding parameter e that could not be shared is transmitted from the multiplexing unit 23 a plurality of times.
(1.2) Explanation of sharing method (2)
The common method (2) is realized by the common unit 22 operating according to the flowchart (algorithm: steps C1 to C4) shown in FIG. That is, also in this case, the encoding parameter of the unit n block to be shared is e, the encoding parameter (output) after the sharing is ek, the sharing information is ecom, and the cluster result of the encoding parameter is C (k), where the threshold for the number of encoding parameters e in a certain cluster is th3, first, the commoning unit 22 applies k average clustering method (k number of blocks) to n blocks of encoding parameters e. Clustering, that is, processing for arranging individuals on average: clustering is performed by a known method such as “Image Analysis Handbook” p641-p652). As a result, the encoding parameter e that was originally present in m types is k _Three Type (however, k _Three <M) (Step C1).
[0069]
Next, the sharing unit 22 obtains a cluster result C (k) indicating which cluster each encoding parameter e belongs to. _Three ) Is calculated (step C2), and the cluster result C (k _Three ), A cluster whose number of encoding parameters e is smaller than the threshold th3 is merged with another cluster having the nearest cluster centroid (step C3).
[0070]
The common unit 22 generates and outputs information such as the block number in each cluster as common information ecom, and codes the centroid (or representative value) of the cluster C to which the encoding parameter e belongs to Is output as the conversion parameter ek (step C4).
As described above, if the sharing is performed using the known clustering, the calculation of the clustering itself may take time, but even if the encoding parameter e is a vector value of two or more dimensions, the distortion due to the sharing is caused. There is an advantage that (deterioration) is smaller than the above-described common method (1) and the common method (3) described later.
[0071]
(1.3) Explanation of sharing method (3)
The above common method (3) is realized by the common unit 22 operating according to the flowchart (algorithm: steps D1 to D4) shown in FIG. 16, for example. That is, also in this case, the encoding parameter of the unit n block to be shared is e, the shared encoding parameter (output) is ek, the sharing information is ecom, and the block division number is u1, If u2,..., ui (u1 to ui are natural numbers) and the threshold value for the difference between the encoding parameters e is th4 (step D1), the commoning unit 22 first n blocks to be commonized. Is further divided into u1 (blocks) spatially and temporally, and if the difference between the coding parameters (values) e of the respective blocks is within the threshold th4, the average value is shared. (Step D2).
[0072]
If the common block 22 cannot be shared, the common block 22 further divides the divided block into u2 pieces, and performs the same processing as step D1 described above (step D3). Then, the sharing unit 22 performs the block division / commonization (averaging) as described above with the number of divisions u3,..., Ui until the block cannot be divided, and when the block cannot be divided, For the remaining blocks, the average value is compulsorily shared and shared, or not shared, and the shared information ecom and the encoded parameter ek after sharing are output.
[0073]
For example, assuming that the encoding parameter e of a block to be shared is in a state as shown in FIG. 17, the commoning unit 22 is configured as shown in FIGS. 18 (a) to 18 (d), for example. Then, the block is divided (subdivided), and the encoding parameter e is shared for each divided block, and for each sharing, the current encoding parameter e and the block number to which the encoding parameter e belongs [ “1-1”, “2-1” to “2-4”, “3-1” to “3-16”, “4-1” to “1-1” shown in FIG. "4-64"] is written in the common information ecom (table 5) as shown in FIG.
[0074]
That is, the common unit 22 in this case assigns block numbers according to the coarseness of division when collecting the encoding parameters e for temporally or spatially neighboring blocks [FIG. 18 (a) to FIG. 18 (d), and in accordance with this, if all neighboring blocks have the same encoding parameter (value) e, the block number of the coarser division unit is attached as much as possible to create the common information ecom. is there. As a result, the information amount of the common information ecom itself is greatly reduced, and the capacity (bandwidth) required for transmission of the common information ecom is greatly reduced.
[0075]
In this case, since only neighboring blocks are referred to, the processing time of the common processing is shortened.
(2) Explanation of operation of the multiplexing unit 23
Next, the operation of the multiplexing unit 23 in the image encoding device 2 will be described in detail. The multiplexing unit 23 basically operates according to a flowchart (algorithm: steps E1 to E3) as shown in FIG. 20, for example. That is, when the multiplexing unit 23 receives the encoding parameter ek and the sharing information ecom of the unit n blocks that are shared by the sharing unit 22 (step E1), the multiplexing unit 23 receives the coding parameter ek and the sharing information. Each information ecom is multiplexed (converted into a bit string according to a predetermined syntax) (step E2), and the multiplexed common information ecom is added to the encoding parameter ek at least once and output (step E3: sending step). .
[0076]
Specifically, for example, as shown in FIG. 21, the common information is ecom [ecom (q, x)], the common information that is variable-length (or fixed-length) encoded for multiplexing is ecommux, and encoded. It is assumed that the parameter is ek (n) and the encoding parameter obtained by variable length (or fixed length) encoding for multiplexing is ekmux (m) (may be simply referred to as “ekmux”) [in this case , N is the number of blocks, m is the number of packets, q is the number of possible encoding parameters ek, x is the data of the block number taking the value q (that is, the above ecom (q, x) is the value of the encoding parameter ek q and the block number order x as an element): Step E4], the multiplexing unit 23 firstly describes the common information ecomtable (q, x) (simply simply “ecomtable”). Emu) variable length (or fixed length) encoding, common information ecomm ux is obtained (step E5).
[0077]
For example, when the common information ecom before encoding (multiplexing) is in a state as shown in FIG. 5A, the multiplexing unit 23, as shown in FIG. The code (bit string) of the block number of ek is sequentially output while inserting a predetermined delimiter code 6 for each group of the same encoding parameter ek, thereby obtaining multiplexed common information ecommux.
[0078]
At this time, for example, as shown in FIG. 22, the multiplexing unit 23 may obtain the common information ecommux in which not only the block number code but also the encoding parameter (value) ek is inserted.
In addition, the multiplexing unit 23 sets m blocks (provided that m ≧ 2) so that blocks having the same value (encoding parameter ek) as much as possible for a predetermined number m packets are mounted on other packets. (Natural number) packets are distributed and mounted (that is, one packet is mounted with blocks having different encoding parameters ek as much as possible).
[0079]
For example, assuming that the common information ecom is in the state shown in FIG. 5A and the number of packets m to be generated is m = 3 as described above, the multiplexing unit 23, for example, , A block having the same encoding parameter ek is distributed and mounted in three packets (packet numbers “1” to “3”). At this time, since the order of the blocks varies, the block number is also mounted on the packet (the mounting position may be an arbitrary position). However, a block having the same encoding parameter (value) ek is k _Four If the number is less than the number, the multiplexing unit 23 describes the same block number redundantly in a plurality of packets (step E6). As a result, the probability that blocks having the same encoding parameter ek will be missed together due to errors or the like is greatly reduced.
[0080]
Here, the processing of step E6 is realized by, for example, executing the flowchart shown in FIG. 24 (steps E61 to E71. In FIG. 24, u is the sign of the common information ecom (q, x). Counter value for searching for a parameter q, i is a counter value for searching a block number x, and j is a counter value for a packet number.
[0081]
First, the multiplexing unit 23 initializes the above counter values u, i, j (u = 1, i = 1, j = 1) (step E61), and the first packet (ekmux (1)). In addition, the information indicated by the common information ecom (1,1), that is, the block number “1” of the first block of the encoding parameter value “1” is added (mounted) (step E62). The multiplexing unit 23 increments the counter values i and j by one (step E63), and then determines whether or not the counter value j> the number of packets m (step E64).
[0082]
As a result, if the counter value j> the number of packets m (if it is determined YES in step E64), the multiplexing unit 23 initializes the counter value j (j = 1) and sets the information mounting target. If the packet number is the first packet (ekmux (1)) (step E65), but the counter value j ≦ the number of packets m, the information mounting target packet number is used as it is as the next (jth) packet (ekmux). (J)) (NO route of step E64).
[0083]
Then, the multiplexing unit 23 determines whether or not the encoding parameter ek having the value q (= u) no longer exists (step E65), and if it exists, the next block number of the encoding parameter value u is packetized. (Ekmux (j)) is added (from the NO route of step E65 to step E62). That is, the multiplexing unit 23 sequentially searches for the block number that becomes the same encoding parameter u of the common information ecom until there is no block number of the same encoding parameter value u, and each block number is different. It is added to the packet (ekmux (j)).
[0084]
When the encoding parameter ek having the value q (= u) no longer exists, the multiplexing unit 23 next selects the counter value i ≦ k. _Four (That is, whether there are fewer than k blocks having the same encoding parameter value u) (step E66). If not (if i> k), the counter value u is set to 1. (The search target is changed to the block number of the next encoding parameter value: from the NO route of step E67 to step E69), and the counter value i is initialized (i = 1) (step E70).
[0085]
At this time, if the counter value u> the encoding parameter value q (YES in step E71), the multiplexing unit 23 determines that all information (block) of the common information ecom (q, x) Has been mounted on the packet, the process ends. If the counter value u ≦ the encoding parameter value q (if NO is determined in step E71), there are still unmounted blocks. Therefore, the processing from step E62 is repeated until all the blocks are mounted.
[0086]
Incidentally, in step E67 described above, the counter value i ≦ k _Four That is, a block having the same coding parameter value u is k _Four When the number is less than the number, the multiplexing unit 23 refers to the packet already added with the information indicated by the common information ecom (u, 1), that is, the first block number of the same encoding parameter value u already added. Duplicate addition to different packets (from YES route of step E67 to step E68).
[0087]
Through the above processing, for example, k _Four = 2 and the common information ecom is in a state as shown in FIG. 14A [blocks with encoding parameters ek = 4, 6 (block numbers = 7, 34) as shown in the shaded portion) When there is only one each], as shown in FIG. 14B, the multiplexing unit 23 mounts the block numbers of the same encoding parameter ek on different packets in order, as shown in FIG. 14B. As shown by the shaded portion in FIG. 1, only one block number 7 and 34 is mounted in different packets.
[0088]
Next, when the multiplexing unit 23 creates the multiplexed encoding parameter ekmux in this way, as shown in FIG. 21, the multiplexing unit 23 adds the common information ecommux to the encoding parameter ekmux to create an encoded bit string. (Step E7). For example, the multiplexing unit 23 uses the encoding parameter ek [= ek1 (n), ek2 (n),..., Ekx (n)] to generate a packet (for example, the video packet 800 described above with reference to FIG. 68). k are created, and as shown in FIG. 25, the common information ecommux is added to the head thereof.
[0089]
The addition position of the common information ecommux may be in the middle (boundary) of each packet or at the end if possible. Further, it may be transmitted a plurality of times by adding a plurality. If the common information ecom is transmitted a plurality of times in this way, the probability of occurrence of a situation in which the image decoding device 3 cannot normally receive the common information ecom (ecom ′) is greatly reduced.
[0090]
Further, when a plurality of pieces of common information ecommux are obtained, as shown in FIG. 26, for example, information indicating which common information ecommux is based on which information is included in the packet (in the common information ecommux) A unique number L or v) is mounted somewhere in each packet, or, as shown in FIG. 27, which packet contains information based on the common information ecommux in the common information ecommux. By mounting information (packet number) indicating whether there is, it may be possible to identify which common information ecommux should be used for which packet on the decoding side (image decoding device 3).
[0091]
(3) Explanation of operation of the multiplex decoding unit 31
Next, the operation of the multiplex decoding unit 31 in the image decoding device 3 will be described in detail. The multiplexing / decoding unit 31 basically performs a process opposite to the process of the multiplexing unit 31 of the image encoding device 2. That is, for example, as shown in FIG. 28, when an encoded bit string (encoding parameter ekmux and common information ecommux) is received from the image encoding device 2 through the transmission path 4 (step F1: reception step), the encoded bit string is The values (encoding parameter ek ′ and common information ecom ′) before encoding (variable length or fixed length encoding) in the image encoding device 2 are restored (step F2).
[0092]
Note that the encoding parameter ek ′ and the common information ecom ′ after the multiplex decoding are each added with an error by being transmitted through the transmission path 4 as described above, and are not necessarily the original encoding parameters. This indicates that ekmux and common information ecom do not match.
(4) Explanation of operation of error detection / correction unit 32
Next, the operation of the multiplex decoding unit 32 in the image decoding device 3 will be described in detail. The multiplex decoding unit 32 basically operates according to a flowchart (algorim: steps G1 to G7) as shown in FIG. 29, for example. That is, when the error detection / correction unit 32 receives the encoding parameter ek ′ and the common information ecom ′ after the multiplexing decoding from the multiplexing decoding unit 31 (step G1), first, the value of the common information ecom ′ is determined. It is determined whether or not it can be trusted (step G2).
[0093]
As a result, if it is reliable (if YES is determined in step G3), the error detection / correction unit 32 proceeds to the next process, and if it is not reliable (if NO is determined in step G3), it is reliable. The value is estimated and the correct common information ecom 'is restored (step G4), and the process proceeds to the next process. The details of the determination method and estimation / restoration method of the reliability of the common information ecom ′ will be described later.
[0094]
After obtaining the reliable common information ecom ′ in this way, the error detection / correction unit 32 next determines whether or not it is known in advance that there is an error in the encoding parameter ek ′ (step). G5). The presence / absence of an error is detected, for example, by detecting a normal error correction code for a coded bit string in the preceding stage of the multiplex decoding unit 31 or an undecodable bit string at the time of multiplex decoding in the multiplex decoding unit 31. Is done.
[0095]
As a result of the above determination, when it is known in advance that the encoding parameter ek ′ has an error, the error detection / correction unit 32 encodes another error-free block having the same value in the common information ecom ′. Error correction is performed by referring to the parameter ek ′ and replacing the erroneously encoded parameter ek ′ with the error-free encoding parameter ek ′ (from the YES route of step G5 to step G6: error correction step). .
[0096]
For example, as shown in FIG. 30, it is known that there is an error in the coding parameter ek ′ (k1) of a certain block by the error correction code, and the coding parameter ek of another block having the same value by the error correction code. When no error is detected in ′ (k2) (step G61), the error detection / correction unit 32 sets ek ′ (k1) = ek ′ (k2), thereby setting the encoding parameter ek ′ (k1). The error is corrected (step G62).
[0097]
That is, when it is known in advance that the encoding parameter ek ′ has an error, the error detection / correction unit 32 described above has the same encoding parameter as the erroneous encoding parameter ek ′ based on the common information ecom ′. The error of the coding parameter ek ′ is corrected by specifying another block having ek ′ and outputting the coding parameter of the block as the coding parameter ek ′ of the erroneous block. Therefore, reliable error correction can be performed with extremely simple processing.
[0098]
On the other hand, when it is not known in advance that there is an error in the encoding parameter ek ′, the error detection / correction unit 32 uses the common information ecom ′ as shown in FIG. 29 to determine a plurality of blocks that should have the same value. The encoding parameters (values) ek ′ (k1),..., Ek ′ (kn) are compared (from the NO route of step G5 to step G7), and the correct values ek ″ (k1),. kn) is obtained and output (step G7: error correction step).
[0099]
For example, if the error is not known even by the error correction code, as shown in FIG. 31, the error detection / correction unit 32 uses the encoding parameters ek ′ of a plurality of blocks that should have the same value for which no error is detected by the error correction code. (K1),... ek ′ (kn) is referred to (step G71), and the most frequently occurring value (that is, the value obtained by majority vote) is set to the correct value ek ″ (k1),..., ek ″ (kn) (step). G72).
[0100]
That is, the error detection / correction unit 32 specifies a plurality of blocks that should have the same encoding parameter ek ′ based on the common information ecom ′ when it is not known in advance that there is an error in the encoding parameter ek ′. An error is detected by comparing the encoding parameters ek ′ of the plurality of blocks, and the error is corrected. Therefore, even when it is not known in advance that there is an error in the encoding / encoding parameter ek ′, it is possible to cope with it reliably.
[0101]
The encoding parameters ek ″ (k1),..., Ek ″ (kn) corrected in error as described above are respectively input to the decoding unit 33, and the decoding unit 33 performs a predetermined decoding process (decoding step). ), A reproduced image signal Fk ′ is obtained.
Note that the error correction process as described above can also be applied to the common information ecom ′ when the common information ecom ′ is transmitted a plurality of times. That is, in the above processing, if the encoding parameter ek ′ is replaced with the common information ecom ′, the common common information ecom ′ is known to be in error, or the common common information ecom ′ is correct. Information (ecom ′) can be obtained (restored).
[0102]
(4.1) Description of Method for Determining Reliability and Estimation / Restoration Method for Common Information ecom 'First, a method for determining the reliability of common information ecom' (a specific example of step G3 in FIG. 29) will be described. For example, as shown in FIG. 32, the error detection / correction unit 32 sets the received parameters (common information ecom ′) to param ′ (k1),... Param ′ (kn), and received parameters param ′ (k1). , ..., param ′ (kn), where tha is the threshold for the frequency of occurrence (step H1), parameters param ′ (k1),. ) Is examined (step H2).
[0103]
As a result, the occurrence frequency HA of the value A of the parameter param ′ (kr) (where r = 1 to n) having the highest occurrence frequency and the value B of the parameter param ′ (kr) having the next highest occurrence frequency are generated. If the difference from the frequency HA is within the threshold tha, the value A is a value having a high occurrence frequency, but the difference from the value B having the next highest occurrence frequency is small (for example, the occurrence frequency HA is prominent when the reliability is high). The received parameters param ′ (k1),... Param ′ (kn) (= common information ecom ′) are determined to have low reliability (step H3).
[0104]
Next, a method for estimating and restoring the common information ecom ′ (a specific example of step G4 in FIG. 29) will be described. If the error detection / correction unit 32 determines that the received common information ecom is unreliable as a result of the above reliability determination, the error detection / correction unit 32 determines the information of another reliable part of the common information ecom ′ or the encoding parameter ek ′, Alternatively, the value of the unreliable part is estimated by referring to the block number information block (n) (may be simply expressed as “block”) or both.
Specifically, for example, as shown in FIG. 33, the common information is eco ', the encoding parameter is ek' (n), and it corresponds to the encoding parameter 'ek (n) included in the bit string such as the video packet. Assuming that the block number information is block (n) (step J1), the error detection / correction unit 32 first determines that the unreliable part of the common information ecom 'is the grouped block number information block (n). It is checked whether it is block number information block (n) of a certain group (step J2).
[0105]
As a result, when the unreliable part is only the block number information of a certain group of the common information ecom '(when it is determined YES in Step J2), the error detection / correction unit 32 is reliable common The block number information block (n) of the unreliable part is estimated and restored by assuming that the number information other than the block number information block (n) of the conversion information ecom ′ is the block number information block (n) of the unreliable part (Step J3).
[0106]
For example, as shown in FIG. 34, the block number information block (n) of a certain group among all the block number information block (n) to be included in the common information ecom 'is low and missing. 34 (indicated by “x” in FIG. 34), all block number information block (n) and block number information block (n) included in another group of the common information ecom ′ (“ The remaining block number information block (n) (block numbers “3” to “5”, “14” in FIG. 34) is missing when the correspondence with “O” is checked. It can be estimated that the block number information is block (n).
[0107]
On the other hand, when the unreliable part of the common information ecom ′ covers a plurality of groups (when NO is determined in step J2), the error detection / correction unit 32 determines the block number (information) block of the reliable common information ecom ′. The block number block having the same encoding parameter ek ′ is searched for by referring to the encoding parameter ek ′ corresponding to, and the block number block is estimated as an unreliable block number (step J4).
[0108]
For example, as shown in FIG. 35, when the block number block is missing in a plurality of groups of the sharing information ecom ′ (indicated by “x” in FIG. 35), the error detection / correction unit 32 can perform reliable sharing. By examining the correspondence between the block number block of the information ecom ′ and the encoding parameter ek ′ (see the solid line in FIG. 35), it is specified which encoding parameter ek ′ belongs to which group (for example, the value “ It can be seen that all the encoding parameters ek ′ of “1” belong to the group “1” and all the encoding parameters ek ′ of the value “5” belong to the group “2”), and the corresponding block number block (in FIG. Estimate the block number block of the missing part (see circled numbers).
[0109]
Such an estimation method may also be performed when the unreliable part of the common information ecom 'is only one group (see the broken line route in FIG. 33). Further, the reliability determination described above may be performed using the block number (information) block instead of the common information ecom ′. In this case, by replacing the parameters param ′ (k1),..., Param ′ (kn) in FIG. 32 with the block numbers block (1) to block (n), the reliability of the block number block is determined. Will be judged.
[0110]
As a result of determining the reliability of the block number block as described above, if it is determined that the reliability is low, the error detection / correction unit 32 estimates the value of the unreliable block number block as follows. That is, the error detection / correction unit 32, as shown in FIG. 36, based on the common information ecom ', the encoding parameter ek', and the block number information block (step K1), the block of the reliable common information ecom ' By referring to the group corresponding to the number block, the block number block having the same encoding parameter ek ′ is searched for, and the value of the unreliable block number information block is estimated (step K2).
[0111]
For example, as shown by “x” in FIG. 37, when some block numbers block are missing due to low reliability, the error detection / correction unit 32 determines the block number block of the reliable common information ecom ′. By checking the correspondence relationship with the same encoding parameter ek ′ belonging to each group (see the solid line in FIG. 37), the missing block number block is estimated and restored (see the circled numbers in FIG. 37).
[0112]
Note that the error detection / correction unit 32 may perform the reliability determination of the common information ecom ′ and the reliability determination of the block number information block as described above. That is, the error detection / correction unit 32 of the present embodiment determines the reliability of the common information ecom ′ and / or the block number information block, and information with low reliability is obtained from information other than the information with low reliability. The correct information is estimated. As a result, the accuracy and reliability of error correction processing are greatly improved.
[0113]
As described above, according to the image encoding / decoding system 1 of the present embodiment, when image data is encoded in a certain block unit, which block indicates which block has the same encoding parameter ek. Since the encoded information ecom is generated and provided to the image decoding device 3, the image decoding device 3 encodes the encoded information having an error from the common information ecom even if there is an error in the received encoding parameter ek ′ to be decoded. It is possible to recognize which block has the same (common) parameter ek ′ as the encoding parameter ek ′, and correct the error by using the encoding parameter ek ′ of another block.
[0114]
Accordingly, since the tolerance to the error of the encoding parameter ek ′ is greatly improved and a highly accurate decoding process can be performed, it is always possible to minimize the deterioration of the image quality of the reproduced image, and to produce a high-quality reproduced image. Obtainable. In particular, it is considered to be extremely effective to apply to a system in which a large error occurs in the transmission path 4 such as wireless image communication using a portable terminal or the like.
[0115]
Further, in the image encoding device 2, the common information ecom is generated by commonization of the encoding parameter e by any one of the above-described common methods {circle around (1)} to {circle around (3)}. The number of blocks having the encoding parameter e (ek) can be increased to reduce the transmission amount of the different encoding parameter e (ek), and the tolerance against the error of the encoding parameter ek is further improved (apparent error) Rate is further reduced).
[0116]
Further, since the image coding apparatus 2 transmits (sends) blocks having the same coding parameter ek to a plurality of packets by the multiplexing unit 23, blocks having the same coding parameter ek are erroneous. Thus, the probability that the data will be lost together is greatly reduced. In the image decoding device 3, the error detection / correction unit 32 can reliably perform error correction, and the decoding process in the decoding unit 33 is reliable. It greatly contributes to improvement.
[0117]
Further, if the common information ecom is transmitted from the image encoding device 2 a plurality of times, the probability of occurrence of a situation in which the image decoding device 3 cannot normally receive the common information ecom (ecom ') is greatly reduced. Therefore, the image decoding apparatus 3 can surely perform error correction based on the common information ecom ′, thereby greatly improving the reliability of the decoding process.
[0118]
In the above-described embodiment, the type in which the image encoding device 2 does not perform local decoding has been described. However, for example, as illustrated in FIG. 38, the local decoding unit 24 is provided and the encoding parameter ek is set to this local decoding unit. The same effect as described above can be obtained even in a type in which the decoding unit 24 decodes to obtain the reproduced image signal Fk, and the encoding unit 21 performs coding with reference to the obtained reproduced image signal Fk.
[0119]
However, when local decoding is performed in this way, only the difference information between the past coding parameter ek and the current coding parameter ek is transmitted to the image decoding device 3, so that as shown in FIG. The decoding unit 33 of the apparatus 3 is configured to decode the current encoding parameter ek ″ using the reproduced image signal Fk ′ decoded in the past.
[0120]
(B) Description of the first modification
Incidentally, in the above-described embodiment, encoding parameters e (e1,..., Ex) that are not shared by the encoding unit 21 are obtained. For example, as shown in FIG. In some cases, already common encoding parameters ek1,..., Ekx (ek) are obtained. For example, in the encoding unit 21A, if the block size for motion compensation is made variable according to the intensity of image change and encoded, one motion compensation vector (for example, ek1) is obtained for a large block composed of a plurality of blocks. Therefore, the motion compensation vector ek1 can be regarded as a value that has already been shared across a plurality of blocks.
[0121]
Such a variable motion compensation block size method is described in, for example, each patent publication such as Japanese Patent No. 2618915, Japanese Patent No. 2618916, Japanese Patent No. 2702718, and Japanese Patent No. 2716702. The technology described in each patent publication will be briefly described.
(1) Japanese Patent No. 2618915
By dividing the block size for motion compensation into smaller blocks until the number of edges in the image data becomes smaller than a predetermined threshold, the block size is reduced for a portion with many edges and a large image change.
[0122]
(2) Japanese Patent No. 2618916
If the image data is divided into a plurality of small blocks, and the number of edges in the small block is larger than a predetermined threshold, the small blocks are combined until the number of edges in one block becomes smaller than the threshold. The block size is reduced for a portion having a large image change with many edges.
[0123]
(3) Japanese Patent No. 2702718
If a prediction error obtained as a result of motion compensation for a block having a certain required block size is larger than a predetermined threshold, the block size is reduced until the prediction error becomes smaller than the threshold, and motion compensation is performed. Even if the block size is reduced to the minimum block size, the prediction error does not become smaller than the threshold value, and intra-frame coding is performed on a portion where the image change is so severe that it is not suitable for motion compensation.
[0124]
(4) Japanese Patent No. 2716702
By combining small blocks and increasing the block size until one of the prediction errors for a plurality of adjacent small blocks becomes larger than a predetermined threshold, blocks for areas where image changes are severe Reduce the size.
[0125]
In such a case, in the commoning unit 22A, the already common coding parameter ek is input from the coding unit 21A. Therefore, the commonization as described above is not performed, and the input coding parameter ek is used. The common information ecom similar to that in the above-described embodiment is generated and output together with the encoding parameter ek.
That is, as shown in FIG. 40, assuming that the encoding parameter of the unit n block that has been shared is ek and the sharing information is ecom (step L1), the sharing unit 22A has already shared the coding parameter. The common information ecom indicating which block has the same (common) encoding parameter ek is calculated from ek (step L2), and the already-shared encoding parameter ek and the generated common information ecom are calculated. Are output to the multiplexing unit 23 (step L3).
[0126]
In FIG. 39, parts other than the encoding unit 21A and the commoning unit 22A (parts denoted by the same reference numerals as those in FIG. 1) are all the same as or similar to the parts described above with reference to FIG.
Therefore, in the first modified example, the same effect as that of the system 1 described above with reference to FIG. 1 can be obtained, and in this case, it is not necessary to share in the common unit 22A. It is possible to speed up the generation process.
[0127]
Also in the first modified example, the image encoding device 2 includes a local decoding unit 24 as shown in FIG. 41, for example, and decodes the encoding parameter ek by the local decoding unit 24 to reproduce the reproduced image signal. Fk may be obtained, and the encoding unit 21A may perform the encoding with reference to the obtained reproduced image signal Fk. In this case, the same operation effect as described above can be obtained, and this type Advantages specific to the image encoding device 2 can also be obtained.
[0128]
(C) Description of the second modification
FIG. 42 is a block diagram showing a second modification of the above-described embodiment. The image encoding / decoding system 1 shown in FIG. 42 is different from that shown in FIG. Instead of the error detection / correction unit 32, a multiplexing / decoding unit 31A including a demultiplexing unit 311, an error detection / correction unit 312 and a multiplexing conversion unit 313 is different.
[0129]
Here, in the multiplexing / decoding unit 31A, the multiplexing / separating unit (receiving unit) 311 transmits a bit string [coding parameter ek ′ (ek1) transmitted from the image coding device 2 (multiplexing unit 23) through the transmission path 4. ′,..., Ekx ′) and common information ecom ′] and demultiplexing at least the encoding parameter ek ′, thereby encoding parameters before multiplexing decoding (multiplexed encoding values) muxek ′ (muxe1 ′,..., muxex ′) (variable length code or fixed length code) is obtained. Note that the common information ecom ′ may or may not be demultiplexed as long as the error detection / correction unit 312 can refer to it.
[0130]
Further, the error detection / correction unit 312 uses the common information ecom ′ to compare the bits [or values (codes) themselves] of the multiplexed encoded value muxek ′ that should have the same value, thereby to obtain the multiplexed code. An error is detected in the multiplex value muxek ′, and the error is corrected to obtain a multiplex encoded value muxek ″. The multiplex conversion unit 313 performs the multiplex that has been error-corrected by the error detection / correction unit 312. The encoded encoded value muxek "is multiplexed and decoded by converting it into the original encoded parameter ek '.
[0131]
That is, the image decoding device 3 according to the second modified example does not perform error correction on the encoding parameter (value) ek ′ after the multiplex decoding as described above, but performs the multiplex code before the multiplex decoding. It is possible to perform error correction on the digitized value muxek ′ in bit units (or the unit of the value itself if the bit unit is impossible).
Hereinafter, the operation of the multiplexing decoding unit 31A of the second modification example configured as described above will be described in detail with reference to the flowchart (steps M1 to M5) shown in FIG.
[0132]
First, before (in the middle of) decoding, muxek ′ represents the multiplexed coding value of the coding parameter ek ′ of the unit n block that has been shared, and ek represents the multiplexed decoding value of the coding parameter ek ′ that has been error-corrected. ″, Assuming that the common information (multiplexed and decoded) is ecom ′ (step M1), the demultiplexing unit 311 performs demultiplexing of the received bit string, and the multiplexed encoded value muxek ′ and the common information ecom 'Is obtained (step M2).
[0133]
Then, in the error detection / correction unit 312, the multiplexed encoded values muxek ′ (which can be specified from the common information ecom ′) that should have the same encoding parameter ek ′ have the same bit values in the absence of errors. If [the same variable length (or fixed length) encoding is performed for the same encoding parameter ek ′], the multiplexed encoded value muxek ′ (k1),..., Muxek ′ (ku) Since bitwise comparison is possible, compare the multiplexed encoded values muxek ′ (k1), ..., muxek ′ (ku) bit by bit (if bit by bit is impossible, compare the values as they are) (Step M3).
[0134]
As a result of this comparison, if there is a bit (or value) that is different from the others, it is assumed that the bit (or value) is incorrect, and a correct bit or correct value is selected for each bit and error-corrected multiplexed code. The multiplexed value muxek "is obtained (step M4). The obtained multiplexed coded value muxek" is multiplexed and decoded by the multiplexing conversion unit 313 to obtain the multiplexed decoded value ek "(step M5). .
[0135]
Specifically, for example, as shown in FIG. 44, multiplexed encoded values muxek ′ (k1),..., Muxek ′ (ku) that should have the same value multiplexed and demultiplexed by the demultiplexing unit 311. Is received (step M6), the multiplexed encoded values muxek ′ (k1),..., Muxek ′ (ku) are inspected (compared) bit by bit, and an error due to the error correcting code is found. Among those that did not exist, the most frequently occurring value (0 or 1) is set as the value of the multiplexed encoded value muxek "(k1), ..., muxek" (ku) after error correction (step M7). ).
[0136]
In order to check the multiplexed encoded value muxek ′ itself, not for each bit, the encoding parameter ek ′ is set to the multiplexed encoded value muxek ′ in the flowchart (algorithm) shown in FIG. The replaced algorithm may be executed.
As described above, according to the image encoding / decoding system 1 in the second modification, the image decoding apparatus 3 does not perform error correction on the encoding parameter (value) ek ′ after multiplexing decoding. In addition, error correction can be performed on the multiplexed coded value muxek ′ before multiplexed decoding in bit units (separated from errors of other bits), so that the same operation as that of the system 1 described above with reference to FIG. In addition to the effect, there is an advantage that the error correction efficiency is improved.
[0137]
By the way, the error detection / correction unit 312 described above also determines the reliability of the common information ecom ′ (or block number information block) in the same manner as the error detection / correction unit 32 (see FIG. 1). When the reliability is low, a portion with low reliability can be estimated and restored. However, the estimation / restoration method of the unreliable portion only needs to replace the encoding parameter ek ′ with the multiplexed encoded value muxek ′ in the method described above with reference to FIGS. Then, the detailed description is abbreviate | omitted and only the determination method of reliability is demonstrated.
[0138]
That is, the error detection / correction unit 312 converts the received parameters (common information ecom ′ or block number information block) into param ′ (k1),..., Param ′ (, for example, as shown in FIG. kn), if the threshold for the frequency of occurrence of the reception parameters param ′ (k1),..., param ′ (kn) is tha (step H1), the reception parameters param ′ (k1),. kn) is checked for each bit, and the occurrence frequency EA (0 or 1) of the error that is not found by the error correction code is checked (step H2 ').
[0139]
Then, the error detection / correction unit 312 determines that the difference between the checked occurrence frequency EA and the total number ET of occurrence frequencies of all parameters param ′ (k1),... Param ′ (kn) is smaller than a certain threshold value tha. If so, it is determined that the reliability is low (step H3 '). In this step H3 ′, step H3 ″ shown in FIG. 45 (b) may be performed. That is, in this case, all parameters param ′ (k1),. If the ratio (EA / ET) of the occurrence frequency of kn) to the total number ET exceeds a certain threshold value tha, it is determined that the reliability is low.
[0140]
Thereby, also in the second modified example, the reliability of the error correction process is improved, and the reliability of the decoding process of the image data is improved.
In the second modification, the image encoding device 2 shown in FIG. 42 is of the same type (basic type) as described above with reference to FIG. 1, but for example, as shown in FIG. The type provided with the decoding unit 24, the type that obtains the encoding parameter ek that is already common at the time of encoding as shown in FIG. 47, and the configuration shown in FIG. 47 as shown in FIG. Any of the types to which is added can be applied. In addition, regardless of which type is applied, the same effects as described above can be obtained, and advantages specific to each type as described above can be obtained.
[0141]
(D) Description of the third modification
49 is a block diagram showing a third modification of the above-described embodiment. The image encoding / decoding system 1 shown in FIG. 49 is different from that shown in FIG. The storage unit 25, the notification reception / retransmission unit 26, and the retransmission transmission unit 27 are provided, and the image decoding apparatus 3 is provided with a storage unit 34, a notification unit 35, and a retransmission reception unit 36. 27 to 34 and 36 to 36 are different in that a retransmission unit 6 is formed which performs retransmission of information when there is an encoding parameter ek ′ that cannot be corrected by the error detection / correction unit 32.
[0142]
Here, in the image decoding device 3, even if the storage unit 34 tries to obtain the correct value by performing error detection / correction in the error detection / correction unit 32, as described above, the encoding parameter ek ′, etc. Information received by the error detection / correction unit 32 is rejected in the case where there are too many portions determined to be wrong for certain information and the reliability is low and the decoding unit 33 cannot perform decoding as it is. It is for accumulating without.
[0143]
In addition, the notification unit (retransmission request unit) 35 is for notifying the image encoding device 2 of the fact when the error detection / correction unit 32 determines that decoding is impossible as described above. However, here, a request (retransmission request) image encoding device for retransmitting only information necessary for re-error correction of the encoding parameter ek ′ that cannot be corrected by the error detection / correction unit 32. 2 to do. This request (notification) may be made through the transmission line 4, or through a transmission line dedicated for retransmission (may be wired or wireless), or via a transmission line dedicated for notification (may be wired or wireless). Also good.
[0144]
The retransmission receiving unit 36 receives information retransmitted from the image encoding device 2 in response to the notification from the notification unit 35.
On the other hand, in the image encoding device 2, the storage unit 25 stores the encoding parameter ek and the common information ecom so that retransmission can be performed at any time in response to the notification from the notification unit 34. The notification reception / retransmission unit 26 receives the notification (request) from the notification unit 35, and reads information corresponding to the request from the storage unit 25 and outputs the information to the retransmission transmission unit 27.
[0145]
Then, the retransmission transmission unit (retransmission unit) 27 receives only the information received from the notification reception / retransmission unit 26, that is, the information corresponding to the request from the notification unit 35 of the image decoding device 3. 3 is transmitted (retransmitted). Note that this information may also be transmitted using the transmission path 4 or via a dedicated transmission path (wired or wireless).
[0146]
Hereinafter, the operation of the image encoding / decoding system 1 according to the third modified example configured as described above will be described in detail. However, parts other than the retransmitting unit 6 (parts denoted by the same reference numerals as those shown in FIG. 1) are the same as or similar to those described above with reference to FIG. In the following, the operation focusing on the retransmission unit 6 will be described with reference to the flowchart (steps N1 to N7) shown in FIG.
[0147]
First, when error detection / correction unit 32 detects information (encoding parameter ek ′) having low reliability and undecoding as a result of error correction (step N1), the information is stored in storage unit 34. (If it is not used, it does not need to be accumulated.) Information that can be corrected is output to the decoding unit 33. Here, when error correction is possible for all information in the error detection / correction unit 32, or when the notification by the notification unit 35 exceeds a predetermined number of times, or the transmission path 4 [ ) Dedicated transmission path] When there is no margin in the transmission capacity, or when there is no margin in the error detection / correction processing capability of the error detection / correction unit 32, the notification is not performed by the notification unit 35 and the process is terminated. (Step N2).
[0148]
Note that the case where there is no room in the transmission line 4 [or the transmission line dedicated for retransmission (notification)] means that a predetermined bit rate has been used up and the retransmission is taking time due to congestion of the line. There are cases where there is no margin in processing capacity, such as when there is no margin in the capacity of the storage unit 34, or when there is a large delay in the decoding unit 33 and practical image communication cannot be tolerated.
[0149]
Next, when there is information that cannot be corrected (decoded), the type of information (encoding parameter ek, common information ecom, group number, block number, error bit position, etc.) is notified. (Step N3), the notification unit 35 requests only the information (partial or all) desired to be retransmitted from the image encoding device 2 (notification reception / retransmission unit 26) (step N4).
[0150]
When receiving this request, the notification receiving / retransmitting unit 26 extracts necessary information from the storage unit 25 and outputs it to the retransmission transmitting unit 27 (step N5). The retransmission transmission unit 27 transmits the received information to the image decoding device 3 (retransmission reception unit 36) (step N6). The retransmission receiving unit 36 outputs the received retransmission information to the error detection / correction unit 32. The error detection / correction unit 32 performs error detection / correction processing again using the retransmission information and the information stored in the storage unit 34 (or using only the retransmission information) (step N7). The process from the beginning (step N1) is repeated until the above-described conditions are satisfied in step N2.
[0151]
As a result, the amount of information that cannot be error-corrected is greatly reduced, so that the amount of information that cannot be decoded by the decoding unit 33 is also greatly reduced. As a result, an extremely high quality reproduced image signal Fk ′ can be obtained. it can. Further, in the third modification example, even when information retransmission is required, only the minimum necessary information is retransmitted. Therefore, the amount of retransmission information is greatly reduced, and the transmission path 4 [ This greatly contributes to the effective use of the transmission (dedicated transmission line) bandwidth.
[0152]
Also in the third modification, the image encoding device 2 shown in FIG. 49 is, for example, a type having the local decoding unit 24 as shown in FIG. 51, and already common at the time of encoding as shown in FIG. Any of the types in which the encoded parameter ek is obtained, as shown in FIG. 53, and the type in which the local decoding unit 24 is further added to the configuration shown in FIG. 52 can be applied. In addition, regardless of which type is applied, the same effects as described above can be obtained, and advantages specific to each type as described above can be obtained.
[0153]
Also, the image decoding device 3 shown in FIG. 49 is different from the above-described types of image encoding devices 2 in that after multiplexing and demultiplexing instead of the encoding parameter ek ′ itself as shown in FIGS. 54 to 57, for example. The multiplexed coded value muxek ′ may be of a type provided with a multiplexing / decoding unit 31A that performs error detection / correction. In this case, the advantage unique to this type of multiplexing / decoding device 3 is further provided. Is obtained.
[0154]
When the image decoding device 3 is of a type having the multiplexing decoding unit 31A as described above, the bit-by-bit (or value itself) inspection can be performed as described above. For example, this can be performed as shown in FIG.
That is, when there is information that cannot be corrected, the error detection / correction unit 312 notifies the notification unit 35 of the type of information and the number of error bits (position information) (step P1). Then, the notification unit 35 sets the requested information type and the error bit position information as a single unit (for each bit) or a plurality of bits, and attaches the identification information (request number) 7 to the image encoding device 2 (notification). A request (transmission) is made to the reception / retransmission unit 26) side (step P2).
[0155]
In the image encoding device 3, when this request is received by the notification reception / retransmission unit 26, the requested information is extracted from the storage unit 25 and output to the retransmission transmission unit 27, and the retransmission transmission unit 27. Then, the same identification information 7 as the identification information 7 requested from the notification unit 35 is attached, and the information is retransmitted to the image decoding apparatus 3 (step P3). Accordingly, in this case, since information necessary for re-error correction can be transmitted for each bit, the amount of transmission information at the time of retransmission can be further reduced. In the case of checking the value itself, the procedure is the same as above, except that the bit to be retransmitted becomes the value.
[0156]
(E) Other
In each of the above-described embodiments and modifications, the image data encoded by the image encoding device 2 is transmitted to the image decoding device 3 through the transmission path 4, and the image data encoded by the image decoding device 3 is reproduced. The image communication system 1 has been described as an example. For example, as shown in FIG. 59, the image data encoded (compressed) by the image encoding device 2 is stored in the storage medium 8, and the image data is stored as necessary. A system 1 'that appropriately reads out and reproduces image data stored in the storage medium 8 (for example, compresses and stores image data on a single personal computer, and decodes and reproduces the image data as necessary. However, the same effects as those of the above-described embodiments and modifications can be obtained.
[0157]
That is, in the case of such a system 1 ′, there is a possibility that an error occurs in information when information (encoding parameter ek) is stored in the storage medium 8 or when information is read from the storage medium 8. In this case, as in the above-described embodiment and each modification, the error can be efficiently corrected by the common information ecom, so that error tolerance can be greatly improved, and an extremely high quality reproduced image can be obtained. Obtainable.
[0158]
In FIG. 59, the storage medium 8 and the storage unit 25 are provided separately. However, whether the contents of the information stored in the storage medium 8 and the storage unit 25 have passed through the multiplexing unit 23, respectively. Since the only difference is whether or not it is passed, as shown in FIG. 60, it is possible to reduce the scale of the apparatus if the storage medium 8 'is shared. However, in this case, since the information to be retransmitted does not have to be multiplexed, the retransmission unit 6 needs to have a multiplexing / decoding unit 28 having the same function as the multiplexing / decoding unit 32 (image coding). It may be provided in either the device 2 or the image decoding device 3).
[0159]
59 and 60 both show the configuration in which the retransmission unit 6 described above in the third modification is provided, but of course, the type in which the retransmission unit 6 is not provided. The same applies to the system.
Further, also in this case, the image encoding device 2 shown in FIG. 59 includes, for example, the type including the local decoding unit 24 (see, for example, FIG. 51), and the encoding parameters already shared at the time of encoding. Either a type that provides ek (for example, see FIG. 52) or a type in which the local decoding unit 24 is added to this type (for example, see FIG. 53) can be applied. It is possible to apply a type (for example, see FIGS. 54 to 57) provided with a multiplexing / decoding unit 31A that performs error detection / correction on the multiplexed encoded value muxek ′ after demultiplexing. .
[0160]
In the above-described system 1 (1 ′), for the sake of convenience of explanation, the case where the image encoding device 2 and the image decoding device 3 are configured as devices dedicated for encoding and decoding, respectively, is taken as an example. However, in practice, as shown in FIG. 61, for example, the above-described image encoding device 2 and image decoding device 3 are provided in the image transmission devices 2A and 3A provided to face each other, respectively. In general, the image transmission apparatuses 2A and 3A have both encoding and decoding functions as image encoding / decoding apparatuses.
[0161]
That is, also in this case, each of the image transmission devices 2A and 3A includes an encoding unit 21 (or 21A), a commoning unit 22 (or 22A), and a multiplexing 23 as the image encoding device 2, respectively. The device 3 includes a multiplex decoding unit 31 (or 31A), an error detection / correction unit 32, and a decoding unit 33, and is generated and transmitted by the commoning unit 22 (or 22A) of the opposing image transmission devices 3A and 2A. The error correction of the encoding parameter ek ′ is performed using the common information ecom ′.
[0162]
Further, in the system 1 (1 ′) described above, the image encoding device 2 is provided with a function (multiplexing unit 23) for transmitting the common information ecom, and the image decoding device 3 receives the common information ecom ′. However, the present invention is not limited to this, and at least the image decoding device 3 transmits the common information ecom 'to the common unit 22. (22A) should just be the structure which can be received.
[0163]
And this invention is not limited to embodiment mentioned above and each modification, It can implement in various deformation | transformation in the range which does not deviate from the meaning of this invention.
[0164]
【The invention's effect】
As described above in detail, according to the present invention, when image data is encoded in units of a certain block, common information indicating which blocks have the same encoded information is generated and transmitted to the decoding side. Therefore, the decoding side recognizes which block has the same encoded information as the encoded information from the common information, even if there is an error in the encoded information to be decoded. The error can be corrected using the encoded information. Therefore, the resistance to errors in the encoded information is greatly improved, and high-precision decoding processing can be performed. As a result, deterioration of the image quality of the reproduced image is always minimized and a high-quality reproduced image can be obtained. (Claims 1, 3, 4, 11, 16 to 18).
[0165]
Here, on the encoding side, a histogram of the frequency of occurrence of encoded information in a plurality of blocks is created for the above-mentioned common information, and encoded information whose occurrence frequency is a predetermined value or less is coarsely quantized in this histogram. Or, it is the same if the encoding information is shared and generated by correcting to the neighboring encoding information in the histogram or by correcting to the encoding information of the neighboring block. Since the number of blocks having encoded information can be increased and the amount of transmission of different encoded information can be reduced, it is possible to further improve the tolerance against errors in the encoded information (claims 5 to 7).
[0166]
Also, from the encoding side, if blocks having the same encoding information are distributed and transmitted in a plurality of packet data, there is a probability that blocks having the same encoding information will be lost together due to errors or the like. Since the error is greatly reduced, the above-described error correction can be reliably performed, which greatly contributes to the improvement of the reliability of the decoding process on the decoding side (claim 8).
[0167]
Furthermore, if the common information is sent from the encoding side a plurality of times, the probability of occurrence of a situation in which the decoding side cannot normally receive the common information can be greatly reduced. Thus, it is possible to reliably perform error correction based on the common information, thereby greatly improving the reliability of the decoding process (claim 9).
[0168]
If there is encoded information that cannot be corrected on the decoding side, a re-transmission request for information necessary for re-error correction of the encoded information is made to the encoding side, and the request is made from the encoding side. If only the received information is retransmitted to the decoding side, the amount of retransmitted information at the time of error correction capability can be greatly reduced (claims 2, 10, 15).
[0169]
Further, the error correction described above specifies other blocks having the same coding information as the coding information having an error based on the common information, when it is known in advance that the coding information has an error, If the encoding information of the block is output as the encoding information of the erroneous block, it can be surely realized by an extremely simple process.
[0170]
If it is not known in advance that there is an error in the encoded information, identify a plurality of blocks that should have the same encoded information based on the common information and compare the encoded information of these multiple blocks. Thus, if an error is detected and the error is corrected, even when it is not known in advance that there is an error in the encoded information, it is possible to cope with it reliably (claim 13).
[0171]
Furthermore, at the time of error correction, if the reliability of common information and / or block identification information is determined, and correct information is estimated from information other than the information with low reliability for low reliability information, Thus, the accuracy and reliability of the error correction process can be greatly improved.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an image communication system (image encoding / decoding system) as an embodiment of the present invention.
FIGS. 2A to 2D are diagrams for explaining unit blocks to be shared in the image coding apparatus according to the present embodiment.
FIG. 3 is a flowchart for explaining the operation of a common unit in the image coding apparatus according to the present embodiment.
4A is a diagram illustrating an example of an encoding parameter value of a certain block, and FIG. 4B is a diagram illustrating an example of a corresponding block number.
5A is a diagram showing an example of common information according to the present embodiment, and FIG. 5B is a diagram showing an example of multiplexed common information according to the present embodiment.
FIG. 6 is a flowchart for explaining a detailed operation of a common unit of the present embodiment.
FIG. 7 is a diagram for explaining a modification of the flowchart shown in FIG. 6;
FIG. 8 is a diagram for explaining a modified example of the flowchart shown in FIG. 6;
FIG. 9 is a flowchart for explaining a specific example of detailed operation of a common unit according to the present embodiment;
FIG. 10 is a flowchart for explaining another specific example of the detailed operation of the common unit according to the embodiment;
FIG. 11 is a flowchart for explaining another specific example of the detailed operation of the common unit according to the embodiment;
FIGS. 12A and 12B are diagrams for explaining an example of generation of an occurrence frequency histogram by a common unit of the present embodiment.
FIG. 13 is a flowchart for explaining another detailed operation of the common unit of the present embodiment;
FIG. 14A is a diagram showing an example of common information according to the present embodiment, and FIG. 14B is a diagram for explaining a process when creating a packet based on FIG.
FIG. 15 is a flowchart for explaining another detailed operation of the common unit of the present embodiment;
FIG. 16 is a flowchart for explaining another detailed operation of the common unit of the present embodiment;
FIG. 17 is a diagram illustrating an example of an encoding parameter value of a certain block according to the present embodiment.
FIGS. 18A to 18D are diagrams for explaining block division processing for the block shown in FIG. 17;
FIG. 19 is a diagram illustrating an example of common information obtained by block division processing according to the present embodiment.
FIG. 20 is a flowchart for explaining the operation of the multiplexing unit in the image coding apparatus according to the present embodiment;
FIG. 21 is a flowchart for explaining the detailed operation of the multiplexing unit of the present embodiment;
FIG. 22 is a diagram showing another example of multiplexed common information according to the present embodiment.
FIGS. 23A and 23B are diagrams for explaining the operation of the multiplexing unit of the present embodiment.
FIG. 24 is a flowchart for explaining detailed operation of a multiplexing unit of the present embodiment;
FIG. 25 is a diagram for explaining the operation of the multiplexing unit of the present embodiment;
FIG. 26 is a diagram for explaining the operation of the multiplexing unit of the present embodiment;
FIG. 27 is a diagram for explaining the operation of the multiplexing unit of the present embodiment;
FIG. 28 is a flowchart for explaining the operation of a multiplexing decoding unit in the image decoding apparatus of the present embodiment.
FIG. 29 is a flowchart for explaining the operation of an error detection / correction unit in the image decoding apparatus according to the present embodiment;
FIG. 30 is a flowchart for explaining the operation of the error detection / correction unit of the present embodiment;
FIG. 31 is a flowchart for explaining the operation of the error detection / correction unit of the present embodiment;
FIG. 32 is a flowchart for explaining reliability determination processing by an error detection / correction unit according to the present embodiment;
FIG. 33 is a flowchart for explaining information estimation processing by an error detection / correction unit of the present embodiment;
FIG. 34 is a diagram for explaining information estimation processing by an error detection / correction unit according to the present embodiment;
FIG. 35 is a diagram for describing information estimation processing by an error detection / correction unit according to the present embodiment;
FIG. 36 is a flowchart for explaining another information estimation process by the error detection / correction unit of the present embodiment;
FIG. 37 is a diagram for explaining information estimation processing by an error detection / correction unit according to the present embodiment;
38 is a block diagram illustrating another configuration example of the image encoding / decoding system illustrated in FIG. 1. FIG.
FIG. 39 is a block diagram illustrating a first modification of the image encoding / decoding system according to the present embodiment.
FIG. 40 is a flowchart for explaining an operation of a common unit of the image encoding / decoding system according to the first modification.
41 is a block diagram illustrating another configuration example of the image encoding / decoding system illustrated in FIG. 40. FIG.
FIG. 42 is a block diagram illustrating a second modification of the image encoding / decoding system according to the present embodiment.
FIG. 43 is a flowchart for explaining the operation of the multiplexing / decoding unit of the image encoding / decoding system according to the second modified example;
FIG. 44 is a flowchart for explaining the operation of the multiplexing / decoding unit of the image encoding / decoding system according to the second modification.
45 (a) and 45 (b) are flowcharts for explaining reliability determination processing in the multiplex decoding unit of the second modified example.
46 is a block diagram showing another configuration example of the image encoding / decoding system shown in FIG. 42. FIG.
47 is a block diagram showing another configuration example of the image encoding / decoding system shown in FIG. 42. FIG.
48 is a block diagram showing another configuration example of the image encoding / decoding system shown in FIG. 42. FIG.
FIG. 49 is a block diagram illustrating a third modification of the image encoding / decoding system according to the present embodiment.
FIG. 50 is a flowchart for explaining the operation of a retransmission unit of an image encoding / decoding system according to a third modification.
51 is a block diagram illustrating another configuration example of the image encoding / decoding system illustrated in FIG. 49. FIG.
52 is a block diagram illustrating another configuration example of the image encoding / decoding system illustrated in FIG. 49. FIG.
53 is a block diagram illustrating another configuration example of the image encoding / decoding system illustrated in FIG. 49. FIG.
54 is a block diagram showing another configuration example of the image encoding / decoding system shown in FIG. 49. FIG.
55 is a block diagram illustrating another configuration example of the image encoding / decoding system illustrated in FIG. 49. FIG.
56 is a block diagram illustrating another configuration example of the image encoding / decoding system illustrated in FIG. 49. FIG.
57 is a block diagram illustrating another example configuration of the image encoding / decoding system illustrated in FIG. 49. FIG.
FIG. 58 is a flowchart for explaining a retransmission operation in the image encoding / decoding system shown in FIGS. 54 to 57;
Fig. 59 is a block diagram illustrating a configuration example of an image encoding / decoding system in a case where encoded image data is stored.
60 is a block diagram showing another configuration example of the image encoding / decoding system shown in FIG. 59. FIG.
Fig. 61 is a block diagram illustrating a configuration of an image encoding / decoding device according to an embodiment of the present invention.
FIG. 62 is a block diagram showing a basic configuration of a conventional image communication system (image encoding / decoding system).
FIG. 63 is a block diagram showing a modification of the image encoding / decoding system shown in FIG. 61.
FIG. 64 is a diagram illustrating an output example of a multiplexing unit in a conventional image encoding / decoding system.
[Fig. 65] Fig. 65 is a block diagram illustrating a configuration example of a main part of a conventional image encoding / decoding system.
66 is a block diagram illustrating a configuration example of an encoding unit illustrated in FIG. 64. FIG.
67 is a diagram for explaining a function of the multiplexing unit illustrated in FIG. 64; FIG.
[Fig. 68] Fig. 68 is a diagram illustrating a format example of a video packet.
[Explanation of symbols]
1. Image communication system (image encoding / decoding system)
2 Image coding device
3 Image decoding device
2A, 3A image transmission device (image encoding / decoding device)
4 transmission lines
5 Common information (common information)
6 Retransmission part
7 Identification information (request number)
8,8 'Storage medium
21,21A encoding unit
22,22A Common unit (common information generation unit)
23 Multiplexing unit (sending unit)
24 Local decoding unit
25, 34 Accumulator
26 Notification reception / retransmission unit
27 Retransmission Transmitter (Retransmission Unit)
31 Multiplexing decoding unit (receiving unit)
31A Multiplexing decoder
32,312 error detection / correction unit
33 Decryption unit
35 Notification part (Retransmission request part)
36 Retransmission receiver
311 Demultiplexer (Receiver)
313 Multiplex conversion unit

Claims (18)

In an image encoding / decoding system comprising an image encoding device that encodes image data and an image decoding device that decodes and reproduces the image data encoded by the image encoding device,
The image encoding device
An encoding unit that obtains predetermined encoding information about the block by encoding the image data in units of one or more pixels; and
A common information generating unit that generates common information indicating which blocks have the same encoded information among the encoded information about the plurality of blocks obtained by the encoding unit;
The image decoding device
An error correction unit that corrects an error of the encoded information based on the common information generated by the common information generation unit of the image encoding device;
An image encoding / decoding system comprising: a decoding unit that decodes encoded information that has been error-corrected by the error correction unit and reproduces the image data. The image decoding device
The error correction unit includes a re-transmission request unit that makes a re-transmission request to the image encoding device for information necessary for re-error correction of encoded information that cannot be corrected by the error correction unit,
The image encoding device
The image code according to claim 1, further comprising a re-sending unit that re-sends only information corresponding to the re-sending request from the re-sending request unit of the image decoding device to the image decoding device. Conversion / decoding system. An encoding step of obtaining predetermined encoding information for the block by encoding the image data in units of one or more pixels;
A common information generating step for generating common information indicating which blocks have the same encoded information among the encoded information for the plurality of blocks obtained by the encoding step;
An error correction step for correcting an error in the encoded information based on the common information generated in the common information generation step;
An image encoding / decoding method comprising: a decoding step of decoding the encoded information error-corrected in the error correction step and reproducing the image data. An encoding unit that obtains predetermined encoding information about the block by encoding the image data in units of one or more pixels; and
A common information generating unit that generates common information indicating which blocks have the same encoded information among the encoded information about the plurality of blocks obtained by the encoding unit;
An image encoding device comprising: at least a transmission unit that transmits the common information generated by the common information generation unit to an image decoding device. The common information generation unit
A histogram of the frequency of occurrence of encoded information in a plurality of blocks is created, and the encoded information is shared by coarsely quantizing encoded information whose occurrence frequency is a predetermined value or less in the histogram. The image encoding device according to claim 4, wherein the image encoding device is configured to generate the common information. The common information generation unit
A histogram is generated regarding the frequency of occurrence of encoding information in a plurality of blocks, and the encoding information having the occurrence frequency equal to or lower than a predetermined value in the histogram is corrected to neighboring encoding information in the histogram. 5. The image coding apparatus according to claim 4, wherein the common information is generated by commonization and the common information is generated. The common information generation unit
A histogram is generated regarding the frequency of occurrence of encoding information in a plurality of blocks, and the encoding information having the occurrence frequency equal to or lower than a predetermined value is corrected to the encoding information of neighboring blocks in the histogram. The image encoding apparatus according to claim 4, wherein the common information is generated by performing the common processing. The sending unit is
The image coding apparatus according to any one of claims 4 to 7, wherein blocks having the same coding information are configured to be distributed and transmitted in a plurality of packet data. The sending unit is
The image encoding device according to claim 4, wherein the common information is transmitted a plurality of times. Upon receiving a re-transmission request for information necessary for re-error correction of encoded information that cannot be error-corrected from the image decoding device, a re-transmission unit that re-transmits only information corresponding to the re-transmission request to the image decoding device 10. The image encoding device according to claim 4, wherein the image encoding device includes: By encoding image data in units of one or more pixels in an image encoding device, predetermined encoding information can be obtained for the block, and which blocks among the encoding information for a plurality of blocks are the same encoding A receiver that receives at least the common information from the image encoding device when the common information indicating whether the information is included is obtained;
An error correction unit that corrects an error of the encoded information based on the common information received by the reception unit;
An image decoding apparatus comprising: a decoding unit that decodes encoded information that has been error-corrected by the error correction unit and reproduces the image data. The error correction unit
When it is known in advance that the coding information has an error, another block having the same coding information as the coding information having the error is identified based on the common information, and the code of the other block is identified. 12. The image decoding apparatus according to claim 11, wherein the encoded information is output as encoded information of the erroneous block to correct an error of the encoded information. The error correction unit
When it is not known in advance that there is an error in the encoded information, by identifying a plurality of blocks that should have the same encoded information based on the common information, and comparing the encoded information of the plurality of blocks The image decoding device according to claim 11, wherein the image decoding device is configured to detect an error and correct the error. The error correction unit
It is configured to judge the reliability of the common information or the identification information of the block or both, and to estimate correct information from information other than the information with low reliability for information with low reliability The image decoding device according to any one of claims 11 to 13. 12. A re-transmission request unit that makes a re-transmission request for information necessary for re-error correction of encoded information that cannot be corrected by the error correction unit to the image encoding device. The image decoding apparatus of any one of -14. An encoding step of obtaining predetermined encoding information for the block by encoding the image data in units of one or more pixels;
A common information generating step for generating common information indicating which blocks have the same encoded information among the encoded information for the plurality of blocks obtained by the encoding step;
An image encoding method comprising: a sending step for sending at least the common information generated in the common information generation step to a decoding side. By encoding the image data in units of one or more pixels, predetermined encoding information can be obtained for the block, and which of the encoding information for a plurality of blocks has the same encoding information A reception step for receiving at least the common information from the encoding side when the common information indicating whether or not
An error correction step for correcting an error in the encoded information based on the common information received in the reception step;
An image decoding method comprising: a decoding step of decoding the encoded information that has been error-corrected by the error correction step and reproducing the image data. An image encoding device that encodes image data, and an image decoding device that decodes and reproduces image data encoded by an image encoding / decoding device provided opposite to the image encoding device In an image encoding / decoding device,
The image encoding device
An encoding unit that obtains predetermined encoding information about the block by encoding the image data in units of one or more pixels; and
A common information generating unit that generates common information indicating which blocks have the same encoded information among the encoded information about the plurality of blocks obtained by the encoding unit;
The image decoding device
Which block has the same encoding information among the encoding information for a plurality of blocks obtained by encoding the image data in the image encoding / decoding device generated by the opposing image encoding / decoding device. An error correction unit that corrects an error in the encoded information based on the common information indicating having,
An image encoding / decoding apparatus, comprising: a decoding unit that decodes encoded information that has been error-corrected by the error correction unit and reproduces the image data.

Images