CN100438631C - Image encoding method and image decoding method - Google Patents

Abstract

The image coding method includes a step for executing coding of an image (Step 100), a step for judging whether a picture in the memory which is never used as reference exists (Step 102), a step for coding memory management information for releasing the picture in the memory which is never used as reference when a picture in the memory which is never used as reference exists (Step 103), a step for releasing the picture in the memory which is never used as reference (Step 104), a step for judging whether the memory management information that releases the picture in the memory which is never used as reference is coded by coding of an image immediately before (Step 105), and a step for coding again the memory management information that releases the picture in the memory which is never used as reference when the memory management information is coded (Step 106).

Description

Image encoding method and image encoding device

technical field

The present invention relates to an image encoding method for efficiently compressing a moving image signal by utilizing inter-image correlation, an image decoding method for correctly decoding it, and a program for performing the operation by software.

Background technique

In recent years, in the era of multimedia that uniformly processes sound, images, and other pixel values, devices that convey information from existing information media, such as newspapers, magazines, TVs, radios, and telephones, to people have become a hot topic as multimedia objects. . Generally speaking, the so-called multimedia refers to not only texts, but also information media that are associated with graphics, sounds, and especially images. However, in order to use the above-mentioned existing information media as multimedia objects, the necessary condition is Information is represented in numerical form.

However, if the amount of information held by the above-mentioned information media is estimated as the amount of digital information, the amount of information per character is 1 to 2 bytes when the information is text, and the information per second when the information is sound The volume is 64kbits (telephone quality), and for animation, the amount of information above 100Mbits (current TV reception quality) is required per second, and it is unrealistic to process this huge information as it is in digital form in the above-mentioned information media. For example, video telephony has been realized using an Integrated Services Digital Network (ISDN: Integrated Services Digital Network) with a transmission speed of 64kbps to 1.5Mbps, but the images of TVs and cameras cannot be sent as they are by ISDN.

Therefore, this requires information compression technology, for example, in the case of videophones, the animation compression technology of H.261 and H.263 standards internationally standardized by ITU-T (International Telecommunication Union, International Telecommunication Standardization Sector) is being used . In addition, according to the information compression technology of the MPEG-1 standard, it is also possible to write image information together with audio information on a normal music CD (Compact Disc).

Here, the MPEG (Moving Picture Experts Group, namely, the Moving Picture Experts Group) is an international standard for digital compression of moving image signals, and MPEG-1 compresses moving image signals to 1.5 Mbps, that is, the information of television signals is compressed to about 100%. one of the standards. In addition, the transmission rate for the MPEG-1 standard is mainly limited to about 1.5 Mbps. Therefore, in MPEG-2 standardized to meet the demand for high image quality, moving image signals are compressed to 2 to 15 Mbps.

In addition, at present, MPEG-4 with a higher compression rate has been regulated by the working group (ISO/IECJTC1/SC29/WG11) promoting the standardization of MPEG-1 and MPEG-2. In MPEG-4, not only can high-efficiency encoding be performed at a low bit rate from the beginning, but also a strong error prevention technology that can reduce subjective image quality degradation even if an error occurs in the transmission line is introduced. In addition, ISO/IEC and ITU have jointly promoted the standardization activities of JVT (Joint Video Team, the joint video team) as the next-generation image coding method, and the latest one is now called Joint Model 2 (JM2).

In JVT, unlike conventional moving picture encoding, an arbitrary picture (picture) can be selected as a reference picture from a plurality of pictures (pictures) as a forward reference picture. Here, the image represents a frame or a field.

FIG. 1( a ) is an explanatory diagram of image encoding in which encoding is performed with reference to an image selected from a plurality of reference images stored in a memory. FIG. 1( b ) is a structural diagram showing the structure of a memory for storing images.

As shown in FIG. 1(b), the memory is composed of a short-term storage memory and a long-term storage memory. The short-term storage memory stores a plurality of pictures decoded immediately before, which correspond to reference pictures of P pictures (forward predictive coded pictures) and B pictures (bidirectional predictive coded pictures) called MPEG-1 and MPEG-2. The long-term storage memory is used to store image signals for a longer period of time than the short-term storage memory.

Usually, the short-term storage memory is a FIFO (first-in-first-out) memory. When storing images exceeding the upper limit of the memory in the short-term storage memory, the image at the earliest time in the short-term storage memory is deleted, and a new one is stored in this area. image. Therefore, generally, when it is desired to refer to a reference image deleted from the memory using the FIFO structure, the reference image is moved from the short-term storage memory to the long-term storage memory in advance, and stored in the long-term storage memory. can be referred to for a long time. Long-term memory is a method of expressly specifying a storage area, and images stored in this area can be referred to unless the same area is designated for rewriting.

FIG. 1( a ) shows the prediction status at the time of picture encoding. A picture of picture number 2 refers to a picture of picture number 0 , and a picture of picture number 1 refers to a picture of picture number 0 or picture number 2 . Similarly, the picture of picture number 4 refers to the pictures of picture numbers 0 and 2, and the picture of picture number 6 refers to the picture of picture number 0. In addition, in the picture of picture number 5, pictures of picture numbers 0, 2, 4, and 6 can be referred to.

In addition, in this FIG. 1( a ), images with image numbers 0, 6, and 12 can still be referred to after a relatively long time, and images with image numbers 2, 4, and 8 are only referenced by images after a short time. refer to. Therefore, as shown in Figure 1(b), the memory area for storing images is divided into short-term storage memory and long-term storage memory, and images (frames) numbered 0, 6, 12 images.

In addition, in order to efficiently use the memory shown in FIG. 1( a ), advanced memory management is required, and a structure for controlling the memory is introduced into the JVT.

The instructions to control the memory are as follows:

1. An instruction to select an image that can be referred to;

2. An instruction to release a memory area in the short-term storage memory, the memory area is to save an unnecessary image memory area as a reference image for predictive coding;

3. An instruction to move the contents of the short-term storage memory to the long-term storage memory.

In image encoding and decoding, since an image with a small prediction error is selected as a reference image in block units from among available reference images, a signal indicating a reference image in block units is required. By selecting a picture that can be referred to in advance and compressing the number of reference picture candidates to an appropriate value, it is possible to save the number of bits of a reference picture designation signal required in units of blocks.

Also, when moving from the short-term storage memory to the long-term storage memory, it is wasteful to store the same content in both the short-term storage memory and the long-term storage memory, so images in the short-term storage memory are deleted.

Fig. 2(a)(b) is a flowchart showing a conventional image coding method and image decoding method.

FIG. 2( a ) shows the operation of the image encoding device when freeing a memory area that stores pictures that are unnecessary as reference pictures for predictive encoding. In FIG. 2( a ), first, the image encoding device encodes the input image (step 100 ). After encoding, unnecessary areas (pictures not to be referred to in subsequent encoding) are checked in the memory (step 101), and it is determined whether there is an unnecessary memory area (step 102). When it is determined that there is an unnecessary memory area (Yes in step 102), an instruction to release the unnecessary memory area is coded as memory management information (step 103), and the unnecessary memory area is released (delete image in memory) (step 104), end processing. On the other hand, when the image encoding device determines that there is no unnecessary memory area (No in step 102), the operations in steps 103 and 104 are not performed, and the processing is terminated.

Next, the operation of the image decoding device when freeing a memory area storing unnecessary images as reference images for predictive encoding will be described according to the flowchart of FIG. 2(b). First, the image decoding device decodes the memory management information (step 110), and then decodes the image signal according to the coded signal (step 111). The image decoding device judges whether the result of inspection has memory release instruction (step 112), if there is memory release instruction (step 112 Yes), just judges whether there is an image that should be eliminated by this instruction, or whether the memory has been released (image Eliminated) (step 113). If it is determined that the release has been completed (Yes in step 113), it is set to error (ERROR). This is because, in the JVT, it is prohibited to issue a command to erase the same image again after the image has been erased from the memory, and therefore, an error occurs when the memory that has been released is released again. On the other hand, if the image decoding device judges that the memory has not been released (No in step 113), it releases the memory (step 114), and then ends the process. In addition, when it is determined that there is no memory release command (No in step 112), the operations in steps 113 and 114 are not performed, and the process ends. Furthermore, the order of step 110 and step 111 is different, and the order of the two steps can be interchanged.

3(a)(b) are flowcharts showing other conventional image coding methods and image decoding methods.

Fig. 3(a) shows the operation performed by the image encoding device when moving an image from the short-term storage memory to the long-term storage memory.

In FIG. 3( a ), first, the image encoding device encodes an input image (step 120 ). After encoding, it is checked whether there is an image that should be moved to the long-term storage memory (step 121), and it is determined whether there is an image that should be moved (step 122). If there is an image that should be moved (Yes in step 122), the instruction of how to move to the long-term storage memory is encoded as memory management information (step 123), and the image is moved to the long-term storage memory according to the instruction (step 124) ), and then end processing. On the other hand, when the image encoding device determines that there is no image to be moved to the long-term storage memory (No in step 122), it does not perform the operations in steps 123 and 124, and ends the process.

Next, the operation of the image decoding device when moving an image from the short-term storage memory to the long-term storage memory will be described in accordance with the flowchart of FIG. 3(b). First, the image decoding device decodes the memory management information (step 130), and then decodes the image signal according to the coded signal (step 131). Then, the image decoding device judges whether there is an instruction to move to the long-term storage memory in the decoded memory management information (step 132), and if it is judged to be (Yes in step 132), then it is judged whether there is an instruction to press the The image to command movement, or whether it has been moved (the image does not exist because it has been eliminated after the movement) (step 133). In the JVT, it is prohibited to issue a command to move the same image to the long-term storage memory again after moving to the long-term storage memory. Therefore, when moving an image that has already been moved to the long-term storage memory to the long-term storage memory again, Just set it to error. Therefore, if the image decoding device determines that the movement to the long-term storage memory has been completed (Yes in step 133), it will be set as an error (ERROR); if it is determined that the movement has not been completed, it will move to the long-term storage memory (step 134) , then end processing.

On the other hand, when the image decoding device determines that there is no command to move to the long-term storage memory (No in step 132), it does not perform the operations in steps 133 and 134, and ends the processing. Furthermore, the order of step 130 and step 131 is different, and the order of the two steps can be exchanged.

4(a), (b) are flowcharts showing another conventional image coding method and image decoding method.

First, the operation performed by the image encoding device when selecting a referenceable image will be described according to the flowchart of FIG. 4( a ).

First, the image coding apparatus selects a reference picture (usually a temporally close reference picture) that is expected to be closely related to the picture to be coded, as a candidate for the reference picture (step 200). Next, the instruction information (a type of memory management information) indicating the selected reference image candidates is encoded (step 201), and an appropriate reference image is referred to in block units from the selected reference image candidates, and the encoding is performed ( Step 202), and then end the processing. Furthermore, the order of step 201 and step 202 is different, and the order of the two steps can be exchanged.

Next, the operation of the picture decoding device when selecting a referenceable picture will be described according to the flowchart of FIG. 4(b).

First, the image decoding device decodes instruction information, which is a type of memory management information (step 210), and as a result, selects a candidate for a reference picture from the memory (step 211), and selects a reference picture from the selected candidate for reference picture Select an appropriate reference picture in units of blocks to refer to and decode (step 212), and then end the process.

In addition, in such a conventional image encoding method and image decoding method, an instruction to delete an unnecessary image from the memory and an instruction to move an image from the short-term storage memory to the long-term storage memory are executed by the image encoding device. After being encoded, it is output and transmitted to the image decoding device for decoding. However, since the number of transmissions is limited to one image, it cannot The image configuration in memory is correctly restored, so the image cannot be decoded.

In addition, in the encoding and decoding of pictures, when selecting a reference picture, if a temporally close picture is simply selected as a reference picture candidate, the variability (scalability) of the decoding of the picture cannot be considered (in the case of In the example of the prediction structure in Fig. 1(a), even if the B picture is not decoded, the I picture and the P picture can be decoded, or even if the P picture with picture numbers 4, 10, and 16 is not decoded, the other P pictures can also be decoded) optimal coding. That is, although the pictures temporally close to the picture of picture number 6 are pictures of picture numbers 4 and 2, in fact, only the picture of picture number 0 can be referred to. If a picture is included as a candidate for a reference picture, the coding efficiency is not so good.

In addition, in the conventional image coding method, an instruction to delete an unnecessary image in the memory along with transmission of an image not stored in the memory, or an instruction to move an image from the short-term storage memory to the long-term storage memory is prohibited. Therefore, Instruction transmission of flexible memory management information is prevented. This command is prohibited from being transmitted with images that are not stored in the memory for the following reasons. That is, this is because, since the image not stored in the memory has the lowest importance, it is highly likely that it cannot be decoded according to the scalability (scalability). The image configuration in the memory cannot be correctly restored due to being decoded.

Contents of the invention

Therefore, in order to solve the above problems, the present invention aims to provide an image encoding method, an image decoding method, etc., which can be correctly restored even if a part of the memory management information is lost due to a transmission line error, and which can be more appropriately selected and referred to. An image encoding method, an image decoding method, etc., which improve encoding efficiency by referring to candidates of an image.

In order to solve this problem, the image encoding method according to the present invention refers to a reference image selected from a plurality of reference images stored in the memory to perform encoding, and includes the following steps: the image encoding step refers to the selected reference image, and Encoding the encoding object image; encoding the management information encoding step, encoding the memory management information for controlling and managing the reference image stored in the above-mentioned memory along with the encoding object image encoded; the management information re-encoding step, and the above-mentioned management The encoding in the information encoding step encodes the above-mentioned memory management information again separately.

In this way, since the memory management information is coded and output multiple times, even if a transmission line error occurs when transmitting to the decoding device, it is considered that one of the memory management information to be transmitted multiple times is transmitted and cannot be transmitted. It is decoded, so the possibility of correctly restoring the image increases.

In addition, in the management information re-encoding step, the re-encoded memory management information may be accompanied by information specifying the encoding target image to be attached to the memory management information in the management information encoding step.

In this way, when a transmission error occurs when the first encoded memory management information is transmitted to the image decoding device along with the encoding target image, since the encoding target image accompanying the memory management information is identified, it is possible to detect at what time the transmission occurred. mistake.

In addition, in the step of encoding the management information, when the memory management information is attached to the image to be encoded that is not stored in the memory, in the step of re-encoding the management information, the memory management information may also be stored as an accompanying image. The encoding target image in the above memory.

In this way, since the important image stored in the memory after decoding is attached to the memory management information, the decoding of the memory management information can be performed accurately, and the possibility of correctly restoring the image increases.

In addition, the image decoding method of the present invention performs decoding by referring to a reference image selected from a plurality of reference images stored in the memory, wherein the memory management for controlling and managing the reference images stored in the memory Information is decoded, and based on the decoded memory management information, when releasing an unnecessary memory area in the above memory, if the memory area to be released has not been released, the memory area will be released; if the memory area to be released If it has been released, no processing will be done on the above memory.

In this way, even when the memory management information indicating that the image is deleted from the memory is received multiple times, the image can be correctly decoded without performing error processing.

In addition, in the image decoding method for decoding with reference to a reference image selected from a plurality of reference images stored in the memory, the memory includes a short-term storage memory whose storage time of the reference image is short, and a storage time of the reference image that is shorter than the reference image. In the long-term storage memory having a longer short-term storage memory, the image decoding method decodes memory management information for controlling and managing reference images stored in the memory, and based on the decoded memory management information, converts When the reference image stored in the memory is moved from the short-term storage memory to the long-term storage memory, if the reference image to be moved exists in the short-term storage memory, the reference image is transferred from the short-term storage memory to the short-term storage memory. The storage memory is moved to the long-term storage memory; if the reference image to be moved does not exist in the short-term storage memory, the movement in the memory is not performed.

In this way, even when the memory management information is received multiple times, the image can be correctly decoded without performing error processing.

In addition, in the image coding method for coding with reference to a reference picture selected from a plurality of reference pictures stored in the memory, the reference picture stored in the memory that is more important than the picture to be coded may be used as the reference picture. The alternates are encoded.

In this way, it is possible to more appropriately select candidates for pictures that can be referred to, thereby improving coding efficiency.

Furthermore, the image coding method according to the present invention is characterized in that it includes the steps of: coding the coding target picture; after coding the coding target picture, judging whether there is a reference picture that is not referred to in the memory ; If there is a reference picture that is not referred to above, as an instruction to release the unnecessary memory area due to not being referenced, after decoding the above-mentioned coding target picture in the decoding device for decoding coded data , a step of encoding an instruction indicating to release the above-mentioned unnecessary memory area; a step of releasing the above-mentioned unnecessary memory area; when encoding another encoding target image that is encoded later than the above-mentioned encoding target image, when encoding A step of encoding a command indicating to release the contents of the unnecessary memory area before decoding the other encoding target image.

In this way, even if the first instruction is missed, indicating that the release of an unnecessary memory area is missed, the next transferred instruction can be executed before the image is decoded, so that the delay in instruction execution can be reduced.

Furthermore, the image decoding method according to the present invention is characterized in that it includes the steps of: decoding memory management information for managing the memory accompanying the image to be decoded; and a first judging step of judging whether the memory management information is is an instruction before decoding, and the instruction before decoding indicates the processing of managing the memory before decoding the above-mentioned decoding target image; the second judging step is judged as the above-mentioned memory management information in the above-mentioned first judging step When it is the above-mentioned pre-decoding command, it is judged whether the processing of the management memory is completed; when it is judged that the processing of the management memory is completed in the above-mentioned second judgment step, the above-mentioned decoding target image is decoded, and in the above-mentioned second judgment step When it is determined that the process of managing the memory has not been completed, after the process of managing the memory is performed based on the memory management information, the decoding target image is decoded.

In this way, even if the first instruction is missed, indicating that the release of an unnecessary memory area is missed, the next transferred instruction can be executed before the image is decoded, so that the delay in instruction execution can be reduced.

In addition, the image encoding method according to the present invention is characterized in that it includes the following steps: a step of encoding the encoding target image; a judging step of judging whether all the reference images in the memory are is a non-referenced picture; when it is determined in the above-mentioned judging step that all the reference pictures in the memory are non-referenced pictures, a step of encoding an initialization command as a command to delete all the reference pictures in the memory ; The initialization step of deleting all the reference pictures in the above-mentioned memory; When encoding another encoding object image that is encoded later than the above-mentioned encoding object image, according to the additional information, the step of encoding the initialization retransmission instruction, said The additional information is to delete all reference pictures stored in the memory earlier than the picture to be encoded that are deleted when the picture to be coded is encoded, and the initialization retransmission command indicates to delete the reference pictures stored in the memory. The command.

In this way, when the initialization command is transmitted to the decoding device, even if the initialization command is missed due to a transmission line error, the initialization in the memory can be normally performed based on the additional information of the initialization retransmission command.

Furthermore, the image decoding method according to the present invention is characterized by including the steps of: decoding memory management information for managing memory accompanying the image to be decoded; and decoding the image to be decoded. Step; the initialization judging step, judging whether there is an order to delete all reference images in the memory in the above-mentioned memory management information, that is, an initialization instruction; The retransmission judging step, that is, in order to delete the reference picture, it is judged whether there is a command to delete the reference picture in the memory, that is, an initialization resend command, in the above-mentioned memory management information based on the additional information indicating the deletion object. The above-mentioned reference picture to be deleted is : A reference picture that should be initialized and deleted when decoding another decoding target picture that is decoded earlier than the decoding target picture, and stored in the memory before the decoding target picture; initialization After the judging step is completed, when it is judged in the above-mentioned initialization retransmission judging step that the above-mentioned memory management information is the above-mentioned initialization resend instruction, it is judged whether all reference images in the memory have been deleted; the deletion step is judged in the above-mentioned initialization completion judging step When not all reference images in the memory are deleted, the images in the memory are deleted based on the above additional information.

In this way, when the initialization command is transmitted to the decoding device, even if the initialization command is missed due to a transmission line error, the initialization in the memory can be normally performed based on the additional information of the initialization retransmission command.

In addition, in a recording medium on which a data stream encoded in units of slices is recorded with reference to a picture selected from a plurality of reference pictures stored in the memory, when the reference picture stored in the memory is deleted from the memory, the Information specifying a reference picture to be deleted is coded along with at least two slices.

In this way, in the case of encoding in units of slices, even if a transmission line error occurs when transmitting to the decoding device, due to the consideration of the error in the information specifying the reference picture to be deleted from the memory that has been transmitted multiple times, Since any one of them is transmitted and decoded, the possibility of correctly restoring an image on a slice-by-slice basis increases.

Furthermore, the recording medium of the present invention records a data stream coded in units of slices by referring to a picture selected from a plurality of reference pictures stored in the memory, and when the reference picture stored in the memory is deleted from the memory , encoding the information specifying the reference picture of the elimination object by attaching at least two slices; in addition, encoding the information indicating that the slice has the information of the information specifying the reference image of the elimination object along with the slice; When referring to information specifying a reference picture to be deleted in a slice that does not have information specifying a reference picture to be deleted, information indicating reference to information to specify a reference picture to be deleted may be encoded.

In this way, in a slice that does not have information specifying the reference picture to be deleted, the addition of the above-mentioned information can be omitted, and the coding efficiency can be improved.

As described above, according to the image coding method and the image decoding method of the present invention, even if a part of the memory management information is lost due to a transmission line error, the image coding method and the image decoding method that can be restored correctly can be realized, and further An image encoding method and an image decoding method for improving encoding efficiency by appropriately selecting candidates of reference images that can be referred to are highly practical.

Furthermore, the present invention can realize not only the image coding method and the image decoding method as described above, but also the image coding device and the image decoding device using the above method, and can also realize the recording of the image coded by the image coding method. The recording medium of the data stream can also realize a program for executing the steps in the image coding method and the image decoding method in a computer. Of course, such a program can also be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

In addition, this specification extracts the content of the former Japanese patent application "Japanese Patent Application No. 2002-110424", "Japanese Patent Application No. 2002-190955", "Japanese Patent Application No. 2003-49711" and the US application "60/377656".

Description of drawings

FIG. 1( a ) is an explanatory diagram of picture coding that is coded with reference to a picture selected from a plurality of reference pictures stored in a memory, and FIG. 1( b ) is a structural diagram showing the structure of a memory for storing pictures.

FIG. 2( a ) is a flowchart showing a conventional image coding method, and FIG. 2( b ) is a flowchart showing a conventional image decoding method.

FIG. 3( a ) is another flowchart showing a conventional image encoding method, and FIG. 3( b ) is another flowchart showing a conventional image decoding method.

FIG. 4(a) is another flow chart showing a conventional image encoding method, and FIG. 3(b) is another flow chart showing a conventional image decoding method.

FIG. 5 is a block diagram showing the structure of an image encoding device of the present invention.

FIG. 6 is a flowchart showing an image encoding method in Embodiment 1 of the present invention.

Fig. 7 is a block diagram showing the structure of an image decoding device of the present invention.

FIG. 8 is a flowchart showing an image decoding method in Embodiment 2 of the present invention.

FIG. 9 is a flowchart showing an image encoding method in Embodiment 3 of the present invention.

FIG. 10 is a flowchart showing an image decoding method according to Embodiment 4 of the present invention.

FIG. 11 is a flowchart showing an image coding method in Embodiment 5 of the present invention.

FIG. 12 is a flowchart showing an image coding method in Embodiment 6 of the present invention.

FIG. 13 is a flowchart showing an image coding method in Embodiment 7 of the present invention.

Fig. 14(a) is an explanatory diagram showing the relationship between picture numbers, stored picture numbers, and transfer order of pictures, and Fig. 14(b) is a picture number to be decoded, stored picture numbers, and deleted pictures As for the relationship diagram of the relationship between numbers, FIG. 14(c) is a relationship diagram showing other relationships among decoded picture numbers, stored picture numbers, and deleted picture numbers.

FIG. 15 is a correspondence diagram showing commands of memory management information in the present invention.

FIG. 16 is a flowchart showing the command execution procedure in Embodiment 8 of the present invention.

Fig. 17 is a schematic diagram showing the relationship between header information and frame data in the encoded signal of each image.

Fig. 18 is a schematic diagram showing commands of memory management information in header information of an encoded signal.

FIG. 19 is an explanatory diagram showing the relationship between the picture number of each picture, the stored picture number, and the transfer order.

FIG. 20 is a flowchart illustrating a method of encoding initialization instructions.

FIG. 21 is a flowchart illustrating a method of decoding encoded initialization commands.

FIG. 22 is a map showing commands of memory management information used in Embodiment 8 of the present invention.

FIG. 23 is a flowchart showing an image coding method using an initialization retransmission command in the present invention.

Fig. 24 is a flow chart showing the method of decoding the coded initialization resend command in the present invention.

FIG. 25 is an explanatory diagram showing another relationship among image numbers, stored image numbers, and transfer order of each image.

FIG. 26 is a map showing commands of memory management information used in Embodiment 9 of the present invention.

Fig. 27 is a flowchart showing an image coding method in Embodiment 9 of the present invention.

Fig. 28 is a flowchart showing an image decoding method according to Embodiment 9 of the present invention.

FIG. 29( a ) is a correspondence diagram showing the content of a command and additional information, and FIG. 29( b ) is a correspondence diagram showing execution timing of a command.

Fig. 30 is a schematic diagram showing a memory management information command in header information of an encoded signal.

Fig. 31 is a schematic diagram showing memory management information commands in header information of another coded signal.

Fig. 32 is a schematic diagram showing the structure of a data stream encoded in units of slices.

33(a) and (b) are schematic diagrams showing the structure of a data stream coded in units of slices.

34(a), (b) and (c) are explanatory diagrams of a storage medium for storing a program for a computer system to realize the image encoding method and image decoding in Embodiments 1 to 10 of the present invention method of procedure.

Fig. 35 is a block diagram showing the overall structure of a content supply system using the image coding method and the image decoding method of the present invention.

Fig. 36 is an external view showing an example of a mobile phone using the image coding method and the image decoding method of the present invention.

Fig. 37 is a structural block diagram showing the structure of the above-mentioned mobile phone.

Fig. 38 is a configuration diagram showing the configuration of a digital broadcasting system using the image coding method and the image decoding method of the present invention.

Detailed ways

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)

First, Embodiment 1 will be described.

FIG. 5 is a block diagram showing the configuration of an image encoding device for realizing the image encoding method in this embodiment.

The image coding device 100 includes a memory information control unit 101, a short-term storage memory management unit 102, a long-term storage memory management unit 103, a non-storage memory management information unit 104, a management information coding unit 105, a reference image selection unit 106, and a storage area designation unit 106. unit 107, reference area specifying unit 108, image memory 109, image decoding unit 111, image encoding unit 110, variable length encoding unit 112, counter 113, counter 114, and the like.

The reference picture selection unit 106 selects a candidate for a reference picture based on the importance indication signal Pri input from outside and the picture type information PicType, and notifies the memory information control unit 101 of the candidate.

The memory information control unit 101 judges whether one or both of the front and rear pictures (pictures) can be referred to based on the picture type information PicType, and instructs the reference area specifying unit 108 to output the corresponding reference from the picture memory 109 to the picture encoding unit 110. image.

The image encoding unit 110 refers to the reference image output from the image memory 109 to encode the input image signal Vin, and the variable length encoding unit 112 further performs variable length encoding to output the encoded image stream VideoStr. In addition, the output of the image encoding unit 110 is decoded by the image decoding unit 111 to become a decoded image, which is stored in the image memory 109 as a reference image.

At this time, the memory location where the decoded image can be stored in the image memory 109 is designated as follows. That is, the memory information control unit 101 inquires from the short-term storage memory management unit 102 to specify a memory location in the short-term memory where the image has been deleted, and the storage area specifying unit 107 instructs the image memory 109 to save the image at the memory location. Record the decoded image.

The short-term save memory management unit 102 detects unnecessary (non-referenced) images in the short-term save memory, and notifies the memory information control unit 101 of an instruction to delete (release the memory). Also, the long-term storage management unit 103 notifies the memory information control unit 101 of an instruction to move the images in the short-term storage memory to the long-term storage memory. The command to delete unnecessary images (release memory) and the command to move images in the short-term storage memory to the long-term storage memory are encoded by the management information encoding unit 105 into a memory management information stream CtlStr.

On the other hand, in order to prevent a part of the memory management information stream CtlStr from disappearing due to a transmission line error, thereby destroying the memory management information, the counter 113 for the short-term storage memory and the counter 114 for the long-term storage memory are used to measure unnecessary The number of encoding times of the command to delete the image and the command to move the image in the short-term storage memory to the long-term storage memory can be transmitted as many times as necessary.

In addition, the non-storage memory management information unit 104 manages whether an instruction to delete an unnecessary image or an instruction to move an image in the short-term storage memory to the long-term storage memory is accompanied by a low-importance and difficult-to-decode image that has been deleted. For encoding, when the above-mentioned command is encoded with an image of low importance, an instruction is given to the memory information control unit 101 to encode the command again with an image of higher importance.

Next, the image coding method in Embodiment 1 of the present invention will be described. FIG. 6 is a flowchart showing the image coding method in Embodiment 1, and shows operations performed by the image coding device 100 shown in FIG. 5 . In addition, in FIG. 6, the same code|symbol is attached|subjected to the part which operates the same as FIG. 2(a).

The picture encoding method shown in FIG. 6 is characterized in that, when there is an unnecessary picture (picture) in the memory as a reference picture for predictive coding, the process of releasing the memory area storing the picture (deleting the picture) is repeated. Instructions for memory management information are encoded. In this way, by repeatedly encoding the memory management information instructions, even if one of the memory management information instructions disappears due to a transmission line error, the image management information stored in the memory can be restored based on the other memory management information instructions. , therefore, the probability that the image can be correctly restored even if there is a transmission line error increases.

In FIG. 6, first, the input image is encoded (step 100). After encoding, unnecessary areas (pictures not to be referred to in subsequent encoding) are checked in the memory (step 101), and it is determined whether there is an unnecessary memory area (step 102). If there is an unnecessary memory area (Yes in step 102), the management information encoding unit 105 encodes a command to release the unnecessary memory area as memory management information (step 103). Then, the unnecessary memory area is released (step 104). If there is no unnecessary memory area (No in step 102), the operations in steps 103 and 104 are not performed.

Next, the memory information control unit 101 judges whether the command to release unnecessary memory area is accompanied by a picture coded immediately before (picture before the coding target) and coded as memory management information (step 105). If it is coded (No in step 105), the process ends, and if the command has been coded (Yes in step 105), the management information coder 105 again uses the command to release the unnecessary memory area as a memory management command. The information is encoded (step 106), and processing ends.

In this way, when an instruction to release unnecessary memory (memory management information) is encoded in the encoding of the immediately preceding image, an instruction to the memory management information is encoded again. The memory management information coded accompanying the coding of the immediately preceding picture and the memory management information coded again are respectively output from the picture coding device, and are transmitted to the picture decoding device to be decoded.

Furthermore, in step 105, if an instruction to release an unnecessary memory area is encoded along with the coded signal of the image that has been encoded immediately before, the instruction is coded again. In the case of a few pictures, the previous picture may be attached, and the above-mentioned command may be repeatedly encoded as memory management information, and may be transmitted together with a plurality of pictures.

In addition, it would be desirable if the command to release the above-mentioned unnecessary memory area can be transmitted as memory management information multiple times, and when the command is re-encoded and transmitted, it does not necessarily have to be transmitted along with the coded signal of the image.

In addition, in the case of retransmitting the memory management information command, the retransmitted command may not be in the same stream as the coded image, but may be transmitted as a separate stream, for example, or may be recorded in a storage medium. in another area.

As described above, since it can be considered that an instruction to release an unnecessary memory area (of memory management information) is transferred multiple times, even if a transmission line error occurs, one of the instructions transferred multiple times is transferred and Therefore, it is more likely that the image can be correctly restored due to decoding.

(Embodiment 2)

Next, Embodiment 2 of the present invention will be described.

7 is a block diagram of an image decoding device for realizing the image decoding method in Embodiment 2. FIG.

The image decoding device 200 is composed of a memory information control unit 201, a short-term storage memory management unit 202, a long-term storage memory management unit 203, a management information decoding unit 205, a storage area designation unit 207, a reference area designation unit 208, and an image memory 209. , an image decoding unit 210, a variable length decoding unit 212, and the like.

The memory information control unit 201 judges whether one or both of the front and rear pictures can be referred to for the coding target according to the picture type information PicType, and instructs the reference area specifying unit 208 to output the corresponding picture from the picture memory 209 to the picture decoding unit 210. Refer to image.

The variable length decoding unit 212 decodes the coded stream VideoStr, and the image decoding unit 210 further decodes it, outputs it as a decoded image signal Vout, and stores it in the image memory 209 as a reference image.

At this time, the memory location where the decoded image can be stored in the image memory 209 is designated as follows. The memory information control unit 201 inquires the short-term storage memory management unit 202 to determine the memory location of the deleted image, and the storage area specifying unit 207 issues an instruction to the image memory 209 to record the decoded image in the memory location.

The management information decoding unit 205 decodes the memory management information stream CtlStr, and notifies the short-term storage memory management unit 202 of unnecessary (non-referenced) image information in the short-term storage memory through the memory information control unit 201. The time saving memory management unit 203 notifies an instruction to move the images in the short time saving memory to the long time saving memory.

Next, the image decoding method in Embodiment 2 of the present invention will be described. FIG. 8 is a flowchart showing the image decoding method in Embodiment 2, and shows operations performed by the image decoding device 200 shown in FIG. 7 . In addition, in FIG. 8, the same code|symbol is attached|subjected to the part which operates the same as FIG. 2(b).

When the image encoding device has transmitted the command to release unnecessary memory areas multiple times, the image decoding device receives the command to release the same image area in the memory multiple times unless the command disappears due to a transmission line error. Therefore, it is necessary to realize such an image decoding method that, even if the image decoding device receives an instruction to release the memory area that has been released again, it does not treat it as an error, but instead judges that it can be received correctly. . In this embodiment, such an image decoding method is realized.

In FIG. 8, first, the management information decoding unit 205 decodes the memory management information (step 110). Next, the image signal is decoded according to the coded signal (step 111). Then, the memory information control unit 201 judges whether there is an instruction to release the memory in the decoded memory management information (step 112). If there is an instruction to release memory (Yes in step 112), it is judged whether there is an image that should be eliminated with this instruction, or whether it has released (eliminated) (step 113), if memory has released (Yes in step 113), End without doing any processing, if not, release memory (step 114), and end processing. On the other hand, when there is no memory release command (No in step 112), the operations in steps 113 and 114 are not performed, and the process ends. Furthermore, the order of step 110 and step 111 is different, and the order of the two steps can be interchanged.

According to the above-mentioned operation, even if the image decoding device 200 receives the signal multiple times by encoding and transmitting the command to release the area of the same image in the memory multiple times by the image encoding method of Embodiment 1, it does not act as Errors are handled so that an image decoding method capable of correct decoding can be realized.

In addition, it is only necessary to transmit the command to release the unnecessary memory area multiple times as the memory management information, and when re-encoding and transmitting the command, it is not necessary to transmit the command along with the coded signal of the image.

In addition, in the case of retransmitting the memory management information command, the retransmitted command may not be in the same stream as the coded image, but may be transmitted as a separate stream, for example, or may be recorded in a storage medium. in another area.

(Embodiment 3)

Next, an image encoding method in Embodiment 3 will be described. FIG. 9 is a flowchart showing the image coding method in Embodiment 3, and shows operations performed by the image coding device 100 . In FIG. 9 , the same symbols are assigned to the same parts as in FIG. 3( a ).

This embodiment is characterized in that, when there is an image to be moved from the short-term storage memory to the long-term storage memory in the memory, a command to move the memory management information of the image is repeatedly encoded. By repeatedly encoding the instructions of the memory management information, even if some of the instructions of the memory management information disappear due to a transmission line error, the management information of the image stored in the memory can be restored according to the instructions of another part of the memory management information. Therefore, even if there is transmission There is an increased possibility that the image can be correctly restored even if there is a line error.

In FIG. 9, first, the input image is encoded (step 120). After encoding, it is checked whether there is an image that should be moved to the long-term storage memory (step 121). Then, the memory information control unit 101 judges whether or not there is an image to be moved to the long-term storage memory (step 122). If there is an image to be moved (Yes in step 122), the management information encoding unit 105 encodes a command indicating how to move to the long-term storage memory as memory management information (step 123). Then, the image is moved to long-term storage according to the instruction (step 124).

Next, the memory information control unit 101 judges whether or not an instruction to move a coded signal that is attached to an image that has been coded immediately before (before the coding target) is coded as memory management information (step 125). image) signal. If not encoded (No in step 125), the process ends; if encoded (Yes in step 125), the management information encoding unit 105 encodes again the instruction to move to the long-term storage memory as memory management information. (step 126), and then end the process.

As described above, when the instruction to move to the long-term storage memory (for memory management information) is encoded in the encoding of the immediately preceding image, the instruction to encode the memory management information is encoded again. The memory management information coded accompanying the coding of the immediately preceding picture and the memory management information coded again are respectively output from the picture coding device, and are transmitted to the picture decoding device for decoding.

In addition, in step 125, when the instruction to move to the long-term storage memory has been encoded along with the encoded signal of the immediately preceding encoded image, the instruction is encoded again, but the instruction may not be attached to the immediately preceding image. In the case of a single image, a few images before the image are attached, and the above-mentioned command is repeatedly coded as memory management information, and a plurality of images may be attached.

In addition, as long as the command to move to the long-term storage memory can be transmitted multiple times as memory management information, it is not necessary to transmit the command along with the coded signal of the image when encoding and transmitting the command again.

In addition, in the case of resending the memory management information command, the resent command may not be in the same stream as the coded image, but may be transmitted as a separate stream, for example, or may be recorded in the storage medium. in another area.

As described above, since it can be considered that the instruction to move to the long-term storage memory is transferred multiple times, even if a transmission line error occurs, one of the instructions transferred multiple times is still transmitted and decoded, therefore, The possibility of correctly restoring an image increases.

(Embodiment 4)

Next, an image decoding method according to Embodiment 4 will be described.

In the case that the image coding device has transmitted the command to move to the long-term storage memory multiple times, the command will not disappear due to a transmission line error, and the image decoding device will receive and transfer the same image area in the short-term storage memory to the long-term storage multiple times. Instructions for saving memory moves. Therefore, it is necessary to implement an image decoding method that, even when the image decoding device receives an instruction to move the image again, does not treat it as an error, but instead determines that it can be received correctly. The image decoding method in this embodiment is characterized by realizing such an image decoding method.

FIG. 10 is a flowchart showing the image decoding method in Embodiment 4, and shows the operation of the image decoding device 200 shown in FIG. 7 . In FIG. 10 , the same symbols are assigned to the same parts as in FIG. 3( b ).

In FIG. 10, first, the management information decoding unit 205 decodes the memory management information (step 130). Then, the image signal is decoded according to the coded signal (step 131).

Then, the memory information control unit 201 determines whether or not there is an instruction to move an image to the long-term storage memory in the decoded memory management information (step 132). If there is an instruction to move to the long-term storage memory (Yes in step 132), then it is judged whether there is an image that should be moved by the instruction, or whether it has been moved (because there is no image because it has been eliminated after moving) (step 133) , if move to long-term storage memory (Yes in step 133), do not do any processing and end, if not like this, just end processing after moving (step 134) to long-term storage memory.

On the other hand, when there is no command to move to the long-term storage memory (No in step 132), the operations of steps 133 and 134 are not performed, and the processing is terminated. Furthermore, the order of step 130 and step 131 is different, and the order of the two steps can also be exchanged.

According to the above-mentioned operation, even if the image encoding method according to the third embodiment is used to encode multiple times and transmit an instruction to move the image to the long-term storage memory, an image decoding method capable of accurate decoding can be realized.

Furthermore, the command to move to the long-term storage memory may be transmitted multiple times as memory management information, and it is not necessary to transmit the command along with the coded signal of the image when re-encoding the command.

In addition, in the case of resending the memory management information command, the resent command may not be in the same stream as the coded image, but may be transmitted as a separate stream, for example, or may be recorded in the storage medium. in another area.

(Embodiment 5)

Next, the image encoding method in this embodiment will be described. FIG. 11 is a flowchart showing an image coding method in Embodiment 5, and shows operations of the image coding device 100 shown in FIG. 5 . In FIG. 11 , the same symbols are attached to the same parts as in FIG. 6 .

The present embodiment shown in FIG. 11 is characterized in that, when an unnecessary image exists in the memory, an instruction to delete the memory management information of the image is repeatedly coded, and an important image stored in the memory is attached at least once. to transfer. In the case of repeatedly encoding the memory management information command, even if the memory management information command is transmitted along with the less important pictures, the memory cannot be acquired until all the less important pictures are decoded. Instructions for managing information.

For example, in FIG. 1(a), since the picture of picture number 4 becomes unnecessary after encoding the picture of picture number 5, it is possible to release the picture of picture number 4 along with the picture of picture number 5. Instructions for a certain memory region are encoded.

However, in addition to encoding an instruction to release the memory area where the picture with picture number 4 is attached to the picture with picture number 5, when the picture with picture number 7 is coded, the accompanying importance is the lowest (when not In the case of decoding a B picture with less image quality degradation), the above command is coded. Sometimes these B pictures are not decoded, and the command to release the memory area where the picture number 4 is located is not decoded, and the management information in the memory cannot be reproduced correctly. Therefore, the instruction to release the image area must be encoded at least once along with the image stored in the memory which is highly important and must be decoded.

In FIG. 11, first, an input image is encoded (step 100). After encoding, unnecessary areas (pictures not to be referred to in subsequent encoding) are checked in the memory (step 101), and it is determined whether there is an unnecessary memory area (step 102). If there is an unnecessary memory area (Yes in step 102), the management information encoding unit 105 encodes a command to release the unnecessary memory area as memory management information (step 103). Then, the unnecessary memory area is released (step 104). When there is no unnecessary memory area (No in step 102), the operations of steps 103 and 104 are not performed.

Next, the memory information control unit 101 determines whether an important image is attached (stored in the decoded memory), and an instruction to release an unnecessary memory area that has been encoded in the past is encoded (step 140). After coding (Yes in step 140), the process ends. If no important image is coded (No in step 140), the management information coding unit 105 will release the instruction of the unnecessary memory area as the memory management information again. Encoding is performed (step 141), after which the processing ends.

In this way, instructions to free unneeded memory areas are coded along with important images.

As described above, since the above-mentioned command is accompanied by the important image decoded and stored in the memory, the possibility of correctly restoring the image when a transmission line error occurs increases by decoding the above-mentioned command.

In addition, as long as the command to release the unnecessary memory area can be transmitted multiple times as the memory management information, when re-encoding and transmitting the command, it is not necessary to transmit it along with the coded signal of the image.

In addition, in the case of retransmitting the memory management information command, the retransmitted command may not be in the same stream as the coded image, but may be transmitted as a separate stream, for example, or may be recorded in a storage medium. in another area.

(Embodiment 6)

Next, the image encoding method of this embodiment will be described. FIG. 12 is a flowchart showing an image coding method in Embodiment 6. FIG. FIG. 12 shows the operation of the image coding device 100 shown in FIG. 5 . In FIG. 12 , the parts that operate in the same way as in FIG. 9 are denoted by the same symbols.

The present embodiment shown in FIG. 12 is characterized in that memory management information instructions for moving images to the long-term storage memory are repeatedly encoded, and at least once are transmitted along with important images (decoded and stored in the memory). In the case of repeatedly encoding the memory management information command to move images to the long-term storage memory, even if the memory management information command is transmitted along with less important pictures, not all of the less important pictures are translated. In the case of code, the command to acquire memory management information cannot be performed.

In FIG. 12, first, the input image is encoded (step 120). After encoding, it is checked whether there is an image that should be moved to the long-term storage memory (step 121), and it is determined whether there is an image that should be moved (step 122).

If there is an image that should be moved (Yes in step 122), the management information encoding unit 105 has encoded the instruction how to move to the long-term storage memory as memory management information (step 123), and will move the image according to the instruction. to long-term storage memory (step 124).

Next, the memory information control unit 101 judges whether the instruction to move to the long-term storage memory that has been coded in the past is encoded with an important image (stored in the decoded memory) (step 150). Yes), the process ends, and if there is no important image attached (No in step 150), the management information encoding unit 105 encodes the instruction to move to the long-term storage memory as memory management information again (step 151), and then Finish processing.

In this way, the command to move the image to the long-term storage memory is coded along with the important image.

As described above, since the important image decoded and stored in the memory is attached to the above-mentioned command, the possibility of correctly restoring the image when a transmission line error occurs increases when the above-mentioned command is decoded.

In addition, the command to move to the long-term storage memory may be transmitted as memory management information multiple times, and when the command is re-encoded and transmitted, it is not necessary to transmit it along with the coded signal of the image.

In addition, in the case of resending the memory management information command, the resent command may not be in the same stream as the coded image, but may be transmitted as a separate stream, for example, or may be recorded in the storage medium. in another area.

(Embodiment 7)

The image encoding method in Embodiment 7 will be described.

The present embodiment is characterized in that it is an image encoding method for encoding with reference to a reference image selected according to the importance of the image.

FIG. 13 is a flowchart showing an image coding method in Embodiment 7 of the present invention. FIG. 13 shows operations performed by the image coding device 100 shown in FIG. 5 .

In FIG. 13, first, the importance of each image to be encoded is set (step 160). For example, the importance of I pictures and P pictures is high, and the importance of B pictures is low. Also, even with the same P picture, a P picture that is referred to by many pictures has a high degree of importance, and a P picture that is not very referenced has a low degree of importance.

Next, a picture whose importance is equal to or higher than that of the encoding target picture is selected from the reference pictures in the memory, and is used as a candidate for a reference picture (step 161). For example, a B-picture can refer to an I-picture and a P-picture, but a P-picture of low importance among the P-pictures is not a candidate for a reference picture.

Next, the instruction information (a kind of memory management information) indicating the candidates of the selected reference picture is coded (step 162), and an appropriate reference picture is referred to in units of blocks from the candidates of the selected reference picture and coded (step 163 ). Furthermore, the order of step 162 and step 163 is different, and the order of the two steps can also be exchanged.

In this way, a picture whose importance is lower than that of the encoding target picture is not added to the candidates of the reference picture.

As described above, since a picture whose importance degree is lower than that of the encoding target picture is not included in the candidates of the reference picture, when a stream capable of achieving scalability is generated, the unreferable stream can be eliminated. The picture is excluded from the candidates of the reference picture, and the encoding efficiency is improved.

Here, an image coding method based on the image importance set as described above will be specifically described with reference to FIG. 14 .

Fig. 14(a) shows the number assigned to each frame (image (frame) number), the number when each frame is stored in the memory (saved image (frame) number), and the number indicating the serial number of each frame transmitted (transmission order ) relationship diagram.

In FIG. 14( a ), the I picture with the picture number 0 is stored in the memory without referring to other pictures, so the stored picture number is 0. Next, since the P picture of picture number 2 referring to the I picture of picture number 0 is stored in the memory, the stored picture number related to the P picture of picture number 2 is 1. Then, since the B picture of picture number 1 that refers to the I picture of picture number 0 and the P picture of picture number 2 is stored in the memory, the stored picture number of the B picture of picture number 1 is 2. The serial number of each image transmitted is the serial number stored in the memory. Determine the relationship between image numbers, stored image numbers, and transfer order in the same order.

Next, an example of the relationship between the decoded picture number, the picture number stored in the memory, and the deleted picture number will be described with reference to FIG. 14(b).

FIG. 14( b ) is a relationship diagram showing the relationship between decoded picture numbers (frame numbers), stored picture numbers (frame numbers), and deleted picture numbers (frame numbers). Here, it is assumed that the maximum number of images that can be stored in the memory is five. Images are stored in memory in the order they were transferred.

In addition, for example, when the P picture whose picture number is 4 is decoded, since the saved picture number of the P picture whose picture number is 4 is 3, the pictures whose saved picture numbers are 0, 1, and 2 are stored in the memory. . When the B picture whose picture number is 3 to be decoded is decoded, as shown in FIG. 14(b), pictures with picture numbers 4, 1, 2, and 0 are stored. Here, as shown in FIG. 14(a), since the B picture with picture number 1 is no longer referred to by any picture after the picture with picture number 3 is decoded, the picture with picture number 3 is decoded. At this time, the B picture of picture number 1 is deleted.

Similarly, when a B picture whose picture number is 5 to be decoded is decoded, pictures with picture numbers 6, 3, 4, 2, and 0 are stored as shown in FIG. 14(b). Here, since the B picture of picture number 3 is no longer referred to by any picture after the picture of picture number 5 is decoded, the B picture of picture number 3 is deleted when the B picture of picture number 5 is decoded. B image.

Also, when the P picture whose picture number is 8 to be decoded is decoded, as shown in FIG. 14(b), pictures with picture numbers 5, 6, 4, 2, and 0 are stored. Here, since only a maximum of 5 frames can be stored in the memory, in order to refer to the P picture with picture number 8 later, it is necessary to ensure that one of the pictures with picture numbers 5, 6, 4, 2, and 0 is deleted and stored. Memory for P-picture number 8. Therefore, as a selection criterion for deleting a frame in FIG. 14( b ), at the time when the P picture of picture number 8 is decoded, the time frame during decoding of the P picture, that is, decoding of even-numbered picture numbers, is deleted. The oldest picture, in this case the I picture with picture number 0.

Similarly, when the B picture whose picture number is 7 to be decoded is decoded, pictures with picture numbers 8, 5, 6, 4, and 2 are stored as shown in FIG. 14(b). Here, after the B picture with picture number 7 is decoded, the B picture with picture number 5 is no longer referred to by any picture, therefore, when the B picture with picture number 7 is decoded, the picture with picture number 5 is deleted. B image.

Also, when the P picture whose picture number is 10 to be decoded is decoded, as shown in FIG. 14(b), pictures with picture numbers 7, 8, 6, 4, and 2 are stored. Here, since only a maximum of 5 frames can be stored in the memory, in order to refer to the P picture with picture number 10 later, it is necessary to ensure that one of the pictures with picture numbers 7, 8, 6, 4, and 2 is deleted and stored. Memory for image number 10. Therefore, as a selection criterion for deleting a frame in FIG. 14(b), at the time when the P picture with picture number 10 is decoded, the time frame during the decoding of the P picture, that is, the decoding of the picture with an even-numbered picture number, is deleted. The oldest image number on the image is 2.

In this way, when erasing an image, the memory management information command for erasing the image is coded and transmitted along with the encoded signal of the image to be decoded.

In the above-mentioned example shown in FIG. 14(b), an example in which there is an unnecessary image (image) in the memory and a memory management information command to delete the image is sent once has been described. As described above, if the memory management information command to be deleted is sent only once, there is a possibility that the memory management information command sent along with the B picture cannot be executed. This is because B pictures are less likely to be used as pictures to be referred to in encoding and decoding of P pictures, and therefore, when sufficient storage capacity and transmission capacity cannot be ensured, the data of B pictures is preferentially discarded. There is a high possibility that, as a result, the command accompanying the memory management information transmitted with the B picture may not be executed.

In order to solve this problem, an example in which a command to delete image memory management information is repeatedly encoded and transmitted will be described. Hereinafter, Fig. 14(c) will be specifically described.

FIG. 14( c ) is a relationship diagram showing another relationship among decoded picture numbers (frame numbers), stored picture numbers (frame numbers), and deleted picture numbers (frame numbers). FIG. 14(c) shows a case where a command to delete a picture of a picture number to be deleted is accompanied by a coded signal of a picture of a picture number to be decoded.

As shown in FIG. 14(c), when the B picture with the picture number 3 is decoded, the pictures with the picture numbers 4, 1, 2, and 0 are stored. Here, as shown in FIG. 14( a ), the B picture of picture number 1 is no longer referred to by any picture after the picture of picture number 3 is decoded. Therefore, when the picture with picture number 3 is decoded, the B picture with picture number 1 is deleted, and the memory management information command for deletion is attached to the picture with picture number 3 .

However, since the picture of picture number 3 is a B picture, it is less important in terms of picture reproduction than the I picture and the P picture as described above, and since the data is easily discarded at the time of transmission, there is a Possibility of not being able to execute a command (in the case of saving a frame as shown in FIG. 25 ), which is a command of the memory management information sent along with the B picture of picture number 3.

Therefore, the instruction indicating the deletion of the memory management information of the picture of picture number 1 attached to the picture number 3 is attached to the next decoded image whose importance is higher than that of the B picture of picture number 3 in terms of reproduction of the picture. P picture of picture number 6 (see FIG. 14(c)).

Similarly, an instruction (indicating deletion of a picture of picture number 3) to be attached to the memory management information of the B picture of picture number 5 is attached to the P picture of picture number 8; The instruction of (showing deletion of the picture of picture number 5) is accompanied by the P picture of picture number 10. Furthermore, since the picture with picture number 8 is a P picture, as shown in FIG. can be attached.

As described above, as shown in FIG. 14(c), the command of the same memory management information as the command of the memory management information attached to the first B picture is repeatedly attached to the B picture stored later than the command to first attach the memory management information or The transmitted picture, and the repeated accompanying picture is a picture whose importance is higher than that of the B picture in picture reproduction. In this way, even if the B picture accompanying the command of the memory management information at first is omitted, the command of the memory management information can be executed normally.

Furthermore, as described with reference to FIG. 14(c), even when the memory management information command is attached to the B picture and the memory management information command is repeatedly attached to the P picture, the set importance is used. . In addition, the setting of the degree of importance is not limited to the form shown in this embodiment.

In addition, in this embodiment, it is not determined whether to transmit each image based on the importance of each image, and the importance of each image cannot be encoded with each image as in the memory management information shown in the above-mentioned embodiment. . Therefore, the decoding process of encoded data in this embodiment is the same as the conventional method.

(Embodiment 8)

Next, Embodiment 8 will be described.

The present embodiment is characterized in that all images (images) in the memory are deleted, and a command (memory management information) for initializing the memory area is encoded and transmitted multiple times.

The memory management information shown in each of the above-described embodiments is given as code information as shown in FIG. 15 .

FIG. 15 is a map showing commands of memory management information, showing a code number (Code), content of the command (command), and its additional information (additional information).

For example, a command to release an unnecessary memory area in the short-term storage memory (release the short-term storage memory) is provided as code information Code1, and an image number (frame number) to be released is added as additional information.

In addition, code information is provided as header information of each frame shown in FIG. 17 .

FIG. 17 is a schematic diagram showing the relationship between header information and frame data in the encoded signal of each image. In FIG. 17 , the coded signals represent coded signals of frames Frm12 , Frm11 , and Frm14 described later. Each encoded signal includes a frame header including header information and frame data related to encoding of an image. For example, the encoded signal of the frame Frm12 includes a frame header Frm12Hdr and frame data composed of data MB12a, MB12b, MB12c, and MB12d.

The details of this coded signal are shown in the schematic diagram of FIG. 18 .

Fig. 18 is a schematic diagram showing commands of memory management information in header information of an encoded signal.

As shown in FIG. 18, the coded signal of the frame FrmA includes a frame header FrmAHdr having header information and frame data composed of respective data MBa, MBb, MBc, MBd, and the like. Then, the code information CodeA of the command is added to the frame header FrmAHdr, and then the additional information AddA of the code information CodeA is added, and then the code information CodeB of the command to be executed after the command of the code information CodeA and the additional information AddB of the code information CodeB are added. . If there is no additional information, just like the code information CodeC, only the code information is added.

Next, the sequence of instruction execution is shown in FIG. 16 .

Fig. 16 is a flowchart showing the instruction execution sequence.

In FIG. 16, first, a command is acquired (step C0), and it is determined whether or not the acquisition of the command has been completed (step C1). If the acquisition of the command is not completed and the command is acquired (No in step C1), the acquired command is executed (step C2), and the process returns to step C0, and this operation is repeated. On the other hand, when the acquisition of the command is completed and the command is not acquired (Yes in step C1), the command execution process is terminated. This process is done once for each frame. Note that even when command information is transmitted in units of slices composed of a plurality of macroblocks, commands are executed in the order described above.

Next, a command of the memory management information for deleting unnecessary images (freeing the memory) in the above-mentioned first embodiment will be described. In addition, in Embodiment 1, by repeatedly encoding the memory management information commands for eliminating unnecessary images, even if some of the memory management information commands disappear due to a transmission line error, it is still possible to use other memory management information commands. Restoring the image management information stored in the memory increases the probability that the image can be correctly restored.

Here, among the code information shown in FIG. 15 , the initialization command Code5 which erases all information in the memory will be discussed.

When the initialization command Code 5 is sent only once, if the initialization command Code 5 disappears due to a transmission line error, it will affect the memory management and other processing performed after the initial initialization. Here, referring to FIG. 19 , a case where the initialization command Code 5 is repeatedly encoded and transmitted as in the first embodiment will be described.

FIG. 19 is an explanatory diagram showing the number assigned to each frame (image (frame) number), the number when storing each frame in the memory (saved image (frame) number), and the number showing the serial number of each frame transferred ( order of transmission).

Hereinafter, FIG. 19 will be specifically described. First, the I picture with picture number 0 is stored in the memory because it does not refer to other pictures, and its stored picture number is 0. Next, since the P picture of picture number 2 referring to the I picture of picture number 0 is stored in the memory, the stored picture number related to the P picture of picture number 2 becomes 1. Then, since the B picture of picture number 1 that refers to the I picture of picture number 0 and the P picture of picture number 2 is stored in the memory, the stored picture number of the B picture of picture number 1 becomes 2. The serial number of each image transferred is set to the serial number stored in the memory. In the same order, determine the relationship between image numbers, stored image numbers, and transfer order.

It is set that the initialization command Code 5 shown in FIG. 15 is transmitted when encoding the I picture of picture number 12 shown in FIG. 19 . Since the stored picture number of the I picture of picture number 12 is 11, all pictures whose stored picture numbers are 10 or less can be deleted from the memory by using the initialization command Code5.

Here, a method of encoding the initialization command Code5 will be described using FIG. 20 .

FIG. 20 is a flowchart illustrating a method of encoding an initialization command Code5, and illustrates operations performed by the image encoding device 100 illustrated in FIG. 5 .

First, an input image is encoded (step A0). After encoding, check whether all images that can be referred to in the memory are unnecessary (whether they are not referenced by any image in subsequent encoding) (initialization check) (step A1), and determine whether the images stored in the memory are not used later. It is better to initialize with reference (step A2).

If the initialization is completed (Yes in step A2), the initialization command Code5 for initializing the memory area is used as memory management information, encoded (step A3), and initialized (step A4), and the process ends. On the other hand, when initialization is not necessary (No in step A2), the operation of step A3 and step A4 is not performed, and the process ends.

Next, a method of decoding the coded initialization command Code5 will be described with reference to FIG. 21 .

FIG. 21 is a flowchart showing a method of decoding the coded initialization command Code5, and shows operations performed by the image coding device 200 shown in FIG. 7 .

First, the memory management information is decoded (step A10), and the image signal is decoded from the coded signal (step A11). Then, determine whether there is initialization instruction Code5 (step A12) in the memory management information after the decoding, if there is initialization instruction Code5 (Yes of step A12), then all the images stored in the memory are eliminated, and initialize (step A12) A13), end processing. However, the decoded image is not deleted at this time (in step A11).

On the other hand, if there is no initialization command Code 5 in the memory management information (No in step A12), the process ends.

Hereinafter, a method of initializing the memory will be specifically described with reference to FIG. 19 . It is assumed that the same initialization command Code 5 as the initialization command Code 5 given to the I picture of picture number 12 is given to the B picture of picture number 11 shown in FIG. 19 .

As shown in FIG. 17, the initialization command Code5 is given to the frame header Frm12Hdr of the frame Frm12 (picture number 12) and the frame header Frm11Hdr of the frame Frm11 (picture number 11). Since the initialization command Code5 does not have additional information as shown in FIG. 15, all images stored in the memory are erased at the time of decoding.

Therefore, if the initialization command Code 5 of the I picture assigned the picture number 12 (saved picture number 11) disappears due to a transmission line error, and the initialization command Code 5 of the B picture assigned the picture number 11 (saved picture number 12) is executed, then Among the pictures decoded before the picture number 11 is stored, all the pictures stored in the memory are deleted. That is, the I picture of picture number 12 (stored picture number 11) that should not be deleted is also deleted.

As described above, when the B picture of picture number 11 is given the same initialization command Code 5 as the initialization command Code 5 given to the I picture of picture number 12, one picture (I picture of picture number 12) is omitted. On the other hand, if the P picture with picture number 14 (stored picture number 13) is given the same initialization command Code 5 as the initialization command Code 5 given to the I picture with picture number 12 (stored picture number 11), and the P picture with picture number 12 The initialization command Code5 assigned to the I picture disappears due to a transmission line error, and the initialization command Code5 assigned to the P picture with picture number 14 is executed, resulting in the omission of two pictures (B picture with picture number 11 and I picture with picture number 12) .

Also, when the initializing command Code5 is coded repeatedly, and the initializing command Code5 transmitted first and the initializing command Code5 transmitted next are all executed without transmission line error, the same problem as above occurs. This is because initialization is performed by the initialization command Code5 sent first, and initialization is performed again by the initialization command Code5 sent next.

A method for solving such a problem in initializing the memory will be described.

FIG. 22 shows commands of memory management information used to solve problems in memory initialization.

The difference from FIG. 15 is that the initialization retransmission command Code6 is newly added in FIG. 22 . Also, this initialization retransmission command Code6 has an initialization image (frame) number (the number of the frame accompanying the initialization command Code5 that initializes the memory area) as additional information.

Hereinafter, the flow of the image coding process using this initialization retransmission command Code 6 will be described with reference to FIG. 23 .

FIG. 23 is a flowchart showing an image encoding method using the initialization retransmission command Code6, showing operations performed by the image encoding device 100 shown in FIG. 5 . In FIG. 23 , the same operations as in FIG. 20 are assigned the same symbols.

First, an input image is encoded (step A0). After encoding, it is checked whether all pictures that can be referred to in the memory are unnecessary (whether any picture will not be referred to in subsequent encoding) (initialization check) (step A1). The memory information control unit 101 determines whether initialization is necessary (step A2), and if initialization is necessary (Yes in step A2), the management information encoding unit 105 encodes the initialization command Code 5 for initializing the memory area as memory management information (step A3), and Initialization is performed (step A4). When initialization is unnecessary (No in step A2), the operations of steps A3 and A4 are not performed.

Next, the memory information control unit 101 judges whether or not the coded signal accompanying the coded picture (picture earlier than the coding target) is coded as memory management information with the initialization command Code 5 for initializing the memory area (step A30), If encoding is performed (Yes in step A30), the management information encoding unit 105 encodes the initialization resend command Code6 for initializing the memory area as memory management information (step A31), and then ends the process.

In addition, if there is no coded signal accompanying the picture coded immediately before (picture earlier than the coding target), the initialization command Code 5 for initializing the memory area is coded as memory management information (No in step A30), and the process ends.

In addition, in the method shown in FIG. 23, when the initialization command Code 5 for initializing the memory area is coded accompanying the coded signal of the image coded immediately before, the initialization retransmission command Code 6 is coded again, but it is also possible When encoding the initialization command Code 5 for initializing the memory area not following the encoding of the immediately preceding encoded picture but following the encoding of an image that has been encoded several pictures before, the initialization retransmission command Code 6 is encoded again. , In addition, it is also possible to repeatedly perform encoding using the initialization resend command Code 6 for initializing the memory area as memory management information along with a plurality of images.

Specifically, as shown in FIG. 19 , when the initialization command Code 5 is encoded along with the encoding of the I picture of the picture number 12, the initialization retransmission command Code 6 may be accompanied by the encoding of the B picture of the picture number 11. The encoding is performed, and the initialization retransmission command Code 6 may be encoded along with the encoding of the P picture with the picture number 14.

In the former case shown in FIG. 17, the initialization command Code5 is given to the frame header Frm12Hdr of the frame Frm12, and the initialization retransmission command Code6 is given to the frame header Frm11Hdr of the frame Frm11. In the latter case, the initialization command Code5 is given to the frame header Frm12Hdr of the frame Frm12, and the initialization retransmission command Code6 is given to the frame header Frm14Hdr of the frame Frm14.

Alternatively, the initial retransmission command Code 6 may be encoded with the encoding of the P picture with picture number 14 at the same time as the encoding of the B picture with picture number 11 and the initial retransmission command Code 6 . In this case, as shown in FIG. 17, the initialization command Code5 is given to the frame header Frm12Hdr of the frame Frm12, and the initialization retransmission command Code6 is given to the frame header Frm11Hdr of the frame Frm11 and the frame header Frm14Hdr of the frame Frm14.

Next, the process of decoding the data encoded with the initialization retransmission command Code 6 will be described with reference to FIG. 24 . FIG. 24 is a flowchart showing a method of decoding the coded initialization retransmission command Code6, showing the operation of the image decoding device 200 shown in FIG. 7 . In FIG. 24 , the same reference numerals are attached to the same parts as in FIG. 21 .

First, the management information decoding unit 205 decodes the memory management information (step A10). Then, the image signal is decoded based on the coded signal (step A11).

Determine whether there is initialization instruction Code5 (step A12) in the memory management information after decoding, if there is initialization instruction Code5 (Yes of step A12), then all eliminate the image in memory, and initialize (step A13), if not Initialization instruction Code5 (No in step A12) does not perform initialization.

Next, the memory information control unit 101 judges whether or not there is an initialization resend command Code 6 in the memory management information (step A40). If there is no initialization retransmission command Code6 (No in step A40), then end the process, if there is an initialization retransmission command Code6 (Yes in step A40), then check whether the initialization is complete (step A41). If the initialization has been completed (Yes in step A41), the process is ended, and if initialization has not been carried out (No in step A41), the initialization frame (accompanying by the initialization instruction Code5 of the initialization memory area) is deleted based on the additional information of the initialization retransmission instruction Code6 frame) before saving the frame (at the time of encoding the initialization frame, refer to the frame stored in the image memory), and set the long-term storage memory size to 0 (step A42), and end the process. In addition, when the long-term save frame is not used, it is not necessary to set the size of the long-term save memory to 0.

Therefore, in the case where the initialization command Code 5 shown in FIG. 19 is encoded with the picture number 12 attached, and the initialization retransmission command Code 6 is encoded with the picture number 14, in the case where the initialization command Code 5 does not disappear due to a transmission line error Next, just by initialization command Code5, under the situation that initialization command Code5 disappears because of transmission line error, just by initialization retransmission command Code6, all delete the picture that saves the picture number below 10 among the pictures that are stored in memory.

In this way, since the initialization retransmission command Code 6 to which the initialization picture number is added as additional information is encoded and transmitted after the second time when the initialization command Code 5 is repeatedly encoded, the initialization is performed based on the additional information. All the saved frames before the frame (frames stored in the image memory for reference when encoding the initialization frame accompanying the initial initialization command Code5) are all deleted. Therefore, the above-mentioned problem of omission of necessary images (pictures) can be solved.

In addition, as shown in FIG. 25, the initialization retransmission command Code 6 explained above is effective even in a method of assigning a stored image number different from that in FIG. 19.

Below, it demonstrates concretely.

Fig. 25 shows the numbers assigned to each frame (image (frame) number), the number when each frame is stored in the memory (saved image (frame) number), and the number indicating the number of transmission of each frame (transfer order) An illustration of the relationship.

The method of assigning these numbers will be described. First, since the I picture with picture number 0 does not refer to other pictures, it is stored in the memory, and its stored picture number is 0. Next, since the P picture of picture number 2 referring to the I picture of picture number 0 is stored in the memory, the stored picture number related to the P picture of picture number 2 is 1. Then, the B picture of picture number 1 that refers to the I picture of picture number 0 and the P picture of picture number 2 is stored in the memory. The saved picture number of the P picture of picture number 2 is the same as 1. The serial number of each image transferred is set to the serial number stored in the memory. In the same order, determine the relationship between image numbers, stored image numbers, and transfer order.

As shown in FIG. 25 , it is assumed that the initialization command Code 5 shown in FIG. 15 is sent along with the encoding of the I picture of picture number 12 . Since the stored picture number of the I picture with picture number 12 is 6, all pictures whose stored picture numbers are 5 or less can be deleted from the memory by this initialization command Code 5 .

Here, when encoding the initialization command Code 5 repeatedly, specifically, the case where the same initialization command Code 5 as the initialization command Code 5 given to the I picture of the picture number 12 is given to the P picture of the picture number 14 is performed. illustrate.

As shown in FIG. 15, the initialization command Code 5 has no additional information, and therefore, when it is decoded, all images stored in the reference memory are erased. Therefore, if the initialization command Code 5 of the I picture assigned the picture number 12 (stored picture number 6) disappears due to a transmission line error, and the initialization command Code 5 of the P picture assigned the picture number 14 (stored picture number 7) is executed, All the images stored in the memory are deleted among the images stored under image number 6. That is, all I pictures of picture number 12 (stored picture number 6) that should not be deleted are all deleted.

However, since the above-mentioned initialization retransmission command Code 6 is attached to the P picture with picture number 14 instead of the initialization command Code 5, if the initialization command Code 5 accompanying the I picture with picture number 12 does not disappear due to a transmission line error, then From the initialization command Code5, when the initialization command Code5 disappears due to a transmission line error, all the pictures stored in the memory are deleted by the initialization retransmission command Code6 of the P picture with the picture number 14. The picture number stored in the memory is 5 images below.

That is, since the number of the initialization frame (in this case, picture number 12) is added to the initialization retransmission command Code6 as additional information, the saved frame before the initialization frame is deleted (saved in the image memory for reference when saving the initialization frame). Save frames in which the save image number is 5 or less).

As described above, due to the initialization retransmission command Code6 having additional information, even when the initialization command Code5 is missed due to a transmission line error, there is an increased possibility that initialization can be normally performed. Furthermore, the additional information may be used as the image number accompanying the initialization resend command, and the initialization resend command shown in this embodiment may be used instead of the initialization command, and Code5 and Code6 shown in FIG. 22 may be implemented with one command. This is because, when initial retransmission is performed for retransmission of initialization information, since the initializing command designates the number of the accompanying frame, the picture number for retransmitting the frame is not used. At this time, the initialization command Code5 may be invalidated.

Furthermore, as above, in the case where the initialization retransmission command and the initialization command Code5 shown in the above-mentioned embodiment are realized by a single command, the initialization retransmission command may be used as the command having the same function as the initialization command Code5 sent initially, The initial retransmission command is a command having, as additional information, a special value not used in the initial retransmission command described in the above-mentioned embodiment.

In addition, as described in each of the above-mentioned embodiments, when retransmitting memory management information such as instructions for releasing unnecessary memory areas and initialization instructions, as shown in FIGS. 17 and 18 , it may not be included in the encoding of images The related frame data is transmitted in the header information attached, but the header information included in the memory management information is transmitted separately from the frame data. That is, the retransmitted above-mentioned command is not in the same stream as the encoded image, but may be transmitted, for example, as a separate stream. In addition, it may also be recorded in another area of the storage medium.

In addition, in this embodiment, when retransmitting the initialization command, the picture number (initialization frame number) of the picture accompanying the initialization command first is added as additional information to the initialization retransmission command, but of course, it is also possible to In the case of the memory management information command shown in each of the above-mentioned embodiments, the picture number (information specifying the picture) of the coding target picture that is initially transmitted along with the command may be included as a parameter and transmitted. The memory management information is a command indicating a memory area to be released, a command specifying an image of an object to be moved from the short-term storage memory to the long-term storage memory, and the like. In this way, it is possible to detect which image a transmission line error has occurred in transmission.

(Embodiment 9)

Next, an image encoding method and an image decoding method in Embodiment 9 will be described.

The present embodiment is characterized in that when the memory management information is transmitted multiple times, the timing of processing based on the memory management information transmitted after the second time is changed.

When decoding data obtained by repeatedly encoding the memory management information described in the above-mentioned embodiments, it is necessary to decode an image signal accompanying the memory management information before processing the repeatedly transmitted memory management information. The case where a command to release an unnecessary memory area is transmitted multiple times as described in Embodiment 2 as a specific example will be described again with reference to FIG. 19 .

The code of Code 1 shown in FIG. 22 is encoded along with the picture of picture number 12 shown in FIG. 19 , and the code of Code 1 is also coded along with the picture of picture number 11 . At this time, decoding is performed according to FIG. 8 .

First, Code1 of the picture accompanying picture number 12 is decoded (step 110). Next, the picture with picture number 12 is decoded (step 111). Here, if the Code 1 that should have been attached to the image with the image number 12 is lost during transmission (No in step 112), the processing related to the frame is terminated.

In the transmission order, it is the picture of picture number 11 that is decoded after the picture of picture number 12.

First, Code 1 encoded with the picture with picture number 11 is decoded (step 110). Next, the picture with picture number 12 is decoded (step 111). When this Code 1 is transmitted without being missed during transmission, since there is a memory release command Code 1 in the decoded memory management information (Yes in step 112), it transitions to the following processing (step 113).

Here, since the memory is not released when the picture of picture number 12 decoded before the picture of picture number 11 is decoded (No in step 113), memory release processing is performed (step 114) .

As shown in the above specific example, the instruction to be executed for the image (image number 12) that should not be executed in the first instruction is accompanied by the operation of transmitting the instruction to release the unnecessary memory area multiple times, and the image sent from behind The decoding process of the image signal of (image number 11) is executed later, causing a delay in command execution.

Therefore, in this embodiment, a method for solving the above-mentioned problems will be described with reference to FIGS. 26 , 27 and 28 .

FIG. 26 is a correspondence diagram showing the relationship between memory management information and commands used in this embodiment.

In FIG. 26, Code represents the number of the command, the command represents the content of the command, the additional information represents additional information added to the command, and the processing position represents the timing at which the command is executed.

The difference from FIG. 15 is that in FIG. 26, CodeA1 to CodeA4 are set as commands executed after the decoding process of the image, and on the other hand, CodeA6 to CodeA9 corresponding to CodeA1 to CodeA4 are set as commands executed after the image is decoded. The decode process executes the instruction before.

Then, in the case of repeatedly sending the memory management information, the instruction of the first coded memory management information is set as the instruction (from CodeA1 to CodeA4) whose processing position is decoded (executed after decoding of the image), and A command to perform encoding repeatedly (after the second time) is set as a command (from CodeA6 to CodeA9) whose processing position is before decoding (executed before decoding of an image).

In this way, even if the first sent memory management information is missing, the instructions that should be executed according to the first sent memory management information are executed early, and problems such as delay are not likely to be caused.

Hereinafter, the processing procedure when using the command of FIG. 26 will be described using FIG. 27 and FIG. 28 .

FIG. 27 is a flowchart showing the image coding method in this embodiment, showing the operation of the image coding device 100 shown in FIG. 5 .

In Fig. 27, first, an image is encoded (step B0). After encoding, unnecessary areas in the memory (pictures not to be referred to in subsequent encoding) are checked (step B1), and it is determined whether there is an unnecessary memory area (step B2). If there is an unnecessary memory area (Yes in step B2), the instruction to release the unnecessary memory area is used as an instruction to be executed after the decoding of the image signal, and the memory management information used after decoding is encoded (step B3 ), release the unnecessary memory area (step B4). On the other hand, when there is no unnecessary memory area (No in step B2), the operations of steps B3 and B4 are not performed.

Next, the memory information control unit 101 determines whether or not to encode an instruction to release an unnecessary memory area accompanying encoding of a previously encoded image (an earlier image than the encoding target) as memory management information (step B30). If not coded (No in step B30), the process ends, and if coded (Yes in step B30), the management information coder 105 will release the instruction of the unnecessary memory area as an instruction to be executed before the decoding of the image signal. command to encode the memory management information before decoding (step B31), and the process ends.

Furthermore, in step B30, in the case where the instruction to release unnecessary memory areas is encoded along with the coded signal of the immediately preceding coded image, the instruction is coded again, but it does not need to be accompanied by the immediately preceding coded signal. In the case of an image, it is accompanied by several images before the image, and the above command can also be repeatedly encoded as memory management information, and transmitted with a plurality of images.

Next, the procedure for decoding the data encoded in the procedure shown in FIG. 27 will be described with reference to FIG. 28 and FIG. 19 .

FIG. 28 is a flowchart showing the image decoding method in this embodiment, showing operations performed by the image decoding device 200 shown in FIG. 7 .

In the following description, in FIG. 19 , it is assumed that the code of Code A1 shown in FIG. 26 is coded with the image number 12 attached, and that the code of Code A6 is coded with the code of the image number 11. . As shown in FIG. 17, CodeA1 is assigned to the frame header Frm12Hdr of the frame Frm12 of the image number 12, and CodeA6 is assigned to the frame header Frm11Hdr of the frame Frm11 of the image number 11.

Furthermore, in the image decoding device, unless the instruction is lost due to a transmission line error, the instruction to release the same image area in the memory is received multiple times. Therefore, it is necessary for the image decoding method performed by the image decoding device not to treat it as an error when receiving an instruction to release an already released image again, but to judge that it can be received correctly on the contrary.

First, the decoding process for the picture with picture number 12 will be described.

In FIG. 28, first, the memory management information of the picture of picture number 12 is decoded (step B5), and it is checked whether the memory management information is the memory management information used before decoding (step B7). Here, since this memory management information (CodeA1) is memory management information for decoding (No in step B7), the image signal of image number 12 is decoded. Then, as described above, since the memory management information (CodeA1) is the memory management information for decoding (Yes in step B9), the memory is released (step B11), and the memory management information related to the picture of picture number 12 is terminated. processing.

On the other hand, when the CodeA1 of the memory management information is missing, it is not judged to be the memory management information before decoding in step B7 (No in step B7), and it is not judged to be decoded in step B9. Only the image signal of image number 12 is decoded (step B6), and the processing related to the memory management information of image number 12 ends.

Next, the decoding process for the frame of picture number 11 will be described with reference to FIG. 28 .

First, the memory management information of picture number 11 is decoded (step B5), and it is checked whether the memory management information is memory management information for before decoding (step B7). Here, since CodeA6 is the memory management information used before decoding (Yes in step B7), it is checked whether the memory has been released (step B8). In the processing of image number 12, if CodeA1 is being executed, since the memory has been released (Yes in step B8), the memory release process (step B10) is not performed, and the decoding of the image signal of image number 11 is performed. (step B6). Then, it is judged whether the memory management information is the information for decoding (step B9), but since CodeA6 is the memory management information for before decoding (No of step B9), the memory management information for the image with the image number 11 is terminated. related processing.

However, if the memory management information of picture number 12 is missing due to a packet omission during transmission, etc., and the memory is not released in the process related to picture number 12, it is judged as The memory is not released (No in step B8), and in the next step, the memory is released (step B10). After the memory is released, the picture signal of picture number 11 is decoded (step B6). Then, since Code A6 is the memory management information for before decoding (No in Step B9), the process related to the memory management information of the picture with the picture number 11 ends.

As described above, by executing commands in the retransmitted portion prior to decoding the image signal, even if the first transmitted command is missed, the delay in command execution can be reduced.

In addition, as a specific example, the case where the memory management information is CodeA1 and CodeA6 has been described, but in the case of using CodeA2 and CodeA7, the same process can be realized. case, the same process can be used to achieve.

In addition, the initialization command CodeA5 shown in FIG. 26 may be used as memory management information after decoding, and the initialization retransmission command CodeA6 shown in FIG. 22 may be used as memory management information before decoding, and they may be paired. ground use.

In addition, when post-decoding memory management information and a plurality of pre-decoding memory management information are assigned as header information to one frame, the plurality of decoded memory management information may be processed before the post-decoding memory management information. Memory management information used before coding.

That is, it is also possible to encode the memory management information before decoding at the head of the header information shown in FIG. 17 .

In addition, it is also possible to use the combination of commands shown in FIG. 29(a) and FIG. 29(b) to determine whether the memory management information is management information for pre-decoding or management information for post-decoding as another information to implement the instructions shown in the above-mentioned embodiments.

Fig. 29(a) is a diagram showing the correspondence between the content of the command and the additional information. Fig. 29(b) is a corresponding diagram showing the execution timing (processing position) of an instruction.

Fig. 30 is a schematic diagram showing commands of memory management information in header information of an encoded signal.

In FIG. 30, the coded signal of the frame FrmB has a frame header FrmBHdr and frame data such as MBa, MBb, etc., and the frame header FrmBHdr has code information CodeD and the like as header information.

At this time, for example, as shown in FIG. 30 , the frame header FrmBHdr of the frame FrmB can be set from front to back as: code information CodeD of the instruction, FlagD indicating the processing position, and additional information AddD indicating additional information of the instruction. When there is no additional information, as shown in FIG. 30 , CodeE of the command and FlagE indicating the processing position may be added to the frame header FrmBHdr. The processing of Step B7 and Step B9 shown in FIG. 28 is optimized by setting Flag indicating the processing position instead of Add indicating additional information after the Code indicating the command.

In addition, in order to distinguish whether the execution timing of the instruction is before decoding or after decoding the image signal, a new instruction indicating the processing position of the instruction may be used, and a frame located at a position higher than that of the instruction indicating the processing position may be executed after decoding. Before decoding, an instruction located earlier on the header is executed on an instruction located later than the instruction indicating the processing position on the frame header. In this way, when there are multiple commands, the execution timing (processing position) of each command can be indicated by one command. Compared with the case of sending a Flag indicating the processing position for each command, the transmitted information is reduced and the encoding efficiency is improved.

A specific example will be described with reference to FIG. 31 .

Fig. 31 is a schematic diagram showing commands of memory management information in header information of another coded signal.

In FIG. 31, the coded signal of the frame FrmC has frame header FrmCHdr and frame data such as MBa and MBb. The frame header FrmCHdr is used as header information, and the order of the past is command CodeF, command dif, command CodeG, additional information AddG, and command CodeH.

Then, it is judged whether the instruction dif indicating the processing position is in the frame header FrmCHdr, the instruction CodeF located before the instruction dif indicating the processing position is executed after decoding of the frame FrmC, and the instruction located earlier than the instruction dif indicating the processing position is executed before decoding of the frame FrmC. The commands CodeG and CodeH at the rear of the dif are also acceptable. In this case, if there is no instruction dif indicating the processing position, all the instructions in the frame header FrmCHdr are executed after the decoding process of the frame FrmC.

Furthermore, as described in each of the above-mentioned embodiments, when retransmitting memory management information such as a command to release an unnecessary memory area and an initialization command, it may not be included in the header information added to the coded signal of the image. transmission, while the header information contained in the memory management information is transmitted separately from the coded signal of the image. That is, the above-mentioned retransmitted command may not be in the same stream as the coded image, but may be transmitted, for example, as a separate stream. In addition, it may also be recorded in another area of the storage medium.

(Embodiment 10)

Next, Embodiment 10 of the present invention will be described.

In this embodiment, the encoding unit is different from the above-mentioned embodiments. That is, in Embodiment 1 above, when an instruction to release an unnecessary memory area is transmitted multiple times, the memory management information stream CtlStr and the image coding stream VideoStr shown in FIG. The coding is performed in units of (pictures), but in this embodiment, one frame may be coded in units of slices like the stream structure shown in FIG. 32 .

The encoding in units of slices refers to encoding titles, memory management information streams CtlStr, and image encoding streams VideoStr for each slice of a frame, so that slice 1 of frame 1 in FIG. 32 has titles 1-1, ctlStr1, and VideoStr1 -1, slice 2 of frame 1 has title 1-2, ctlStr1, VideoStr1-2. After encoding in the image encoding device, the image encoding device outputs a data stream. Note that a slice is a synchronous reset unit, and is a band-shaped area composed of one or a plurality of blocks, and an image is composed of a plurality of slices. Also, a picture is a basic coding unit corresponding to one picture, and a block is a basic unit of coding and decoding.

In addition, it is assumed that the contents of the memory management stream CtlStr are the same information in the same frame when the memory management information stream CtlStr is transmitted multiple times as described above. By using the same information, it is possible to omit the addition of the memory management information stream CtlStr in units of slices. For example, information indicating whether the multi-transmission is omitted in the slice is added to the title of the slice. When commanded (when not omitted), "1" is added. Specifically, an example is shown in FIG. 33( a ), and will be described below. The title and image coded stream VideoStr differs from slice 1 to slice 3 in frame 1. On the other hand, slice 1 and slice 2 have the same memory management stream CtlStr1, and among the slices in the same frame, slice 1 and slice 2 respectively have information indicating the contents of the same memory management stream CtlStr1 encoded. 1". Also, the slice 3 has information "0" showing that the memory management stream CtlStr1 is omitted. In this way, when the information transmitted multiple times is omitted in this slice, the first slice and the like refer to the memory management information flow CtlStr in the slice indicated by "1" above, so that the addition of the memory management information flow CtlStr can be omitted, and the reduction can be reduced. number of digits.

That is, the information "0" indicating that the memory management information stream CtlStr1 is omitted is information indicating that the specifying information is referred to when the specifying information is referred to in a slice (slice 3) that does not have specifying information. The designation information described above is information for designating a target image to be erased.

Such an additional method of omitting the memory management stream CtlStr is effective because there is less possibility of the memory management stream CtlStr being missed many times during transmission.

In addition, if there is no information indicating that the memory management information flow CtlStr is omitted, the presence or absence of the memory management information flow CtlStr can be determined, but this information may be omitted as shown in FIG. 33(b). For example, when the header of the memory management information stream CtlStr can be distinguished from the header of the coded video stream VideoStr, as shown in FIG. To confirm whether there is information indicating whether the coded memory management information stream CtlStr1 has been coded or not.

Such a method of omitting the addition of the memory management stream CtlStr is effective for reducing the number of times of encoding the memory management stream CtlStr and reducing the number of bits.

Encoding has been described above, but one frame can be decoded on a slice-by-slice basis in the same manner. In Embodiment 2 above, when an instruction to release an unnecessary memory area is transmitted multiple times, in the image decoding device 200 shown in FIG. 7 , the image decoding device 200 shown in FIG. The stream structures of the information stream CtlStr and the coded image stream VideoStr are input in units of pictures (pictures), but may also be input in units of slices.

In addition, in encoding and decoding in other embodiments, one frame may also be encoded and decoded in units of slices.

Furthermore, the encoding method and decoding method shown in Embodiments 1 to 10 above can be implemented in mobile communication devices such as mobile phones, car navigation systems, digital video cameras, and digital cameras using semiconductors such as LSIs. In addition, as an installation form, in addition to a transmitting and receiving type terminal having two types of encoder and decoder, three forms of a transmitting terminal having only an encoder and a receiving terminal having only a decoder can be considered.

(Embodiment 11)

Next, Embodiment 11 of the present invention will be described.

In this embodiment, programs for realizing the image coding method or the image decoding method shown in Embodiments 1 to 10 are recorded on a storage medium such as a floppy disk, and can be executed on an independent computer system. The processing shown in the above-mentioned embodiment is simply carried out in .

Fig. 34 is an explanatory diagram showing a case where the image encoding method or the image decoding method according to the first embodiment is implemented by a computer system using a floppy disk storing the image encoding method or the image decoding method.

FIG. 34(b) shows the appearance, cross-sectional structure, and floppy disk viewed from the front of the floppy disk, and FIG. 34(a) shows an example of the physical format of the floppy disk as the main body of the recording medium. The floppy disk FD1 is housed in a case F. On the surface of the magnetic disk, a plurality of tracks Tr are concentrically formed from the outer circumference to the inner circumference, and each track is divided into 16 sectors Se in the angular direction. Therefore, on the floppy disk storing the above-mentioned program, an image encoding method as the above-mentioned program is recorded in a divided area on the above-mentioned floppy disk FD1.

Furthermore, FIG. 34(c) shows a structure for recording and playing the above-mentioned program on the floppy disk FD1. When the above-mentioned program is recorded on the floppy disk FD1, the image coding method or the image decoding method as the above-mentioned program is written from the computer system Cs through the floppy disk drive FDD. Furthermore, when the above-mentioned image encoding method is installed in the computer system by using the program in the floppy disk FD1, the program is read from the floppy disk FD1 by the floppy disk drive FDD and transferred to the computer system Cs.

In addition, in the above description, a floppy disk was used as the recording medium, but the same can be done using an optical disk. In addition, the recording medium is not limited thereto, and any IC card, ROM cassette, etc. can be similarly implemented as long as the program can be recorded.

In addition, the image coding method and image decoding method shown in the above-mentioned embodiments can be implemented in mobile communication devices such as mobile phones and car navigation systems, or imaging devices such as digital video recorders and ordinary video cameras using semiconductors such as LSI. In addition, as an installation form, in addition to two types of transmitting and receiving terminals having an encoder and a decoder, three types of a transmitting terminal having only an encoder and a receiving terminal having only a decoder can be considered.

Here, application examples of the image encoding method and image decoding method shown in Embodiments 1 to 10 above and a system using them will be described.

Fig. 35 is a block diagram showing the overall configuration of a content supply system ex100 for realizing a content delivery service. The area for providing communication services is divided into desired sizes, and base stations ex107 to ex110 serving as fixed wireless stations are installed in each cell.

The content supply system ex100 is connected to the Internet ex101 via, for example, an Internet service provider ex102, a telephone network ex104, and base stations ex107 to ex110. A computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, Various equipment such as mobile phone ex115 with video camera.

However, the content supply system ex100 is not limited to the combination shown in FIG. 35, and any one of them may be combined and connected. In addition, each device may be directly connected to the telephone network ex104 without passing through the base stations ex107 to ex110 which are fixed wireless stations.

The camera ex113 is a device capable of shooting animation such as a digital video recorder. In addition, portable phones are PDC (Personal Digital Communications, personal digital communication) method, CDMA (Code Division Multiple Access, code division multiple access) method, W-CDMA (Wideband-Code Division Multiple Access, broadband code division Multi-access) method, GSM (Global System for Mobile Communications, Global System for Mobile Communications) mobile phone, or PHS (Personal Handyphone System, personal hand-held phone system), etc., whichever is acceptable.

In addition, the streaming server ex103 is connected from the camera ex113 to the telephone network ex104 via the base station ex109, and can use the camera ex113 to perform live transmission based on encoded data sent by the user. The encoding processing of the captured data may be performed by the camera ex113, or may be performed by a server or the like that performs data transmission processing. In addition, video data captured by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device capable of shooting still images and movies, such as a digital video camera. In this case, encoding of video data may be performed by the camera ex116 or the computer ex111. In addition, encoding processing is performed in the LSI ex117 included in the computer ex111 and the camera ex116. Furthermore, the software for image encoding and decoding may be stored in any storage medium (CD-ROM, floppy disk, hard disk, etc.) that can be read by the computer ex111 or the like. In addition, video data can also be transmitted by the mobile phone ex115 with a video camera. The video data at this time is encoded by the LSI included in the mobile phone ex115.

In this content supply system ex100, on the one hand, the content captured by the user with the camera ex113, camera ex116, etc. The server ex103 transmits, and on the other hand, the streaming server ex103 stream transmits the above content data to the requesting client terminal. As client terminals, there are computers ex111, PDA ex112, video cameras ex113, mobile phones ex114, etc. capable of decoding the encoded data. By doing so, the content supply system ex100 can receive encoded data at the client terminal and play it back, and can receive, decode, and play it at the client terminal in real time, so it is a system that can realize personal broadcasting.

The image encoding method or image decoding method shown in the above-mentioned embodiments may also be used for encoding and decoding of each device constituting the system.

As an example, a mobile phone will be described.

FIG. 36 is a diagram showing a mobile phone ex115 using the image coding method and the image decoding method described in the above-mentioned embodiments. The mobile phone ex115 has the following parts: an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110; a camera unit ex203 capable of taking video and still images such as a CCD camera; and a display unit ex202 such as a liquid crystal display for displaying images captured by the camera unit ex203. Decoded data such as images and images received by the antenna ex201; a main body composed of a group of operation keys ex204; an audio output section ex208 such as a speaker for outputting audio; and an audio input such as a microphone for inputting audio Part ex205; recording medium ex207, used to store encoded data or decoded data such as photographed video or still image data, received mail data, video data or still image data; The card slot ex206 of the recording medium ex207 is inserted into the ex115. The recording medium ex207 is a flash memory element such as an SD card that is electrically loaded into a plastic case that can be rewritten and erased, that is, EEPROM (Electronically Erasable and Programmable Read Only Memory). device.

In addition, the mobile phone ex115 will be described with reference to FIG. 37 . The mobile phone ex115 is connected to the main control unit ex311 through the synchronous bus ex313, and the power supply circuit unit ex310, the operation input control unit ex304, the image coding unit ex312, the camera interface unit ex303, and the LCD (Liquid Crystal Display) control unit are connected to each other. ex302, image decoding unit ex309, multiple separation unit ex308, recording and playback unit ex307, modulation and demodulation circuit unit ex306, and sound processing unit ex305. The main control unit ex311 collectively controls each part of the main body including the display unit ex202 and the operation keys ex204.

The power supply circuit part ex310 activates the digital mobile phone with camera ex115 to be in an operable state by supplying power to each part from the power supply when the call is ended and the power key is turned on by the user.

The portable phone ex115 converts the voice signal collected by the voice input unit ex205 into digital voice data by the voice processing unit ex305 under the control of the main control unit ex311 composed of CPU, ROM and RAM, etc. It is subjected to spectrum diffusion processing by the modulation and demodulation circuit unit ex306, and is transmitted through the antenna ex201 after digital-to-analog conversion processing and frequency conversion processing are performed by the transceiver circuit unit ex301. In addition, when the cellular phone ex115 is in the voice communication mode, it amplifies the reception data received by the antenna ex201, performs frequency conversion processing and analog-to-digital conversion processing on the received data, and performs spectrum inversion by the modem circuit part ex306. Diffusion processing, converted into analog audio data by the audio processing unit ex305, and then output through the audio output unit ex208.

Also, when sending an e-mail in the data communication mode, the text data of the e-mail input by operating the operation keys ex204 of the main body is sent to the main control unit ex311 through the operation input control unit ex304. The main control unit ex311 performs spectrum diffusion processing on the text data by the modem unit ex306, performs digital-to-analog conversion processing and frequency conversion processing by the transceiver circuit unit ex301, and transmits it to the base station ex110 through the antenna ex201.

When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image coding unit ex312 through the camera interface unit ex303. In addition, when the image data is not transmitted, the image data captured by the camera unit ex203 may be directly displayed on the display unit ex202 through the camera interface unit ex303 and the LCD control unit ex302.

The structure of the image encoding unit ex312 includes the image encoding device described in this application, and compresses and encodes the image data supplied from the camera unit ex203 by using the encoding method used in the image encoding device described in the above-mentioned embodiments, and converts it into The coded image data is sent to the multiplexing unit ex308. At the same time, the mobile phone ex115 sends the voice collected by the camera unit ex203 by the voice input unit ex205 during shooting as digital voice data to the multiplexing unit ex308 through the voice processing unit ex305.

The multiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 in a predetermined manner, and the resulting multiplexed data is subjected to spectrum diffusion by the modem circuit unit ex306. The processing is performed through the antenna ex201 after digital-to-analog conversion processing and frequency conversion processing are performed by the transceiver circuit unit ex301.

In the case of receiving video image file data linked to the top page in the data communication mode, the reception data received from the base station ex110 through the antenna ex201 is subjected to spectral inverse diffusion processing by the modem circuit unit ex306, and the resultant multiplexed data is obtained. Send to the multiple separation unit ex308.

In addition, in order to encode the multiplexed data received via the antenna ex201, the multiplexed data is separated by the demultiplexing unit ex308 into an image data bit stream and an audio data bit stream, and the encoded image is supplied to the image decoding unit ex309 through the synchronous bus ex313. data, and supply the audio data to the audio processing unit ex305.

Next, the configuration of the image decoding unit ex309 includes the image decoding device described in the present invention, and decodes the bit stream of image data by using a decoding method corresponding to the encoding method shown in the above-mentioned embodiment. code to generate playback moving image data, and supply it to the display unit ex202 through the LCD control unit ex302, and thus display, for example, the moving image data included in the moving image file linked to the top page. At the same time, the audio processing unit ex305 converts the audio data into analog audio data, and supplies the audio data to the audio output unit ex208, so that, for example, the audio data included in the animation image file linked to the top page is played.

Furthermore, without being limited to the example of the above-mentioned system, recently, digital broadcasting using satellites and terrestrial waves has become a hot topic, and as shown in FIG. or image decoding means. Specifically, the broadcast station ex409 transmits a bit stream of video information to the communication or broadcast satellite ex410 via radio waves. The broadcast satellite ex410 receiving the bit stream transmits radio waves for broadcasting, and the radio waves are received by the home antenna ex406 equipped with satellite broadcast receiving equipment, and the bit stream is processed by devices such as a TV (receiver) ex401 or a set-top box (STB) ex407. Decode and play. In addition, the image decoding device shown in the above-mentioned embodiment may be installed in the device ex403 which reads and plays the bit stream recorded on the storage medium ex402 such as the recording medium CD or DVD. In this case, the broadcast video signal is displayed on the monitor ex404. In addition, a configuration may be considered in which an image decoding device is installed in a set-top box ex407 connected to a cable ex405 for cable TV or an antenna ex406 for satellite/terrestrial broadcasting, and broadcast on a TV monitor ex408. In this case, not only a set-top box device but also an image decoding device may be incorporated in a television. In addition, a car ex412 equipped with an antenna ex411 may receive signals from a satellite ex410, a base station ex107, etc., and an animation may be played on a display device such as a car navigation system ex413 included in the car ex412.

In addition, an image signal may be encoded by the image encoding device described in the above-mentioned embodiments, and recorded on a recording medium. As specific examples, there is a recorder ex420 such as a DVD recorder for recording image signals on a DVD disk ex421 and a disk recorder for recording on a hard disk. In addition, it can also be recorded in the SD card ex422. If the recorder ex420 has the image decoding device shown in the above-mentioned embodiments, the image signals recorded on the DVD disk ex421 and the SD card ex422 can be played back and displayed on the monitor ex408.

Furthermore, the structure of the car driving guidance system ex413 may be, for example, a structure in which the camera unit ex203, the camera interface unit ex303, and the image coding unit ex312 are removed from the structure shown in FIG. ) ex401 etc. to replace the removed part.

In addition, terminals such as the above-mentioned mobile phone ex114 may be of three types: a transmitting terminal having only an encoder and a receiving terminal having only a decoder, in addition to two types of transmitting and receiving terminals having encoders and decoders. installation form.

As described above, the image coding method or image decoding method shown in the above-mentioned embodiment can be used in any of the above-mentioned devices and systems, and the effects described in the above-mentioned embodiment can be obtained by using the present invention.

The present invention is not limited to the above-described embodiments, and various modifications and corrections can be made without departing from the scope of the present invention.

Possibility of industrial use

The image encoding device of the present invention is very useful as an image encoding device included in a personal computer having a communication function, a PDA, a digital broadcasting station, a mobile phone, and the like.

Also, the image decoding device of the present invention is very useful as an image decoding device included in a personal computer having a communication function, a PDA, an STB receiving digital broadcasting, a mobile phone, and the like.

Claims (4)

1. An image encoding method, encoding with reference to a reference image selected from a plurality of reference images stored in a memory, comprising: Encoding the encoding target image with reference to the selected reference image; encoding first memory management information for managing reference images stored in the memory; outputting the encoded first memory management information along with the encoded encoding target image; The first memory management information has been encoded, and when it is judged that the first memory management information is to be encoded and output again, the first memory management information is encoded again as the second memory management information, and will be encoded The second memory management information is attached to an encoded image different from the encoding target image, and the encoded second memory management information is output. 2. The image coding method according to claim 1, wherein the first and second memory management information are information specifying a memory area in the memory that is released as unnecessary. 3. An image encoding device for encoding with reference to a reference image selected from a plurality of reference images stored in a memory, comprising: The image encoding unit encodes the encoding object image with reference to the selected reference image; a management information encoding unit, configured to encode the first memory management information used to manage the reference images stored in the memory; a memory information control unit, by controlling the encoded first memory management information to be attached to the encoded encoding target image, and output the encoded first memory management information; When the first memory management information has been encoded, and the memory information control unit judges to encode and output the first memory management information again, the management information encoding unit takes the first memory management information as The second memory management information is encoded again, The memory information control unit controls to attach the encoded second memory management information to an encoded image different from the encoding target image, and to output the encoded second memory management information. 4. The image encoding device according to claim 3, wherein the first and second memory management information are information specifying a memory area in the memory to be released as unnecessary.

Images