JP2004274732A - Video encoding method and video decoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving picture coding method and a moving picture decoding method, capable of editing in any place of the picture even though it is not an IDR picture which is a special intra picture. <P>SOLUTION: The motion picture coding device comprises a flag information generating unit 112 for generating a flag indicative of discontinuation of display order information POC when the display order information POC becomes discontinuous by editing, etc., and a variable length coding unit 113 for performing a variable length coding etc. in response to an inputted coded signal, and for generating a coded stream Str by adding information such as the flag etc. generated by the flag information generating unit 112. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

  The present invention relates to a moving picture coding method for coding a moving picture signal on a picture basis, a moving picture decoding method for decoding the coded moving picture signal, and a program for implementing the same with software About.

  In recent years, the multimedia era, in which audio, images, and other pixel values are integrated, has been approached, and the traditional information media, that is, means for transmitting information such as newspapers, magazines, televisions, radios, and telephones to humans, has It has been taken up as an object. Generally, multimedia means not only characters, but also figures, sounds, and especially images, etc., that are simultaneously associated with each other. Is an essential condition.

  However, when the amount of information of each information medium is estimated as a digital information amount, the amount of information per character is 1-2 bytes in the case of characters, whereas 64 Kbits per second in the case of voice (telephone quality). In addition, for a moving image, an information amount of 100 Mbits per second (current television reception quality) or more is required, and it is not realistic to handle the vast amount of information in the above-mentioned information medium as it is in a digital format. For example, videophones have already been put into practical use by the Integrated Services Digital Network (ISDN), which has a transmission speed of 64 Kbit / s to 1.5 Mbits / s. It is impossible.

Therefore, it is necessary to use information compression technology. For example, in the case of videophones, videos based on the H.261 and H.263 standards recommended by the ITU-T (International Telecommunication Union Telecommunication Standardization Sector) Compression technology is used. Further, according to the information compression technology of the MPEG-1 standard, it is possible to store image information together with audio information in a normal music CD (compact disc).
Here, MPEG (Moving Picture Experts Group) is an international standard for moving image signal compression standardized by ISO / IEC (International Organization for Standardization), and MPEG-1 is used for moving image signals. It is a standard that compresses information of television signals to 5 Mbps, that is, to about 1/100. In addition, since the target quality is set to a medium quality that can be realized at a transmission speed of about 1.5 Mbps mainly in the MPEG-1 standard, MPEG-2 standardized to meet the demand for higher image quality. Then, the TV broadcast quality of the moving image signal is realized at 2 to 15 Mbps. Furthermore, at present, the working group (ISO / IEC JTC1 / SC29 / WG11), which has been working on the standardization of MPEG-1 and MPEG-2, has achieved compression ratios exceeding those of MPEG-1 and MPEG-2, and furthermore, on a per-object basis. MPEG-4, which enables encoding, decoding, and operation, and realizes new functions required in the multimedia age, has been standardized. MPEG-4 was initially aimed at standardizing a low bit rate coding method, but is now expanded to more general-purpose coding including high bit rates including interlaced images. Further, at present, ISO / IEC and ITU-T are jointly working on standardization of MPEG-4 AVC and ITU H.264 as next-generation image coding systems with higher compression ratios. As of August 2002, a next-generation image coding system called a committee draft (CD) has been issued (for example, see Non-Patent Document 1).

  Generally, in coding of a moving image, the amount of information is compressed by reducing redundancy in the time direction and the space direction. Therefore, in inter-picture predictive coding for the purpose of reducing temporal redundancy, motion detection and a predicted image are created in block units with reference to a forward or backward picture, and the obtained predicted image and coding are performed. Encoding is performed on the difference value from the target picture. Here, a picture is a term representing one screen, and means a frame in a progressive image and a frame or a field in an interlaced image. Here, an interlaced image is an image in which one frame is composed of two fields at different times. In encoding and decoding of an interlaced image, one frame may be processed as a frame, processed as two fields, or processed as a frame structure or a field structure for each block in the frame. it can.

  A picture that does not have a reference picture and performs intra-picture prediction coding is called an I picture. A picture that performs inter-picture predictive encoding by referring to only one picture is called a P picture. A picture that can perform inter-picture predictive encoding by simultaneously referring to two pictures is called a B picture. The B picture can refer to two pictures as an arbitrary combination of display time from the front or the rear. The reference picture (reference picture) can be specified for each block, which is a basic unit of encoding and decoding, but the reference picture described earlier in the encoded bit stream is replaced with the first reference picture. , The one described later is distinguished as the second reference picture. However, as a condition for encoding and decoding these pictures, the picture to be referred to must be already encoded and decoded.

  Motion-compensated inter-picture prediction coding is used for coding a P picture or a B picture. The motion compensated inter-picture predictive coding is a coding method in which motion compensation is applied to the inter-picture predictive coding. Motion compensation does not simply predict from the pixel value of a reference frame, but detects the amount of motion of each part in a picture (hereinafter referred to as a motion vector) and performs prediction in consideration of the amount of motion. Is a method for improving the prediction accuracy and reducing the amount of data. For example, a data amount is reduced by detecting a motion vector of a current picture to be coded and coding a prediction residual between a predicted value shifted by the motion vector and the current picture to be coded. In the case of this method, since information of a motion vector is required at the time of decoding, the motion vector is also encoded and recorded or transmitted.

The motion vector is detected in units of macroblocks. Specifically, the macroblock on the picture to be coded is fixed, and the macroblock on the reference picture is moved within the search range, so that it is most similar to the reference block. The motion vector is detected by finding the position of the reference block.
FIG. 19 is a block diagram showing a configuration of a conventional moving picture encoding device.

The moving picture coding apparatus includes a motion detection unit 103, a subtraction calculation unit 104, a coding unit 105, a motion compensation unit 106, a variable length coding unit 107, a decoding unit 108, an addition calculation unit 109, and memories 110 and 111. Have.
The moving image signal Vin is input to the subtraction operation unit 104 and the motion detection unit 103.
Using the encoded decoded image data read from the memory 110 as a reference picture, the motion detection unit 103 detects a motion vector MV indicating a position predicted to be optimal in a search area in the picture, and performs motion compensation. Output to the unit 106.

The motion compensating unit 106 generates a motion-compensated image signal MCRef using the motion vector MV detected by the motion detecting unit 103, and outputs the motion-compensated image signal MCRef to the subtraction operation unit 104 and the addition operation unit 109.
The subtraction operation unit 104 calculates the difference between the input moving image signal Vin and the motion compensated image signal MCRef input from the motion compensation unit 106, and outputs a difference signal Dif to the encoding unit 105.
The coding section 105 performs coding processing such as frequency conversion and quantization on the input difference signal Dif, generates a coded signal, and outputs the coded signal to the variable-length coding section 107 and the decoding section 108. The variable-length coding unit 107 generates a coded stream Str by performing variable-length coding and the like on the input coded signal and further adding the motion vector MV and the like input from the motion compensation unit 106. Output to the outside of the video encoding device.

The decoding unit 108 performs decoding processing such as inverse quantization and inverse frequency conversion on the input encoded signal, and outputs the decoded difference signal RecDif to the addition operation unit 109.
The addition operation unit 109 adds the difference signal RecDif input from the decoding unit 108 and the image signal RecMCRef input from the motion compensation unit 106 to generate a local decoded image LocalRecon. The generated local decoded image LocalRecon is output to the memory 111.

  The local decoded image is an image that matches the result decoded by the video decoding device, and is used as a reference image when coding the video signal Vin at the next time. Therefore, the local decoded image LocalRecon written in the memory 111 is copied to the memory 110 before the next moving image signal Vin is input, or the memory 110 and the memory 111 are exchanged.

  FIG. 20 is a diagram for explaining the concept of the display order information (Picture Order Count: POC) and the frame number (Frame Number: FN) of the JVT. The display order information POC indicates the display order of the pictures. However, this does not mean the actual display time. For example, since the display order information POC of the picture IDR19 in the figure is “0” and the POC of the next picture B20 is “1”, it is known that the picture B20 should be displayed next to the picture IDR19. I do not know if it should be displayed after the time has passed. The actual display time is obtained from non-video data associated with each picture, and is managed by a device that does not involve a video decoder (video decoding device). The display order information POC is always reset to “0” in an IDR picture, which is a special intra picture, and is added so as to increase by one for each picture in the display order. When it reaches a predetermined maximum value, it is reset to "0" again. In the example of the figure, the display order information POC returns to “0” in the picture IDR pictures IDR19 and IDR29, and the maximum value of the display order information POC is set to “4”. The state of circulating and returning to “0” is shown.

  FN is a number assigned to the referenced picture. (A) in the figure shows the state of the memory before decoding the picture B21, in which three reference pictures are stored. (B) in the figure shows a state after the picture B21 is decoded and stored in the memory. Here, the FN of the picture B21 has the same value as the picture P25 to be decoded next, but when a plurality of pictures consecutive in the decoding order have the same FN as described above, the last picture in the decoding order is the reference picture. And that the other pictures are not reference pictures. In this example, since the picture B21 is not a reference picture, when it is stored in the memory, it is marked as "unused as a reference picture" (the marked state is abbreviated as unused). When the reference picture is stored in the memory, it is marked as “used as a reference picture” (the marked state is abbreviated as used). In the figure, only “unused” is described. Also, whether or not the picture is a reference picture can be seen from a field called nal_ref_idc in the encoded stream, but is not described here because it does not directly relate to the description of the present invention. Also, the frame number FN is always reset to “0” in the IDR picture, similarly to the display order information POC, and is returned to “0” again when it reaches a predetermined maximum value. In this example, it is shown that the picture IDR19 and the picture IDR29 are reset to “0” and the picture B24 is reset to “0”.

  The operation of erasing a picture in order to secure a free area from the memory will be described with reference to FIGS. 21 and 22. FIG. 21 is a diagram for explaining an erasing operation when there is an unused picture. Immediately before decoding the picture P23, pictures obtained by decoding the picture IDs R19, P22, B20, and B21 are stored. Since the picture B20 is not referenced, it is assumed that the picture B20 has been previously unused (see (A) in the figure). Next, the pictures are unused as necessary using a technique such as an MMCO (Memory management control operation) for performing memory management or a sliding window that is determined to be unnecessary from the oldest one. These operations are referred to herein as unused marking processing. Here, it is assumed that the picture P22 has been unused (see (B) in the figure). Next, the picture is erased in order to secure a free area. If there is such an unused picture, the picture with the earliest display order (POC) among the unused pictures is erased. Here, since the display order of the picture P22 is "3" and the display order of the picture B20 is "1", the picture B20 is deleted (see (C) in the figure). The picture P23 is stored in the empty area after the deletion (see (D) in the figure).

Note that a picture includes a frame and a field, and is described as a picture in this specification. However, when storing in a memory, the picture may be stored in a frame unit (an odd field and an even field at the same time). Also, when erasing to secure a free area in the memory, erasing may be performed in frame units.
In the figure, the number indicated by stage indicates the transition stage of the memory, stage 1 is the stage before performing the unused mark processing in the processing of the picture, stage 2 is the stage after the unused mark processing is performed, stage3 means a stage after securing a free area, and stage4 means a stage after storing a picture.

  FIG. 22 is a view for explaining an erasing operation when there is no unused picture in the memory. As shown in the figure, the pictures are decoded in the order of picture IDR19, P22, B20, B21, P23. As shown in (A) in the figure, it is assumed that the pictures of IDR19, P22, B20, and B21 are stored in the memory before the picture P23 is decoded, and none of them is unused. Then, as shown in (B) in the figure, it is assumed that none of the unused marking processes have been unused. As described above, when there is no unused picture, in order to secure a free area, the picture decoded first among the pictures stored in the memory is deleted. As shown in (C) in the figure, since the IDR19 is the first decoded picture among the pictures stored in the memory, the IDR19 is deleted. Finally, as shown in (D) in the figure, the decoded picture P23 is stored in the empty area.

FIG. 23 is a block diagram showing a configuration of a conventional video decoding device.
The moving picture decoding apparatus includes a variable length decoding unit 402, an image decoding unit 202, an MMCO decoding unit 204, a memory 206, and a memory management unit 401.
The video encoded signal Str is input, the variable-length decoding unit 402 performs variable-length decoding, the encoded picture data comp_pic is decoded by the image decoding unit 202, and the decoded image signal Recon is stored in the memory 206. To store. When the picture is inter-coded, the image decoding unit 202 sends the motion information MV to the memory 206 to create a motion-compensated reference image MCPic and performs motion compensation at the time of decoding. The memory management unit 401 outputs a memory management instruction mctrl for determining a picture storage area, securing a free area, and the like. The display order information POC is output from the variable length decoding unit 402 to the memory management unit 401 and held. Further, the MMCO command MMCO, which is one of the above-described non-use marking processes, is input from the variable length decoding unit 402 to the MMCO decoding unit 204, decoded, and an instruction of unused is input to the memory management unit 401. Further, the decoded image signal Vout displayed from the memory 206 is output.

  FIG. 24 is a flowchart of a memory-related operation of the conventional video decoding device. In this flow, the operation for each picture is shown in steps S1 to S2. Unused marking processing is performed, and each picture in the memory is marked as unused as necessary (step S13). Next, a free area securing process is performed to secure a free area in the memory (step S14). Next, the decoded image signal Vout is stored in the empty area (step S15).

  FIG. 25 is a flow chart of the operation of the free space securing process of the conventional moving picture decoding apparatus, and is a flow chart for explaining step S14 of FIG. 24 in detail. The process of securing a free area (step S14) checks whether a picture marked as unused is present in the memory 206 (step S141). Deletes the oldest picture in the display order (step S143), and if not, deletes the first decoded picture among the pictures stored in the memory 206 (step S142).

  FIG. 27 is a conceptual diagram illustrating the operation of the invalid picture process. In the JVT, a memory management operation is defined in which, when some pictures of a sequence input to a moving picture decoding apparatus are lost, invalid pictures are inserted by the number of lost pictures. This operation is performed in the video decoding device when required_frame_num_update_behaviour_flag in the sequence parameter set is “1”. An invalid picture is a picture that has no actual restored image signal and is specially marked, and is not to be referred to as a reference picture. It is assumed that the state of the memory after decoding the pictures I19, P20, P21, P22, and P23 as shown in the figure is as shown in FIG. Next, when decoding the picture B24, a reference index used for specifying a reference picture is assigned to a new non-unused picture in the decoding order so that the value of the reference index ref_idx becomes smaller. This assignment is an example, and the method differs depending on the picture type and the like, but the assignment has the dependency property that the index of the reference relationship is assigned depending on the picture stored in the memory. In the example of this figure, the unused decoded picture P22 decoded last is assigned as ref_idx = 0, and the previously decoded non-used picture P21 decoded as ref_idx = 1.

  Here, if the picture P21 and the picture P23 are lost during transmission or the like and are not input to the decoder, and if an invalid picture is not inserted, when decoding the picture B24, as shown in FIG. Reference index ref_idx is assigned. Originally, reference picture P22 and picture P20 of picture B24 are respectively assigned ref_idx = 0 and ref_idx = 2.However, ref_idx = 0 is assigned to picture P22, and ref_idx = 2 is assigned to picture I19. There is a problem that the picture I19 is referred to by mistake. To avoid this, invalid pictures are inserted.

  FIG. 11C shows the state of the memory before decoding picture B24 when an invalid picture is inserted. If a discontinuity in the frame number FN is detected, invalid pictures are inserted by the number of discontinuities. In this example, when the picture P22 with FN = 3 is decoded, the picture P20 decoded immediately before is FN = 1, so that the number of places where the picture P20 does not increase by one or more is increased by two, so it can be seen that one picture has been lost. . Therefore, before decoding the picture P22, one invalid picture is inserted. As described above, an invalid picture is a special picture, and has no actual restored image signal, but is marked as used, and is treated as a reference picture when a reference picture is assigned. Otherwise, it is marked as "non-exist".

  FIG. 28 is a block diagram showing a configuration of a conventional video decoding device. The difference from the conventional moving picture coding apparatus described with reference to FIG. 23 is that the FN gap detecting section 211 is provided and the operation of the memory managing section 412 is different. The FN gap detection unit 211 acquires the frame number FN from the variable length decoding unit 411, and when there is a gap, instructs the memory management unit 412 to insert a necessary number of invalid pictures. The memory management unit 412 stores invalid pictures in the memory 206 by the designated number.

  FIG. 29 is a flowchart showing the operation of the invalid picture processing of the conventional video decoding device. The difference from the memory-related operation of the conventional video decoding device described with reference to FIG. 24 is that the gap of the frame number FN is checked before the non-use marking process (step S13) (step S11). , The invalid pictures are stored in the memory 206 by the number of gaps in the memory 206 (step S12), and then the process proceeds to the unused marking process (step S13). If there is no gap, the process proceeds to the unused marking process (step S13). . At step S12, invalid pictures are stored by the number of gaps. Every time one picture is to be inserted, the same processing as that of storing a normal picture shown in FIG. 24 is performed.

  FIG. 31 is a conceptual diagram illustrating the structure of a conventional MPEG-2 stream. As shown in the figure, the MPEG2 stream has the following hierarchical structure. A stream is composed of a plurality of groups of pictures, and by using this as a basic unit of encoding processing, editing of moving images and random access are possible. The group of pictures is composed of a plurality of pictures, and each picture includes an I picture, a P picture, or a B picture. The stream, GOP, and picture further include a synchronization signal (sync) indicating a break of each unit and a header (header) which is data common to the unit. In MPEG-2, a P picture can be predictively coded with reference to only one I picture or P picture whose display time is immediately before. The B picture can be subjected to predictive encoding with reference to one I picture or one P picture immediately before and after one display time. Furthermore, the order in which they are arranged in the stream is also determined, and they are arranged immediately after an I picture or a P picture. Therefore, at the time of random access, if decoding is started from an I picture, all pictures arranged after the I picture can be decoded and displayed. Further, since at most two reference pictures can be stored in the memory, the degree of freedom of the reference structure is limited.

  FIG. 32 is a conceptual diagram illustrating a conventional JVT video coding method. In JVT, it is also possible to refer to pictures arbitrarily distant as long as they do not cross an IDR picture which is a special intra picture. Therefore, for example, in order to increase the coding efficiency, it is also possible to rearrange the coding order of a large number of pictures and perform coding. In the figure, it is assumed that the correlation between the pictures 19, 20, 21, 25, 26, and 27 is very strong, and the correlation between the pictures 22, 23, 24, 28, 29, and 30 is very strong. In this case, pictures 19, 20, 21, 25, 26, and 27 are first inter-coded (GOP1), and pictures 22, 23, 24, 28, 29, and 30 are inter-coded (GOP2). It can be expected that the coding efficiency will be increased.

FIG. 33 is an operation flow diagram of a conventional JVT video coding method. In the video coding method of JVT, all uncoded pictures can be set as coding candidates (step S55). Then, a picture is selected and encoded from some viewpoint from the encoding candidates (step S56). For example, when there are 10 uncoded pictures, all 10 pictures may be set as coding candidates, and the 10th picture may be selected and coded in the display order. After the encoding, if there is an uncoded picture, the process returns to step S55. In step S56, it is possible to wait for an uncoded picture to be input without coding.
ISO / IEC 14496-10 Editor's Proposed Changes Relative to JVT-E146d37ncm, revision 4, 2002-12

By the way, in such a conventional video decoding device, and in the conventional video decoding device, it is not possible to edit an encoded stream except for the location of an IDR picture which is a special intra picture as described above. Was. This problem will be described below.
FIG. 26 is a conceptual diagram illustrating the problem that the discontinuity of the sequence causes the discontinuity of the display order information POC and the undisplayed picture is deleted. A case where two parts Clip1 and Clip2 of a certain sequence are connected and decoded is shown. The place where the discontinuity of the sequence caused by editing or the like occurs as described above is referred to as an edit point. In this example, it is assumed that the maximum value of the display order information POC is set so that it is not necessary to consider the circulation of the display order information POC. (A) in the figure shows the state of the memory after decoding of Clip1, in which pictures I19, P22, B20 and B21 are stored. It is assumed that the display order information POC is “4”, “7”, “5”, and “6”, respectively, as shown in the figure, and that the pictures I19, B20, and B21 are marked as unused. Next, the state before decoding the first picture I85 of Clip2 and before decoding the second picture P86 is shown in FIG. Here, it is assumed that picture I85 is stored at the position where picture B20 was. Next, an unused marking process is performed. In the case of this Clip2, it is assumed that the picture I85 is marked as unused ((B) in the figure). Next, a free area securing process is performed. As described above, since there is an unused picture, a picture having the first display order among the unused pictures is deleted, so that the picture I85 is deleted. Here, assuming that the delay from decoding to display is three on average, pictures B21, P22, and I85 have not been displayed yet. However, the picture I85 is deleted from the memory even though it is not displayed yet.

  FIG. 30 is a conceptual diagram illustrating the problem that the discontinuity of the sequence causes the discontinuity of the frame number FN, and the invalid picture deletes the undisplayed picture. This example shows a state where another discontinuous portion Clip1 and Clip2 of a certain sequence are connected and decoded. FIG. 11A shows the state of the memory after decoding the picture P25, in which five pictures P21 to P25 are stored. Next, FIG. 12B shows a state after inserting an invalid picture when decoding the first picture I60 of Clip2. Since the picture I60 has FN = 12 and the picture P25 decoded immediately before has FN = 5, it is determined that six pictures have been lost, and six invalid pictures are inserted. In this case, since all the pictures in the memory are erased, there is a problem that the pictures P23, P24, and P25 are erased even if the pictures P23, P24, and P25 have not been displayed in the state of FIG.

  FIG. 34 is a conceptual diagram illustrating a problem caused by the degree of freedom of JVT coding at the time of editing or random access. FIG. 29B shows an original stream, which is the same as the stream shown in FIG. FIG. 7A shows a state in which only GOP2 is decoded without GOP1. In this case, since the pictures 25, 26, and 27 cannot be obtained, a playback discontinuity occurs in that the pictures 25 to 27 cannot be played after the pictures 22 to 24 have been played. This poses a problem when GOP1 is deleted by editing, or when GOP2 is randomly accessed. FIG. 7C shows a state in which there is no GOP2 and decoding is performed up to GOP1. In this case, the pictures 22, 23, and 24 cannot be obtained, so that discontinuity in reproduction occurs. This is a problem when GOP2 is deleted by editing.

  Therefore, the present invention has been made in view of the above circumstances, and a moving picture coding method and a moving picture decoding method that can perform editing at a picture location other than an IDR picture which is a special intra picture. The purpose is to provide.

  In order to achieve the above object, a moving picture coding method according to the present invention is a moving picture coding method for coding a moving picture signal on a picture basis to generate a coded stream, wherein the order of the pictures is out of order. It is characterized by including a flag information creating step of creating flag information indicating continuity, and an information adding step of adding the flag information to the encoded stream.

Thereby, information indicating that the order of pictures is discontinuous can be added to the encoded stream.
Also, the moving picture coding method according to the present invention is a moving picture coding method for coding a moving picture signal on a picture basis to generate a coded stream, wherein the predetermined coding unit is composed of a plurality of pictures. Is characterized in that it includes a coding step of coding a picture whose display order is later than the first intra-coded picture in, so as to be included in a coding unit subsequent to the coding unit.

As a result, even if decoding is performed after a certain coding unit, reproduction can be performed without causing discontinuity in reproduction.
Further, the moving picture decoding method according to the present invention is a moving picture decoding method for decoding an encoded stream in units of pictures, wherein information for extracting flag information indicating that the order of the pictures is discontinuous is provided. It is characterized by including an extracting step and a managing step of managing an area for storing a decoded picture based on the flag information.

Further, the flag information is information indicating that display order information of pictures is discontinuous. In the managing step, the decoding stored in the area is performed based on the display order information and the flag information. The picture with the earliest display order among the completed pictures may be determined, and the determined picture may be set as the deletion target picture.
This makes it possible to prevent a non-display picture from being erased due to discontinuous display order information of pictures.

  Further, the video decoding method further includes an invalid picture storing step of storing an invalid picture in the area when the encoding order information of the picture is discontinuous, wherein the flag information includes the encoding order. Information indicating that the information is discontinuous. In the managing step, it is determined whether or not an invalid picture is stored in the area based on the flag information and the coding order information. In the step, an invalid picture may be stored in the area based on a determination result in the management step.

This makes it possible to prevent a non-display picture from being deleted due to discontinuity in the coding order information of the pictures.
Further, the present invention can be realized not only as such a moving picture coding method and a moving picture decoding method, but also includes the characteristic steps included in such a moving picture coding method and a moving picture decoding method. It can also be realized as a moving image encoding device and a moving image decoding device provided as means, or as a program for causing a computer to execute those steps. Needless to say, such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

  As is clear from the above description, according to the moving picture coding method according to the present invention, editing can be performed at a place of a picture other than an IDR picture which is a special intra picture.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of the moving picture coding apparatus according to the present embodiment.
The moving picture coding apparatus includes a motion detection unit 103, a subtraction calculation unit 104, a coding unit 105, a motion compensation unit 106, a variable length coding unit 113, a decoding unit 108, an addition calculation unit 109, memories 110 and 111, A flag information generator 112 is provided.

The difference from the conventional moving picture coding apparatus (FIG. 19) is that a flag information generating unit 112 and a variable length coding unit 113 that operates differently from the variable length coding unit 107 are provided.
When the display order information POC is discontinuous due to editing or the like, the flag information generation unit 112 generates a flag indicating that the display order information POC is discontinuous.
The variable-length coding unit 113 performs variable-length coding and the like on the input coded signal, and further generates a flag generated by the flag information generation unit 112, a motion vector MV input from the motion compensation unit 106, and the like. To generate an encoded stream Str, and output the encoded stream Str to the outside of the video encoding apparatus.

  FIG. 2 is a diagram for explaining the concept of the moving picture coding method and the moving picture decoding method of the present invention. This figure illustrates how to solve the case of FIG. 26 which describes the conventional problem. First, the discontinuity of the display order information POC is detected by a flag added at the time of encoding by editing or the like. This flag is called flag A. The flag A is a flag indicating that the display order information POC is discontinuous due to editing or the like.

  This flag A is special information to be placed immediately before Clip as shown in the figure. JVT defines a unit called Supplemental enhancement information (hereinafter abbreviated as SEI) for storing additional information for video decoding, and will store it in this unit. The information may be stored in User data registered SEI that can be uniquely defined by the user, or may be stored in Random access point SEI (hereinafter abbreviated as RAP SEI) that stores information for random access. The RAP SEI contains a broken_link_flag indicating that the video decoded by editing etc. may be different from the original video, and the video decoded from the nth picture in the display order from the position of the RAP SEI contains the original video. When the moving image is equal to or almost equal to the moving image, information such as recovery_frame_cnt indicating the n images is stored. In the present invention, when the broken_link_flag of the RAP SEI is “1”, it is detected that editing has been performed, and the edit point is set immediately before the first picture after the RAP SEI. Or, edit point immediately before the picture pointed to by recovery_frame_cnt. Alternatively, the first independently decodable picture after RAP_SEI (for example, Intra Picture) is set as the edit point. Note that the edit point indicates only the boundary of a picture, and does not define the boundary of SEI. In addition, the file format storing the sequence may store random access information of each picture, which information indicates that the image is being edited, and further includes information on the editing point. May be stored. In that case, the detection of the editing and the specification of the editing point may be performed according to the information of the file format. These storage formats are referred to as storage formats of the flag format A.

FIG. 3 is a block diagram showing a configuration of the moving picture decoding apparatus according to the present embodiment. The difference from the conventional video decoding device (FIG. 23) is that an edit detection unit 203 has been added and that a memory management unit 205 having an operation different from that of the memory management unit 401 is provided.
The edit detection unit 203 acquires a flag indicating that the display order information POC is discontinuous or information storing edit point information from the variable length decoding unit 201, analyzes the information, and controls the memory management unit 205. Outputs signal mctrlc. When the control signal mctrlc indicating the editing is input, the memory management unit 205 manages the picture before the editing point so that the display order is before the picture after the editing point. That is, when a picture to be deleted is selected from unused pictures, the display order of the picture before the edit point is assumed to be before the picture after the edit point.

  In order to manage the display order, the memory management unit 205 causes each picture to hold a Clip counter that increases by one each time the edit point is exceeded. As shown in (B) in the figure, pictures B20, P22, and B21 are recorded as Clip = 1, and picture I85 after the edit point is recorded as Clip = 2. In the unused marking processing in this state, the picture B20 in the display order that is the earliest in the display order is deleted from the pictures of the first decoded clip of the unused picture (pictures B20, P22, and B21 marked Clip = 1). . This solves the problem of deleting an undisplayed picture (picture I85 in the conventional problem).

  FIG. 4 is an operation flowchart of the moving picture decoding method according to the present embodiment. The difference from the conventional moving picture decoding method (FIG. 24) is that step S31 and step S32 are added, and step S14 is modified to step S14B. After the start of processing in units of pictures (step S1), it is checked whether the image has been edited (step S31). If the image has been edited, processing is performed at the edit point (step S32). If the editing is not detected, the unused marking processing (step S13) is performed as in the conventional case, and the free area securing processing (step S14B) is performed in consideration of the decoding order before and after the editing point. The processing at the edit point is to enable the discrimination between before and after the edit point, and the memory management unit 205 increases the Clip counter by one each time the edit point is exceeded.

  FIG. 5 is an operation flowchart of a free area securing process of the moving picture decoding method according to the present embodiment. The difference from the conventional method of securing an empty area (FIG. 24) is that step S43B is a modification of step S43. If the unused picture is stored in the memory (step S41), the clip including the unused picture is searched for in priority to the previous clip in the decoding order, and the picture in the first display order among the unused pictures of the clip is searched. Remove. In other words, the unused pictures in the clip including the unused picture first in the decoding order, and the picture in the first display order among those pictures is deleted. Alternatively, in other words, the picture in the first display order is deleted from among the unused pictures included immediately before and immediately after the first unused picture in the decoding order and between the edit points.

As described above, since the discontinuation of the display order information POC due to editing or the like is detected by the flag added at the time of encoding, and the picture to be deleted is determined, the undisplayed picture (conventional In the example shown in FIG. 26, the problem of deleting picture I85) can be solved.
In order to solve the discontinuity of editing by the moving picture decoding method of the present invention (FIGS. 2, 3, 4, and 5), information indicating an editing point is required. Therefore, it is desirable that the information indicating that the edit point information is included be located in a place easily available to the decoding apparatus.

  In the present embodiment, a flag added during encoding indicating that the display order information POC is discontinuous is added between pictures in which the display order information POC is discontinuous. It is not limited to this. For example, the flag information generation unit 112 generates a flag indicating that the display order information POC is discontinuous, and information identifying a position (edit point) where the display order information POC is discontinuous. It does not matter. Then, store this information in the sequence parameter set, or store it in User data registered SEI, which can be uniquely defined by the user, and place it at a location where the sequence is easily available, for example, at the beginning, or record the sequence It is stored in a medium or stored in a file format for managing the sequence. These storage formats are called flag format A2.

In this case, at the time of decoding, the editing detection unit 203 acquires the flag A2 from these locations, and if the editing point information can be obtained, the moving picture decoding method of the present invention (FIGS. 2, 3, 4, Perform FIG. 5).
(Embodiment 2)
The configuration of the moving picture coding apparatus according to the present embodiment is the same as the block diagram of the first embodiment shown in FIG.

In the present embodiment, when the frame number FN is discontinuous due to editing or the like, the flag information generating unit 112 generates a flag B indicating that the frame number FN is discontinuous.
Note that the flag B generated by the flag information generation unit 112 may be a flag that instructs not to insert an invalid picture. The format of the flag B is the same as the format of the flag A shown in the first embodiment.

FIG. 6 is a block diagram showing a configuration of the moving picture decoding apparatus according to the present embodiment. This figure is obtained by adding an edit detection unit 214 and changing the memory management unit 212 to the conventional video decoding device described with reference to FIG.
The edit detection unit 214 acquires a flag indicating that the frame number FN is discontinuous from the variable length decoding unit 201, and outputs a control signal ctrl_c to the memory management unit 212. Even when a request to insert an invalid picture is input by the control signal ctrl_c from the FN gap detection unit 211, the memory management unit 212 inserts an invalid picture when notified by the edit detection unit 214 that it is being edited. Shall not be performed.

  FIG. 7 is an operation flowchart of the moving picture decoding method of the present invention. The difference from the conventional moving picture decoding method (FIG. 29) is that step S31 is added and step S14 is modified to step S14B. The other steps perform the same operation as the steps having the same reference numerals in FIG. 29, and thus the description will be omitted. Further, Step S31 and Step S14B are the same as Step S31 and Step S14B of the decoding apparatus of the present invention described in Embodiment 1, and thus description thereof is omitted.

As described above, the insertion of an invalid picture is determined based on the flag added at the time of encoding that the frame number FN is discontinuous due to editing or the like. In the example shown in FIG. 30, the problem of deleting the pictures P23, P24, and P25) can be solved.
Note that, in the present embodiment, a flag added at the time of encoding indicating that the frame number FN is discontinuous, as in Embodiment 1, is used between pictures in which the frame number FN is discontinuous. , But is not limited to this. For example, as in the first embodiment, the flag information generation unit 112 specifies a flag indicating that the frame number FN is discontinuous and a position (edit point) where the frame number FN is discontinuous. May be generated. Then, these pieces of information are stored as in the first embodiment.

In this case, at the time of decoding, the editing detection unit 203 acquires the flag B2 from these locations as in Embodiment 1, and if the editing point information is obtained, the moving picture decoding method of the present invention (FIG. 6). , FIG. 7).
FIG. 8 is a diagram showing the structure of data output by the moving picture coding method of the present invention and the structure of data input by the moving picture decoding method of the present invention in Embodiments 1 and 2. The sequence as an encoded moving image signal includes RAP, MMCO, and PICTURE data as shown in FIG. RAP stands for Random access point SEI, in which broken_link_field is the flag A in the first embodiment and the flag B in the second embodiment. PICTURE is a moving picture signal encoded in picture units, and PICTURE may be preceded by MMCO (or may not exist). MMCO is instruction information of Memory management control operation. Also, as shown in FIG. 3B, the flag A2 of the first embodiment is provided inside the sequence, at a predetermined position in a file format associated with the sequence, or on a recording medium for recording the sequence. Further, the flag B2 in the second embodiment is stored.

(Embodiment 3)
FIG. 9 is a block diagram showing a configuration of the video encoding device according to the present embodiment.
The moving picture coding apparatus includes a rearrangement memory 101, a coding scheduling unit 102, a motion detection unit 103, a subtraction calculation unit 104, a coding unit 105, a motion compensation unit 106, a variable length coding unit 107, a decoding unit 108, An addition operation unit 109 and memories 110 and 111 are provided.

The reordering memory 101 stores moving images input in picture units in display time order. The encoding scheduling unit 102 rearranges the pictures stored in the rearrangement memory 101 in the order in which the encoding is performed.
FIG. 10 is a diagram for explaining the concept of the moving picture coding method according to the present embodiment. The moving picture coding method according to the present embodiment that solves the problem shown in FIG. 34 stores only consecutive pictures in a certain GOP in a display order, and the display order of an arbitrary picture in a certain GOP is as follows. Is to be coded so as to be earlier than the display order of an arbitrary picture of the GOP to be decoded. By performing encoding in this manner, in GOP1 and GOP2, reproduction discontinuity does not occur in the case shown in FIG.

  FIG. 11 is an operation flowchart of the moving picture coding method according to the present embodiment. The operation will be described with reference to FIG. Among the uncoded pictures, a picture continuous from the earliest in the display order is set as a display basic unit (step S61). That is, one or more pictures that do not become discontinuous in the display order are set as the display basic units, and the display basic units are determined so that there is no uncoded picture preceding the display basic unit. Next, it is checked whether or not there is an uncoded picture in the display basic unit (step S62). If there is (Yes in step S62), the uncoded picture in the display basic unit is set as a coding candidate. The encoding is selected from the encoding candidates (step S63). It is checked whether there is an uncoded picture (step S64), and if there is one (Yes in step S64), the process proceeds to step S62. If not (No in step S64), the process ends. Note that the display basic unit includes at least the condition “from the earliest picture in display order among uncoded pictures to the last picture in display order among encoded pictures, Can be changed at any time as long as the condition is satisfied.

  FIG. 2B is also an operation flow diagram of the moving picture coding method according to the present embodiment. In this method, the coding is performed from the uncoded pictures in the display order from the earliest picture in display order. Among the coded pictures, up to the picture in the display order which is the last one is set as an essential coding candidate (step S71), and is selected and coded from uncoded pictures, including the essential coding candidate (step S71). S72). Next, it is checked whether there is any uncoded picture other than the I picture (step S73), and if there is (Yes in step S73), the process proceeds to step S71, and if not (No in step S73), the process ends. Here, the GOP candidates up to the next I picture are used, but the present invention is not limited to this. For example, the end of the GOP may be determined by describing the GOP in a file format.

Since the GOP is determined as described above, discontinuity in reproduction can be prevented from occurring in each of GOP1 and GOP2 as shown in FIG. 34, for example.
Note that a flag indicating that encoding has been performed as described in the present embodiment may be added to the encoded stream.

(Embodiment 4)
FIG. 12 is a diagram for explaining the concept of the moving picture coding method according to the present embodiment. In the third embodiment, the problem of editing and the problem of random access are simultaneously solved. However, the main moving picture coding method in the present embodiment is a method for solving the problem of random access. . Since restrictions are less strict than in the method of the third embodiment, coding efficiency and the like can be improved.

  Taking GOP2 in FIG. 12 as an example, in this video coding method, 1) pictures (B26, B27, P28) whose display time is later than the intra-picture (I25) of a certain GOP include the intra-picture It is not coded in the GOP (GOP1) immediately before the GOP (GOP2). By performing such control, as shown by Case A in the figure, even if decoding is started from the first picture (I25) of GOP2, it is possible to correctly display all pictures after the first picture. .

  2) Further, pictures (I19, B20, B21, B22, and B22) whose display time is earlier than the intra picture (I25) of a certain GOP and whose display time is later than the intra picture (I19) of the GOP immediately before the GOP. B23, P24) are encoded within the GOP (GOP2) or the immediately preceding GOP (GOP1). By controlling in this way, even if decoding is started from the first picture (I19) of GOP1, all pictures after the first picture (I19) of GOP1 can be correctly displayed.

  Or, in other words, 1) Taking GOP1 as an example, the picture to be displayed at the end of a certain GOP is selected and coded by a picture that is displayed before the I picture (I25) of the next GOP (that is, Picture P24 or earlier must be selected). 2) Taking GOP2 as an example, the first picture to be displayed in a certain GOP is selected from the pictures displayed after the I picture (I19) of the immediately preceding GOP and encoded (that is, picture B20 and subsequent pictures are selected). Must).

  Or, in other words, the display order of the picture displayed first in a GOP is later than the I picture of the immediately preceding GOP, and the display order of the picture displayed last in the GOP is later than the I picture of the immediately following GOP. This is a moving picture coding method for coding so that the display order is first. Here, the description is made as an I picture, but the present invention is equally applicable to a picture that can be decoded independently.

  FIG. 13 is an operation flowchart of the moving picture coding method according to the present embodiment. First, an uncoded picture is selected and coded as an entry picture (step S81). An entry picture is a picture that can be decoded independently. Next, among the uncoded pictures, a picture whose display order is earlier than the last encoded entry picture is set as an essential encoding candidate, and a display order which is earlier than the next entry picture to be encoded is set as a required encoding candidate. The uncoded picture is set as a coding candidate that can be omitted (step S82). Next, it is checked whether there is an uncoded picture in the essential coding candidate (step S83), and if there is (Yes in step S83), it is selected from the essential coding candidate and the optional coding candidate. (Step S85). Next, it is checked whether there is an uncoded picture (step S86). If there is, the process proceeds to step S83; otherwise, the process ends. In step S83, if there is no uncoded picture in the essential coding candidates (No in step S83), it is determined in step S84 whether to encode the next entry picture (step S84). If the entry picture is to be coded (Yes in step S84), the process proceeds to step S81. If not to be coded (No in step S84), the process proceeds to step S85.

As described above, pictures that are displayed after the first intra picture in the GOP are encoded so as to be included in the GOPs after this GOP, so even if a certain GOP or later is decoded, playback discontinuity occurs. The reproduction can be performed without performing.
Note that a flag indicating that encoding has been performed as described in the present embodiment may be added to the encoded stream.

(Embodiment 5)
Furthermore, by recording a program for realizing the configuration of the moving picture encoding method or the moving picture decoding method described in each of the above embodiments on a storage medium such as a flexible disk, The processing described in the embodiment can be easily performed by an independent computer system.

FIG. 14 is an explanatory diagram of a case where the present invention is implemented by a computer system using a flexible disk storing the moving picture coding method or the moving picture decoding method of each of the above embodiments.
FIG. 14B shows the appearance, cross-sectional structure, and flexible disk of the flexible disk viewed from the front, and FIG. 14A shows an example of the physical format of the flexible disk which is a recording medium body. The flexible disk FD is built in the case F, and a plurality of tracks Tr are formed concentrically from the outer circumference toward the inner circumference on the surface of the disk, and each track is divided into 16 sectors Se in an angular direction. ing. Therefore, in the flexible disk storing the program, the moving image encoding method and the moving image decoding method as the program are recorded in the area allocated on the flexible disk FD.

  FIG. 14C shows a configuration for recording and reproducing the program on the flexible disk FD. When the above program is recorded on the flexible disk FD, the moving picture coding method or the moving picture decoding method as the above program is written from the computer system Cs via the flexible disk drive. When the moving picture encoding method and the moving picture decoding method are to be constructed in a computer system using a program in a flexible disk, the program is read from the flexible disk by a flexible disk drive and transferred to the computer system.

In the above description, the description has been made using a flexible disk as a recording medium. However, the same description can be made using an optical disk. Further, the recording medium is not limited to this, and the present invention can be similarly implemented as long as the program can be recorded, such as an IC card or a ROM cassette.
Further, here, application examples of the moving picture coding method and the moving picture decoding method described in the above embodiment and a system using the same will be described.

FIG. 15 is a block diagram illustrating an overall configuration of a content supply system ex100 that realizes a content distribution service. A communication service providing area is divided into desired sizes, and base stations ex107 to ex110, which are fixed wireless stations, are installed in each cell.
The content supply system ex100 includes, for example, a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, and a camera on the Internet ex101 via the Internet service provider ex102 and the telephone network ex104, and the base stations ex107 to ex110. Each device such as a mobile phone ex115 with a tag is connected.

However, the content supply system ex100 is not limited to the combination as shown in FIG. 15, and may be connected in any combination. Further, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110 which are fixed wireless stations.
The camera ex113 is a device capable of shooting moving images, such as a digital video camera. In addition, a mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global System for Mobile Communications) system. Or PHS (Personal Handyphone System) or the like, and either may be used.

  The streaming server ex103 is connected from the camera ex113 to the base station ex109 and the telephone network ex104, and enables live distribution and the like based on the encoded data transmitted by the user using the camera ex113. The encoding process of the captured data may be performed by the camera ex113, or may be performed by a server or the like that performs the data transmission process. Also, moving image data captured by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device such as a digital camera capable of shooting still images and moving images. In this case, encoding of the moving image data may be performed by the camera ex116 or the computer ex111. The encoding process is performed by the LSI ex117 included in the computer ex111 and the camera ex116. The moving image encoding / decoding software may be incorporated in any storage medium (a CD-ROM, a flexible disk, a hard disk, or the like) that is a recording medium readable by the computer ex111 or the like. Further, the moving image data may be transmitted by a mobile phone with camera ex115. The moving image data at this time is data that has been encoded by the LSI included in the mobile phone ex115.

  In the content supply system ex100, the content (for example, a video of a live music shot) captured by the user with the camera ex113, the camera ex116, or the like is encoded and transmitted to the streaming server ex103 in the same manner as in the above embodiment. On the other hand, the streaming server ex103 stream-distributes the content data to the requesting client. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, and the like, which are capable of decoding the encoded data. In this way, the content providing system ex100 can receive and reproduce the encoded data on the client, and further, can receive, decode, and reproduce the encoded data on the client in real time, thereby realizing personal broadcasting. It is a system that becomes possible.

The encoding and decoding of each device constituting this system may be performed using the video encoding device or the video decoding device described in each of the above embodiments.
A mobile phone will be described as an example.
FIG. 16 is a diagram illustrating the mobile phone ex115 using the moving picture coding method and the moving picture decoding method described in the above embodiment. The mobile phone ex115 includes an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, a video image of a CCD camera or the like, a camera unit ex203 capable of taking a still image, a video image captured by the camera unit ex203, and an antenna ex201. A display unit ex202 such as a liquid crystal display for displaying data obtained by decoding a received video or the like, a main unit including a group of operation keys ex204, an audio output unit ex208 such as a speaker for outputting audio, and audio input. Input unit ex205 such as a microphone for storing encoded or decoded data, such as data of captured moving images or still images, received mail data, moving image data or still image data, etc. Recording medium ex207, and a slot ex20 for allowing the recording medium ex207 to be attached to the mobile phone ex115. The it has. The recording medium ex207 stores a flash memory element which is a kind of EEPROM (Electrically Erasable and Programmable Read Only Memory) which is a nonvolatile memory which can be electrically rewritten and erased, in a plastic case such as an SD card.

  Further, the mobile phone ex115 will be described with reference to FIG. The mobile phone ex115 controls a power supply circuit unit ex310, an operation input control unit ex304, an image encoding unit, and a main control unit ex311 that integrally controls each unit of a main unit including a display unit ex202 and operation keys ex204. Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, demultiplexing unit ex308, recording / reproducing unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via a synchronous bus ex313. Connected to each other.

The power supply circuit unit ex310 activates the camera-equipped digital mobile phone ex115 in an operable state by supplying power to each unit from the battery pack when the call end and the power key are turned on by a user operation. .
The mobile phone ex115 converts an audio signal collected by the audio input unit ex205 into digital audio data by the audio processing unit ex305 in the audio communication mode based on the control of the main control unit ex311 including a CPU, a ROM, a RAM, and the like. This is spread-spectrum processed by a modulation / demodulation circuit unit ex306, subjected to digital-analog conversion processing and frequency conversion processing by a transmission / reception circuit unit ex301, and then transmitted via an antenna ex201. The mobile phone ex115 amplifies received data received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-to-digital conversion processing, performs spectrum despreading processing in the modulation / demodulation circuit unit ex306, and performs analog voice decoding in the voice processing unit ex305. After the data is converted, the data is output via the audio output unit ex208.

  Further, when an e-mail is transmitted in the data communication mode, text data of the e-mail input by operating the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing on the text data in the modulation / demodulation circuit unit ex306, performs digital / analog conversion processing and frequency conversion processing on the transmission / reception circuit unit ex301, and transmits the text data to the base station ex110 via the antenna ex201.

  When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. When the image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

  The image encoding unit ex312 includes the moving image encoding device described in the present invention, and encodes the image data supplied from the camera unit ex203 using the moving image encoding device described in the above embodiment. The image data is converted into encoded image data by performing compression encoding according to a demultiplexing method, and is transmitted to the demultiplexing unit ex308. Further, at this time, the mobile phone ex115 simultaneously transmits the voice collected by the voice input unit ex205 during imaging by the camera unit ex203 to the demultiplexing unit ex308 as digital voice data via the voice processing unit ex305.

  The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and modulates and outputs the resulting multiplexed data. The signal is subjected to spread spectrum processing in ex306 and subjected to digital / analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and then transmitted via the antenna ex201.

When data of a moving image file linked to a homepage or the like is received in the data communication mode, the data received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexed data is obtained. The demultiplexed data is sent to the demultiplexing unit ex308.
To decode the multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data into a bit stream of image data and a bit stream of audio data, and performs synchronization. The coded image data is supplied to the image decoding unit ex309 via the bus ex313, and the audio data is supplied to the audio processing unit ex305.

  Next, the image decoding unit ex309 has a configuration including the moving image decoding device described in the present invention, and converts a bit stream of image data into a decoding method corresponding to the encoding method described in the above embodiment. By decoding, the reproduced moving image data is generated and supplied to the display unit ex202 via the LCD control unit ex302, whereby, for example, the moving image data included in the moving image file linked to the homepage is displayed. . At this time, at the same time, the audio processing unit ex305 converts the audio data into analog audio data, and then supplies the analog audio data to the audio output unit ex208, whereby, for example, the audio data included in the moving image file linked to the homepage is reproduced. You.

  It should be noted that the present invention is not limited to the example of the system described above, and digital broadcasting using satellites and terrestrial waves has recently become a topic. As shown in FIG. Any of the video decoding devices can be incorporated. Specifically, at the broadcasting station ex409, the bit stream of the video information is transmitted to the communication or the broadcasting satellite ex410 via radio waves. The broadcast satellite ex410 receiving this transmits a broadcast radio wave, receives this radio wave with a home antenna ex406 having a satellite broadcast reception facility, and outputs the radio wave to a television (receiver) ex401 or a set-top box (STB) ex407. The device decodes the bit stream and reproduces it. In addition, the moving picture decoding apparatus described in the above embodiment can be mounted on a reproducing apparatus ex403 that reads and decodes a bit stream recorded on a storage medium ex402 such as a CD or DVD that is a recording medium. . In this case, the reproduced video signal is displayed on the monitor ex404. Further, a configuration is also conceivable in which a moving image decoding apparatus is mounted in a set-top box ex407 connected to a cable ex405 for cable television or an antenna ex406 for satellite / terrestrial broadcasting, and this is reproduced on a monitor ex408 of the television. At this time, the moving picture decoding device may be incorporated in the television instead of the set-top box. In addition, a car ex412 having an antenna ex411 can receive a signal from the satellite ex410 or a base station ex107 or the like, and can reproduce a moving image on a display device such as a car navigation ex413 included in the car ex412.

  Furthermore, an image signal can be encoded by the moving image encoding device described in the above embodiment and recorded on a recording medium. As a specific example, there is a recorder ex420 such as a DVD recorder for recording an image signal on a DVD disc ex421 or a disc recorder for recording on a hard disk. Furthermore, it can be recorded on the SD card ex422. If the recorder ex420 includes the moving picture decoding device described in the above embodiment, the video signal recorded on the DVD disc ex421 or the SD card ex422 can be reproduced and displayed on the monitor ex408.

The configuration of the car navigation system ex413 may be, for example, a configuration excluding the camera unit ex203, the camera interface unit ex303, and the image encoding unit ex312 from the configuration illustrated in FIG. 17, and the same applies to the computer ex111 and the television (receiver). ) Ex401 and the like are also conceivable.
In addition, the terminal such as the above-mentioned mobile phone ex114 has three mounting formats, in addition to a transmitting / receiving terminal having both an encoder and a decoder, a transmitting terminal having only an encoder and a receiving terminal having only a decoder. Can be considered.

As described above, the moving picture coding method or the moving picture decoding method described in the above embodiment can be used for any of the devices and systems described above. The effect can be obtained.
Further, the present invention is not limited to the above embodiment, and various changes or modifications can be made without departing from the scope of the present invention.

  As described above, the moving picture encoding method and the moving picture decoding method according to the present invention encode each picture constituting a moving picture by using, for example, a mobile phone, a DVD device, and a personal computer to form an encoded stream. It is useful as a method for generating or decoding the generated coded stream.

FIG. 1 is a block diagram (Embodiment 1) showing a configuration of a video encoding device of the present invention. FIG. 3 is a diagram (first embodiment) for explaining the concept of the moving picture decoding method of the present invention. [Fig. 3] Fig. 3 is a block diagram (first embodiment) illustrating a configuration of a video decoding device of the present invention. FIG. 4 is an operation flowchart (first embodiment) of the moving picture decoding method of the present invention. FIG. 9 is an operation flowchart (Embodiment 1) of the free area securing process of the moving picture decoding method of the present invention. FIG. 3 is a block diagram (Embodiment 2) showing the configuration of the video decoding device of the present invention. FIG. 10 is an operation flowchart (second embodiment) of the moving picture decoding method of the present invention. It is a figure which shows the structure of the data which the moving image encoding method of this invention outputs, and the structure of the data which the moving image decoding method of this invention inputs. FIG. 9 is a block diagram (Embodiment 3) showing a configuration of a video encoding device of the present invention. FIG. 10 is a diagram (third embodiment) for explaining the concept of the moving picture encoding method of the present invention. FIG. 9 is an operation flowchart (third embodiment) of the moving picture encoding method of the present invention. FIG. 14 is a diagram (fourth embodiment) for explaining the concept of the moving picture coding method of the present invention. FIG. 9 is an operation flowchart (Embodiment 4) of the moving picture encoding method of the present invention. It is an explanatory diagram of a recording medium for storing a program for realizing the moving image encoding method and the moving image decoding method of each embodiment by a computer system, (a) of a flexible disk which is a recording medium body Explanatory diagram showing an example of a physical format, (b) Appearance, sectional structure, and explanatory diagram showing a flexible disk as viewed from the front of a flexible disk, (c) Configuration for recording and reproducing the above program on the flexible disk FD FIG. FIG. 1 is a block diagram illustrating an overall configuration of a content supply system that realizes a content distribution service. It is a figure showing an example of a mobile phone. FIG. 2 is a block diagram illustrating an internal configuration of the mobile phone. FIG. 1 is a block diagram illustrating an overall configuration of a digital broadcasting system. FIG. 11 is a block diagram illustrating a configuration of a conventional moving image encoding device. It is a figure explaining the concept of a display order (POC) and a referenced picture number. FIG. 11 is a diagram illustrating an operation of deleting a picture in order to secure a free area in a memory when there is a picture marked as unused. FIG. 11 is a diagram illustrating an operation of deleting a picture in order to secure a free area in a memory when there is no picture marked as unused. FIG. 15 is a block diagram illustrating a configuration of a conventional video decoding device. FIG. 10 is a flowchart of a memory-related operation of the conventional video decoding device. FIG. 10 is a flowchart of an operation of a free space securing process of the conventional video decoding device. FIG. 9 is a conceptual diagram illustrating a problem that a discontinuity in a sequence causes a discontinuity in display order information POC and an undisplayed picture is deleted. It is a conceptual diagram explaining operation | movement of an invalid picture. FIG. 15 is a block diagram illustrating a configuration of a conventional video decoding device. FIG. 10 is a flowchart of an operation of an invalid picture of the conventional video decoding device. FIG. 7 is a conceptual diagram illustrating a problem that a discontinuity in a sequence causes a discontinuity in a frame number FN, and an invalid picture deletes an undisplayed picture. FIG. 3 is a conceptual diagram illustrating the structure of a conventional MPEG-2 stream. FIG. 11 is a conceptual diagram illustrating a conventional JVT video encoding method. FIG. 11 is an operation flowchart of a conventional JVT video encoding method. FIG. 7 is a conceptual diagram illustrating a problem caused by the degree of freedom of JVT coding at the time of editing or random access.

Explanation of reference numerals

Reference Signs List 101 rearrangement memory 102 encoding scheduling unit 103 motion estimation unit 104 subtraction operation unit 105 encoding unit 106 motion compensation unit 107 variable length encoding unit 108 decoding unit 109 addition operation unit 110, 111 memory 112 flag information generation unit 113 variable Long encoding section 201, 213, 402, 411 Variable length decoding section 202 Image decoding section 203, 214 Edit detection section 204 MMCO decoding section 205, 212, 401, 412 Memory management section 206 Memory 211 FN gap detection section

Claims (15)

A moving picture coding method for coding a moving picture signal on a picture basis to generate a coded stream,
A flag information creating step of creating flag information indicating that the order of the pictures is discontinuous;
An information adding step of adding the flag information to the encoded stream. The moving picture coding method according to claim 1, wherein, in the flag information creating step, when the display order information of the pictures is discontinuous, the order of the pictures is discontinuous. The moving picture encoding method according to claim 1, wherein, in the flag information creating step, if the encoding order information of the pictures is discontinuous, the order of the pictures is discontinuous. The moving picture coding method according to claim 1, wherein, in the information adding step, the flag information is added between two pictures in which the order of the pictures in the coded stream is discontinuous. The video encoding method further includes:
Including a position information creating step of creating position information specifying a position where the order of the pictures is discontinuous,
The moving image encoding method according to claim 1, wherein, in the information adding step, the position information is added together with the flag information. A moving picture coding method for coding a moving picture signal on a picture basis to generate a coded stream,
An encoding step of encoding a picture whose display order is later than the first intra-coded picture in a predetermined coding unit composed of a plurality of pictures so as to be included in a coding unit subsequent to the coding unit. A moving picture coding method comprising: In the encoding step, the display order of the pictures in the predetermined coding unit is continuous, and the display of the pictures in the predetermined coding unit immediately after the predetermined coding unit in the coding order is performed. The moving image encoding method according to claim 6, wherein encoding is performed before the order. A moving image decoding method for decoding an encoded stream in units of pictures,
An information extracting step of extracting flag information indicating that the order of the pictures is discontinuous;
A management step of managing an area for storing a decoded picture based on the flag information. The flag information is information indicating that display order information of pictures is discontinuous,
In the managing step, a picture whose display order is the earliest among decoded pictures stored in the area is determined based on the display order information and the flag information, and the determined picture is deleted. The moving picture decoding method according to claim 8, wherein the moving picture is a picture. In the managing step, clip information to be updated when the flag information is extracted is provided to the decoded picture stored in the area, and the area is assigned to the area based on the display order information and the clip information. The moving picture decoding method according to claim 9, wherein a picture whose display order is the earliest among the stored decoded pictures is determined, and the determined picture is set as a deletion target picture. The video decoding method may further include:
When the coding order information of the picture is discontinuous, the method includes an invalid picture storing step of storing an invalid picture in the area,
The flag information is information indicating that the encoding order information is discontinuous,
In the managing step, based on the flag information and the encoding order information, determine whether to store an invalid picture in the area,
The moving picture decoding method according to claim 8, wherein in the invalid picture storing step, an invalid picture is stored in the area based on a determination result in the management step. A moving image encoding apparatus that encodes a moving image signal on a picture basis to generate an encoded stream,
Flag information creating means for creating flag information indicating that the order of the pictures is discontinuous,
A video encoding device, comprising: information adding means for adding the flag information to the encoded stream. A moving picture decoding apparatus for decoding an encoded stream in units of pictures,
Information extraction means for extracting flag information indicating that the order of the pictures is discontinuous,
Management means for managing an area for storing a decoded picture based on the flag information. A program for encoding a moving image signal on a picture basis to generate an encoded stream,
A flag information creating step of creating flag information indicating that the order of the pictures is discontinuous;
An information adding step of adding the flag information to the encoded stream. A program for decoding an encoded stream in units of pictures,
An information extracting step of extracting flag information indicating that the order of the pictures is discontinuous;
And a management step of managing an area for storing a decoded picture based on the flag information.

Images