JP5299861B2 - Image display device - Google Patents

Description

  The present invention relates to an image display device.

  As background art of this technical field, for example, there is JP-A-2003-18410 (Patent Document 1). The gazette states that “the image decompression device of this transform code refers to an image code that has already been wavelet transformed from an image size specified by the user when decompressing a natural wave image from a code that has already been subjected to hierarchical wavelet transform, The hierarchy number that is the highest hierarchy that is the largest or equal to the inner ring of the specified decompressed image size and the smallest hierarchy that is the smallest or equal to the largest that exceeds the designated stretched image size is obtained, and this hierarchy number is determined from the highest hierarchy. "The decompressed image is created by performing inverse wavelet transform of the layers existing up to the layer of layer number +2."

JP 2003-18410 A

  In recent years, digital devices that record and reproduce images, such as recorders and digital cameras, have increased capacity and have increased opportunities to handle large amounts of images. With such a background, the graphical user interface (GUI) overlays multiple thumbnail images on one screen so that the user can easily find the target image from a large amount of image data. Higher functions such as displaying or displaying a large amount of small thumbnail images on one screen are advancing. However, in order to achieve this, a large amount of thumbnail images must be read from the recording medium and displayed in an enlarged / reduced manner. Therefore, the bandwidth of the recording medium, the processing amount of the enlargement / reduction unit, and the like become a bottleneck. There is a problem that the display speed of the display decreases.

  Japanese Patent Application Laid-Open No. 2004-228561 describes a method of using a wavelet transform method to obtain a high-quality expanded image at high speed according to an image size designated by a user. However, when the number of processed images increases, the processing time increases accordingly, and no specific solution is disclosed for the display speed becoming slow. An object of the present invention is to provide an image display device that realizes high-speed processing of graphics display and high image quality.

  The above object can be achieved by the invention described in the claims.

  According to the present invention, high-speed processing and high image quality of graphics display of an image display device can be realized.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is an image display device representing a first embodiment of the present invention. This apparatus is a recording / reproducing apparatus capable of recording / reproducing digital broadcasts, and describes a function of displaying thumbnail images for displaying a list of recorded programs.

  1 includes a tuner unit 100, a recording medium 101, a decoding unit 102, a subband dividing unit 103, a subband synthesizing unit 104, an enlargement / reduction unit 105, a graphic output unit 106, and a display device. It consists of 107. When recording a broadcast wave, the MPEG2 video stream received by the tuner unit 100 is recorded on the recording medium 101. The stream recorded in the recording medium 101 generates a decoded image by decoding only the first frame in the decoding unit 102 in order to create a thumbnail image used for program list display. Further, the sub-band division unit 103 performs sub-band division on the decoded image to create a sub-band division image and records it on the recording medium 101. Here, in the case of using encoding in which subband division is performed and compressed, such as JPEG2000, the decoding unit 102 may directly record the subband division image being decoded on the recording medium 101.

  When playing back the stream recorded on the recording medium 101, the decoding unit 102 decodes the stream to generate a decoded image, and the graphics output unit 106 generates an output image from the decoded image, and outputs the output image to the display device 107.

  On the other hand, when displaying a thumbnail image for selecting which stream to play, only the necessary subband components of the subband divided image are read from the recording medium 101, and the subband synthesis unit 104 performs subband synthesis, Generate a composite image. Next, the enlargement / reduction unit 105 enlarges / reduces the subband composite image, and the graphics output unit 106 generates a graphics image and outputs it to the display device 107.

  Next, a subband division synthesis method will be described.

  FIG. 2 shows a configuration example of a filter bank for subband division synthesis. A dotted line 201 represents the subband division processing. 202 and 204 indicate that half-sampling is performed after the low-pass filter is applied, and 203 and 205 indicate that half-sampling is performed after the high-pass filter is applied. First, the input data is divided into high frequency components at 203 to create H component data. Next, after the input data is divided into low-frequency components at 202 to create L-component data, the L-component data is further divided into low-frequency components at 204 to create LL component data. 205 is divided into high-frequency components to create LH component data.

  A dotted line 202 represents the subband synthesis process. 207 and 209 mean that the low-pass filter is applied after upsampling twice, and 208 and 210 mean that the high-pass filter is applied after upsampling twice. Subband synthesis is performed in order from the low-frequency component. The L component data is generated by processing the LL component data 207 and the LH component data 208 and adding them together. Next, reconstructed data is created by adding the data obtained by processing the L component data at 209 and the data processed by the H component at 210. The reconstructed data matches the input data when a completely reconfigurable low-pass filter and high-pass filter are used.

  FIG. 3 shows a sub-band divided image when the filter bank shown in FIG. 2 is applied to a thumbnail image of 360 pixels × 240 pixels. When the filter bank shown in FIG. 2 is applied to an image, the subband divided image shown in FIG. 3 can be generated by performing the subband division twice in the horizontal direction and the vertical direction. The sub-band split image includes an LLLL component 211 (90 pixels × 60 pixels), an LLLH component 212 (90 pixels × 60 pixels), an LLHL component 213 (90 pixels × 60 pixels), and an LLHH component 214 (90 pixels × 60 pixels). , LH component 215 (180 pixels × 120 pixels), HL component 216 (180 pixels × 120 pixels), and HH component 217 (180 pixels × 120 pixels). Further, the LLLL component 211, the LLLH component 212, the LLHL component 213, and the LLHH component 214 surrounded by a thick line are collectively referred to as an LL component 218. The upper left is a low-frequency subband component, and the lower left is a higher-frequency subband component. Here, the LLLL component 211 is a reduced image of 90 pixels × 60 pixels in which the image before subband division is ¼ both vertically and horizontally.

  Next, a method for synthesizing sub-band divided images will be described with reference to FIG. When subband synthesis is performed, synthesis is performed from the low-frequency subband components. For example, by performing subband composition of the LL component 218, a reduced image of 180 pixels × 120 pixels, which is a half of the image before subband division in both vertical and horizontal directions, can be formed. Further, by performing subband synthesis of this reduced image, LH component 215, HL component 216, and HH component 217, a thumbnail image of 360 pixels × 240 pixels before subband division can be restored.

  The present invention utilizes the fact that a reduced image can be created using only low-band sub-band component data, and thereby improves the drawing speed by changing the sub-band component for sub-band synthesis. For example, the amount of data that can be processed in one frame of the enlargement / reduction unit is one image of HD size (1920 pixels × 1080 pixels). Here, if 16 HD-size images are reduced and displayed on the HD-size display, only one image can be processed per frame, so that it takes 16 frames. On the other hand, when sub-band divided images are used, by combining only the LLLL component 211 with the sub-band, the amount of data processed by the enlargement / reduction unit becomes 1/16 and can be processed in one frame. In this manner, by changing the subband component for performing the subband synthesis, the processing amount of the enlargement / reduction unit can be reduced, so that the drawing speed can be increased. As for the image quality, when displaying on an HD size display, the resolution is the same as the subband composite image that is subband combined using only the LLLL component 211, because the HD size image is reduced to 1/16 and displayed on the display. Absent. Therefore, high image quality and high speed can be realized.

  Next, an example in which the recording medium access bandwidth becomes a bottleneck will be described in detail.

  FIG. 4 is a graphics display that displays a large number of thumbnail images of the same size and searches for viewed content, and has a function of switching the size of thumbnail images between a large size and a medium size. The thumbnail display 300 displays 16 medium-sized thumbnail images 301, and the size of each thumbnail image is 180 pixels × 120 pixels (4: 2: 0). The thumbnail display 303 displays four large thumbnail images, and the size of each thumbnail image is 360 pixels × 240 pixels (4: 2: 0). The large thumbnail display 300 is an enlarged display of four medium size thumbnail images within the dotted line 302 within the dotted line 305.

  As an example, an HDD is used as a recording medium, and the transfer speed is set to 35 Mbytes / sec. At this time, a case where a thumbnail image is displayed in real time on a 60 fps television will be described. The bandwidth of 35 Mbytes / sec can be used for graphics display processing. Since each thumbnail image of 360 pixels × 240 pixels (4: 2: 0) is about 0.13 Mbytes, calculating how many frames can be displayed per frame is 35 Mbytes / sec ÷ 60 fps ÷ 0.13 Mbytes≈4.5 It becomes.

  FIG. 5 shows the bandwidth of both thumbnail displays 300 and 301. Since the thumbnail display 303 of a large size has four thumbnail images of 360 pixels × 240 pixels, all the subband components 211 to 217 are read from the recording medium for all four images, and even if subband synthesis is performed, about 31 Mbytes / sec And can be displayed in real time. On the other hand, 16 thumbnail images of medium size are required, and if all subband components 211 to 217 are read from the recording medium and subband combined and reduced, the band is about 124 Mbytes / sec, which is 4 times, and can be processed in real time. It will exceed the width. Therefore, in order to suppress the bandwidth to 35 Mbytes / sec, if the subband component for performing subband synthesis per sheet is limited to ¼ LL component 218, the recording medium access bandwidth is also reduced to about a quarter. It becomes 31 Mbytes / sec, and can be suppressed within the bandwidth that can be displayed in real time.

  In this way, by changing the subband component read from the recording medium in accordance with the number of thumbnail images displayed, the number of images displayed can be kept below the bandwidth that can be displayed in real time. So it can be displayed in real time. As for the image quality of the thumbnail display 300, there are a case where a thumbnail image of 360 pixels × 240 pixels is reduced to 180 pixels × 120 pixels and a case where a reduced image of 180 pixels × 120 pixels is created using subband synthesis. The image quality is the same level. Therefore, high-speed processing and high image quality can be realized.

Further, there are advantages regarding the bandwidth of the recording medium and the recording capacity of the recording medium. In a normal recording / reproducing apparatus, when the image size changes in the same thumbnail image such as a large size and a medium size, it can be realized by the following two methods.
(1) When a large-sized thumbnail image 304 is stored in a recording medium and a medium-sized thumbnail image 301 is displayed, the large-sized thumbnail image 304 is read from the recording medium and reduced.
(2) All the medium-sized thumbnail images 301 and the large-sized thumbnail images 304 are stored in the recording medium, and the necessary-sized thumbnail images are read from the recording medium.
Here, consider the 16 thumbnail images displayed in the thumbnail display 300.

  Regarding the recording medium access bandwidth, the method (1) is about 124 Mbytes / sec, and the method (2) is about 31 Mbytes / sec.

  On the other hand, regarding the storage capacity of the recording medium, in the method (1), the storage capacity of the recording medium is about 16 large thumbnail images (360 pixels × 240 pixels, 4: 2: 0). However, in the method (2), both the large-sized thumbnail image and the medium-sized thumbnail image (180 pixels × 120 pixels, 4: 2: 0) must be stored. The capacity is about 2.6 Mbytes.

  In the present invention, since one sub-band divided image has only to be stored for each thumbnail image, the recording capacity of the recording medium is about 2.1 Mbyte, which is the same as (1). Furthermore, the bandwidth of the recording medium access is also about 31 Mbytes / sec, and the advantages of both the methods (1) and (2) can be obtained.

  Next, an example in which a thumbnail display 303 having a large size is displayed on an SD card having a transfer rate of 8 Mbytes / sec and an SD card having a transfer rate of 5 Mbytes / sec.

  In both SD cards, when the subband components 211 to 217 are read from the recording medium, it becomes about 31 Mbytes / sec and cannot be displayed in real time. Therefore, in the case of an SD card with a transfer rate of 8 Mbytes / sec, when the LL component 218 is read from the recording medium, it is about 7.8 Mbytes / sec and can be displayed in real time. On the other hand, in the case of an SD card with a transfer rate of 5 Mbytes / sec, if only the LLLL component 211 is read from the recording medium, it becomes about 2 Mbytes / sec, which can be displayed in real time.

  In this way, by changing the subband component read from the recording medium in accordance with the transfer speed of the recording medium, the image quality deteriorates, but it is possible to display in real time while minimizing the deterioration.

FIG. 6 shows a method for storing subband divided images for efficient access to a recording medium when a plurality of subband divided images are stored. As shown in FIG. 6, a data group 500 in which LLLL components 211 of a plurality of subband divided images are collected, and a data group 501 in which LLLH components 212 of a plurality of subband divided images are collected. Save to consecutive addresses. Saving in this way is effective, for example, when displaying a large number of small thumbnail images 301 of FIG. Normally, when storing in units of sub-band divided images, jumping addresses are accessed, so if data cannot be divided in units of data reading, fractional data is discarded. On the other hand, in the storage method, the LLLL component 211 can be read from the recording medium as continuous data. As a result, data to be discarded is minimized, and access without waste is possible.
As an application example of FIG. 6, an example in which the subband components to be stored together is changed is shown in FIG. 7. FIG. 7 shows the data group 600 of the LLLL component 211, the data group 601 of the LLLH component 212, the LLHL component 213, and the LLHH component 214, and the data group 602 of the LH component 215, HL component 216, and HH component 217. It is an example. By storing in this way, it is possible to access the recording medium without waste when the size is changed in three stages of 90 pixels × 60 pixels, 180 pixels × 120 pixels, 360 pixels × 240 pixels. It should be noted that the subband components to be stored in a solid state may be changed according to the intended use.

  FIG. 8 shows a graphics display 700 when thumbnail images of different sizes are displayed on the same screen. This graphics display is used when searching for a program to be viewed by scrolling up and down. In this example, medium-sized thumbnail images (180 pixels × 120 pixels) 701 and 703 are displayed above and below, and a large-sized thumbnail image (360 pixels × 120 pixels) 702 is displayed in the center.

  As an example, a case will be described in which only the half bandwidth of 17.5 Mbytes / sec can be used for graphics display processing because the HDD of FIG. In order to synthesize all the subband components 211 to 217 with three thumbnail images, a bandwidth of about 23 Mbytes / sec is required, so real-time display cannot be performed. Therefore, priority is given to those having a large image size, and the subband components read from the recording medium are changed. Specifically, in the large-sized thumbnail image 702, all the subband components 211 to 217 are read from the recording medium and are subband synthesized. On the other hand, the medium-sized thumbnail images 701 and 703 read out the LL component 218 from the recording medium and perform subband synthesis. Then, the total bandwidth of the recording medium access is about 11.5 Mbytes / sec, and real-time display is possible. As described above, the sub-band component read from the recording medium is suppressed within the bandwidth that can be displayed in real time according to the size of the image, thereby enabling real-time display. Further, since the size of the thumbnail image created by the sub-band synthesis and the display size are the same, the image quality is the same level as compared with the case where the reduction is performed by sub-band synthesis with all components. Therefore, high-speed processing and high image quality can be realized.

  FIG. 9 is a graphic display 800 in which thumbnail images are gradually reduced in size as they go backward in order to express perspective in the same screen. This graphics display is used when searching for a program to be viewed by scrolling left and right. The same effect can be obtained by changing the sub-band components that are read from the recording medium according to the image size in the same way as in the example of FIG. Is omitted.

  In the graphics display 900 of FIG. 10, the size of the thumbnail images in the screen is all 360 pixels × 240 pixels, and the thumbnail image 906 surrounded by a thick line is a thumbnail image being selected by the user's operation. . An example of changing subband components to be subband synthesized according to importance will be described. As in the example of FIG. 4, assuming that the bandwidth that can be displayed in real time is 35 Mbytes / sec, the six thumbnail images 901 to 906 have a bandwidth of about 46.5 Mbytes when all the subband components 211 to 217 are read from the recording medium. Since it is / sec, it cannot be displayed in real time. Therefore, the subband component read from the recording medium is changed according to the importance. Since the thumbnail image 906 is a material for determining whether the user views the thumbnail image 906, the importance is high. Therefore, all the subband components 211 to 217 are read from the recording medium, and are combined with each other and displayed at a high resolution. On the other hand, the other thumbnail images 901 to 905 are less important because it is only necessary to know what programs are. Therefore, only the LL component 218 is read from the recording medium and displayed in an enlarged manner. As a result, the image quality of the important thumbnail image 906 is not degraded, and the bandwidth is about 17.5 Mbytes / sec, so that real-time display is possible. Further, the unselected thumbnail images 901 to 905 are blurred thumbnail images because the 120-pixel subband composite image of 180 pixels is enlarged and displayed to 360 pixels × 180 pixels. On the other hand, since the selected thumbnail image is a clear thumbnail image, there is an effect that contrast can be given to the selected thumbnail image.

FIG. 11 is an image display device representing the second embodiment, and is a device that records and reproduces an image taken by a digital camera or a video camera. Portions common to those in FIG. 1 are denoted by the same symbols, and description thereof is omitted. The sensor unit 108 has a function of converting the input light into a digital image. The encoder unit 109 converts the digital image into MPEG2 or H.264. H.264 or the like is used to generate a stream, and the stream is written to the recording medium 101. The output determination unit 110 has a function of determining to which display device the data is output, and determines whether or not an external terminal such as an HDMI terminal or a D terminal is connected. Reference numeral 111 denotes a low-resolution display device such as an LCD, and 112 denotes a high-resolution display device such as a television. A case where a subband component for subband synthesis is changed according to the resolution of the output device will be described by taking a video camera as an example.
In general, a video camera takes a picture while viewing an image of a low-resolution display device such as an LCD at the time of photographing, and reproduction for confirmation is performed on the LCD, but it is not necessary to display at a high resolution. Therefore, when the output determination unit determines that there is no external output, the sub-band composite image of only the LLLL component 211 is displayed. Then, since the whole processing amount can be suppressed to 1/16, the power consumption can also be suppressed to 1/16. As a result, there is an effect that the recording time is increased.

  FIG. 12 shows an image display apparatus according to the third embodiment. Parts common to those in FIG. 1 are denoted by the same symbols, and description thereof is omitted. The memory 113 is a memory that temporarily stores the subband divided images. In the previous examples, even if the same thumbnail image was displayed, it was read from the recording medium every frame. However, by using the memory, the subband component read from the recording medium once is held in the memory. In addition, since the subband composite image can be created using the subband components stored in the memory for each frame, access to the recording medium is eliminated and power consumption can be reduced.

  Further, a method of using a memory that is effective when subband division is used will be described. For example, the bandwidth that can be displayed in real time in the graphics display of FIG. 8 is defined as 8.75 Mbytes / sec, which is a quarter of the example of FIG. If all the subband components 211 to 217 are combined in a subband in order to create the middle thumbnail image 702, a bandwidth of about 11.5 Mbytes / sec is required, and real time display cannot be performed. To achieve thumbnail display at the same resolution, if a normal thumbnail image is stored on the recording medium, half of the thumbnail image data is read from the recording medium and stored in the memory at the first frame. In the second frame, the remaining half of the data is read from the recording medium and displayed after all of the data has been collected. Then, the display is not performed in real time but after one frame is delayed. On the other hand, when sub-band divided images are used, the center large thumbnail image 702 is enlarged and displayed by reading the three LL components 218 from the recording medium in the first frame and synthesizing the sub-bands. Can be displayed. In the second frame, the LH component 215, HL component 216, and HH component 217 are read from the recording medium and combined with the LL component 218 stored in the memory to create a sub-band composite, so that a large thumbnail image in the center can be obtained. 702 can be displayed with high resolution. As described above, when the sub-band divided image is used, it can be displayed in stages, so that the image quality of the large thumbnail image 702 at the center is lowered but can be displayed in real time. In addition, when scrolling thumbnail images continuously down or up, only the low-resolution thumbnail images that can be processed in one frame period are displayed, and the central thumbnail image that is originally performed in the second frame is increased in resolution. By performing the next scroll screen process without performing the process, it is possible to display the scroll screen for each frame.

  As another example, an example will be described in which a predicted use subband component expected to be read from the recording medium when a specific operation is performed next is read from the recording medium and held in the memory.

  For example, when the LL component 218 is read from the recording medium, when the same thumbnail image is displayed and the recording medium is not accessed, the LH component 215, the HL component 216, and the HH component 217 are read from the recording medium, Keep it in memory. Then, even if the thumbnail image 701 is moved to the center as shown in FIG. 13, the only subband component to be read is the LL component 218 of the thumbnail image 1201. It can be displayed with the eyes. When the thumbnail display in FIG. 8 is scrolled down as the expected use subband component, the thumbnail image 1201 in the thumbnail display in FIG. 13 is also displayed. Therefore, the subband component in the thumbnail image 1201 is used as the expected subband data. In the case of displaying a list of thumbnail images as shown in FIG. 4, the subband components of the thumbnail images displayed on the next page may be stored in the memory.

  In this way, when the same thumbnail image is displayed and access to the recording medium does not occur, the expected operation subband component is read from the recording medium and stored in the memory, and the following operation is performed Since there is no access to the recording medium, high-speed processing is possible.

  FIG. 14 shows an image display apparatus according to the fourth embodiment. Although the configuration is the same as that of FIG. 12, the data stored in the memory 113 is different, and the image data of the sub-band synthesized image or the thumbnail image stored in the recording medium is stored in the memory. When the same image is displayed or when the image is enlarged or reduced at the same resolution, access to the recording medium and subband synthesis processing are eliminated, so that power consumption can be further reduced than in the case of FIG.

  Furthermore, by storing the expected use subband composite image or expected use image data that is expected to be read out from the recording medium in the same manner as the expected use subband component in the example of FIG. When the next operation is performed, access to the recording medium is eliminated, so that high-speed processing is possible.

  Note that the memory 113 shown in FIGS. 14 and 12 may be shared and hold both data, or each memory may be held using different memories.

It is an example of the image display apparatus of a 1st Example. It is an example of a filter bank structure. It is an example of a subband division | segmentation image. This is an example of displaying the same thumbnail image with different image sizes. It is an example of the processing amount of FIG. It is an example regarding the memory | storage method. It is an application example regarding the storage method of a memory. It is an example of graphics display when displaying thumbnail images of different sizes within the same screen. It is an example of a graphics display in which thumbnail images are gradually reduced and displayed in the same screen. It is an example of the graphics display when changing the resolution according to the importance in the same screen. It is an example of the image display apparatus of a 2nd Example. It is an example of the image display apparatus of a 3rd Example. It is an example of the graphics display when the thumbnail image with a small display size at the top of FIG. 7 moves to the center. It is an example of the image display apparatus of a 4th Example.

Explanation of symbols

100 tuner section
101 recording media
102 Decoding part
103 Subband division part
104 Subband synthesis unit
105 Enlarging / reducing part
106 Graphics output section
107 Display device

Claims (3)

A recording medium having stored a subband division image obtained image data by a sub-band division,
A memory capable of holding at least one subband component of a subband divided image read from the recording medium;
And sub-band synthesis using at least one sub-band component held with the memory to at least one sub-band component sub-band division image read from the subband division images or the recording medium is read from said recording medium A subband synthesis unit for generating a subband synthesized image;
A graphics output unit for generating and outputting a graphics image from the subband composite image;
With
When displaying a sub-band composite image output from the graphics output unit and displayed as a graphics image at a higher resolution, the high-frequency sub-band component not stored in the memory read from the recording medium and the memory Subband synthesis using the subband components held in the subband to generate a subband composite image,
An image display apparatus , wherein a use prediction subband component is stored in the memory when access to the recording medium does not occur . The image display device according to claim 1,
In the case of storing a plurality of subband divided images on the recording medium, the subband divided images are stored by storing the same subband components at consecutive addresses. The image display device according to claim 1 or 2,
An image display device that changes a sub-band divided image read from the recording medium in accordance with a transfer speed of the recording medium.

Images