JP2012156640A - Image display apparatus and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display apparatus for displaying displacement with time of an identical object, and an imaging apparatus capable of performing imaging in consideration of a display mode thereof.SOLUTION: An image display apparatus with a display means 18 for displaying an image includes a region designation means 16 for designating, as a reference part, a partial image region of reference images selected from a plurality of images recorded in a recording means 14; a similar image retrieval means 11b for retrieving and extracting an image containing an image region substantially the same as the reference part designated by the region designation means from the plurality of images recorded in the recording means; and a control means 11 for performing image processing on the basis of the reference part that becomes an image region common to the image retrieved and extracted by the similar image retrieval means and the reference image, and displaying a resulting image on a display screen of the display means.

Description

  The present invention relates to an image display device and an imaging device, and more specifically, to select and extract a similar image from a plurality of recorded images, and to display the selected and extracted similar image in a predetermined form. It relates to the device.

  2. Description of the Related Art Conventionally, there is a so-called fixed-point shooting method in which a camera is fixed and shooting is periodically performed at a predetermined time interval in order to record a change with time of the same shooting target (subject). However, for example, when plant or child growth is performed with a fixed point shooting method, it is necessary to set the time interval to a long time, and the overall shooting period will also be long. I can't keep it fixed. In addition, as a subject to be photographed using the fixed point photographing method, there is a tendency that the change can be expressed more easily as the subject gradually changes over a long time.

  Therefore, for example, Japanese Patent Application Laid-Open No. 2005-2005 discloses a technique for easily capturing an image in accordance with the previous shooting state when shooting again after a long time has passed without fixing the camera. Various proposals have been made in Japanese Patent No. -26874.

  The means disclosed in Japanese Patent Application Laid-Open No. 2005-26874 makes it easy to match the angle of view, magnification, hue, etc. of the current captured image with the previous captured image by displaying the previous captured image during observation shooting. And can shoot.

  On the other hand, various techniques have been proposed in the prior art, for example, in Japanese Unexamined Patent Application Publication No. 2004-48358, etc., for displaying a change easily using a plurality of images photographed using a fixed point photographing technique. ing.

  The means disclosed by the above Japanese Patent Application Laid-Open No. 2004-48358 relates to an image display device, a display method, and a display program that display changes in a captured image obtained from a fixed point camera that performs intermittent shooting in an easy-to-understand manner. The latest image is displayed in a large size, and the previous three images are displayed side by side with this latest image. Each time an image is received from the camera, the image is sequentially shifted and displayed.

  On the other hand, a general user of a camera usually uses one camera owned by the user under various circumstances such as family record, travel commemorative photo, and hobby photography. Among such usage situations, for example, photography is performed to record changes in the growth of plants in the garden and the growth of children.

  In recent cameras, the recording medium for recording image data has been increased in capacity and price has been reduced. Therefore, for example, a large amount of image data acquired over a long period of time can be collected in one recording medium, or the recording medium can be used for each application or for each user. Has been done.

JP 2005-26874 A JP 2004-48358 A

  However, the means disclosed in Japanese Patent Application Laid-Open No. 2005-26874 relates to a technique for performing image capturing on the premise of fixed point image capturing, and is disclosed in Japanese Patent Application Laid-Open No. 2004-48358. The means is a technique for displaying a plurality of images acquired using a fixed point photographing method.

  In a normal case, the user of the camera 1 usually performs shooting unconsciously, taking an object of interest as a shooting target, without being particularly conscious of a technique such as fixed-point shooting. As described above, when the user performs photographing according to his / her own interest, a large amount of photographed image data is collected on the recording medium. A large amount of captured image data in such a recording medium may include a plurality of images that have been captured again after a certain amount of time for the same subject.

  In this way, even when a photographer is shooting without being aware of a technique such as fixed-point shooting, similar images are collected based on a plurality of collected image data, If it can be displayed in such a way that changes can be seen, it can be considered to contribute to the enjoyment of using the camera.

  The present invention has been made in view of the above-described points, and an object of the present invention is to perform normal shooting without being particularly conscious of the user of the shooting apparatus (camera) and to record a plurality of recording media. When a plurality of image data is recorded, a plurality of images including substantially the same (similar) imaging target are selected from the plurality of image data recorded on the recording medium, and predetermined image processing is appropriately performed on the selected images. And providing an image display device that displays the plurality of images in an optimum form, and a photographing device that includes the image display device and that can perform photographing in consideration of the display form.

  In order to achieve the above object, an image display device according to an aspect of the present invention includes an image display device including a display unit that displays an image, and a reference image selected from a plurality of images recorded in the recording unit. Of the plurality of images recorded in the recording means, an area designating unit for designating a part of the image area as a reference part and an image including an image area substantially the same as the reference part designated by the area designating unit. Image obtained by performing image processing based on the reference portion that is a common image area for the reference image and the image searched and extracted by the similar image search means and the reference image Control means for displaying on the display screen of the display means.

  In addition, an imaging device of one embodiment of the present invention includes an imaging unit that photoelectrically converts a subject image to acquire an image, a recording unit that records an image acquired by the imaging unit, and a display unit that displays the image. In the photographing apparatus comprising the apparatus, an area designating unit that designates a part of an image area of a reference image selected from a plurality of images recorded in the recording means as a reference part, and the area designation Similar image search means for searching and extracting from the plurality of images already recorded in the recording means for an image including substantially the same image area as the reference portion specified by the means, and the similar image search means for searching and extracting Control means for performing image processing on the basis of the reference portion which is an image area common to the obtained image and the reference image, and displaying the resulting image on the display screen of the display means. And wherein the door.

  According to the present invention, when a user of a photographing apparatus (camera) performs normal photographing without being particularly conscious and records a plurality of pieces of image data on a recording medium, the plurality of pieces recorded on the recording medium are recorded. An image display device that selects a plurality of images including substantially the same (similar) imaging targets from image data, performs predetermined image processing on the selected images as appropriate, and displays the plurality of images in an optimum form; In addition, it is possible to provide a photographing device that includes this image display device and can perform photographing in consideration of the display form.

The block block diagram which shows the main structures of the camera of one Embodiment of this invention, The block diagram which shows the schematic structure of the imaging part corresponding to 3D imaging | photography in the camera of FIG. The flowchart which shows the main sequence of the control processing in the camera of FIG. FIG. 4 is a flowchart showing details of a growth progress display process (step S13 in FIG. 3) in the process sequence of FIG. 3; FIG. 5 is a flowchart showing details of a reference portion search process (step S27 in FIG. 4) in the process sequence of FIG. 4 is a flowchart showing details of parallel display processing (step S15 in FIG. 3) in the processing sequence of FIG. FIG. 4 is a flowchart showing details of animation display processing (step S16 in FIG. 3) in the processing sequence of FIG. 3; The figure explaining the example of a condition in the case of image | photographing a time-dependent change of the same imaging | photography object using the camera of FIG. The figure explaining the example of the image acquired in the situation example of FIG. The figure explaining the example in the case of synthesize | combining the image acquired in the condition example of FIG. The figure which shows an example of the guide display displayed on a display part at the time of imaging | photography in the camera of FIG. The figure which shows the image display example of the image processing result (enlargement / reduction process) performed in the growth progress display process sequence of FIG. The figure which shows the example of an image display of the parallel display process result of FIG. The figure explaining the other example of a situation when image | photographing the time-dependent change of the same imaging | photography object using the camera of FIG. The figure which shows each common part superimposed using the image acquired in the other example of a situation of FIG. The figure which shows the example of a display of the image processing result about the image acquired in the other example of a situation of FIG. The figure which shows the example of a display at the time of displaying in parallel the image obtained in FIG.

  The present invention will be described below with reference to the illustrated embodiments.

  In one embodiment of the present invention, for example, an optical image formed by an optical lens can be photoelectrically converted using a solid-state imaging device, and an image signal obtained as a result can be displayed using a display device. An image capturing apparatus configured to generate image data, record the image data on a recording medium, and reproduce and display an image based on the image data recorded on the recording medium using a display device ( In the following description, the camera is simply referred to as a camera.

  First, the configuration of the camera of the present embodiment will be described below with reference to FIG. FIG. 1 shows only main components related to the present invention, and illustration of general components as a camera is omitted. In the following description, only the components directly related to the present invention (the components shown in FIG. 1) are described in detail, and the other components are substantially the same as those of a camera as a photographing apparatus that has been conventionally put into practical use. The detailed description thereof will be omitted.

  As shown in FIG. 1, the camera 1 includes a control unit 11, an imaging unit 12, a situation determination unit 13, a recording unit 14, an operation unit 15, a selection operation unit 16, a display unit 18, and the like. Configured.

  The control unit 11 is a control unit that comprehensively controls the entire camera 1. As the control unit 11, for example, an integrated circuit such as an LSI (Large-Scale Integration) or an ASIC (Application Specific Integrated Circuit) is applied.

  The control unit 11 serving as a control means includes components such as an image processing unit 11a, a similar image search unit 11b, and a display control unit 11c.

  The image processing unit 11a is a circuit unit that receives the image signal photoelectrically converted by the imaging unit 12 and performs normal image signal processing. Examples of processing contents performed by the image processing unit 11a include analog-digital (AD) / digital-analog (DA) conversion processing based on the image signal acquired by the imaging unit 12, contrast adjustment, and color. In addition to normal image processing such as image signal processing such as correction adjustment, image signal compression processing and expansion processing, image processing such as image enlargement processing, reduction processing, and trimming processing is performed.

  The similar image search unit 11b is a circuit unit that functions as a similar image search unit that searches for and extracts a similar image from image data recorded in the recording unit 14, for example, an image with a similar main subject.

  The display control unit 11c controls the display unit 18 to control image display, for example, display a normal display image on the display screen, or display a plurality of images searched by the similar image search unit 11b in the display screen. In addition to the control of displaying side by side, the circuit unit performs display control such as menu display and command display on the display screen in accordance with an instruction signal from the operation unit 15, the selection operation unit 16, and the like. For example, a liquid crystal display device or the like is applied as the display unit 18.

  The imaging unit 12 includes an imaging unit including an optical member such as an optical lens and a photoelectric conversion element, and performs an optical conversion process on the optical image formed by the optical lens using the photoelectric conversion element to generate an image signal. It is the component which performs.

In addition to the configuration of a normal camera, the imaging unit 12 can acquire a stereoscopic image, a so-called 3D image (hereinafter simply referred to as a 3D image), for example, by adopting a configuration as shown in FIG. Also good.
FIG. 2 is a configuration diagram illustrating a schematic configuration of an imaging unit for supporting 3D imaging in the camera 1 of the present embodiment.

  In order to perform 3D shooting, it is necessary to acquire a pair of images having parallax with respect to the shooting target, and record the pair of images in association with each other. Therefore, the configuration of the imaging unit 12 compatible with 3D imaging includes a pair of optical lenses 12La and 12Ra and a pair of imaging elements 12Lb and 12Rb, as shown in FIG. The pair of optical lenses 12La and 12Ra are arranged so that their optical axes are substantially parallel to each other with a predetermined distance (distance indicated by the symbol B in FIG. 2; usually a distance between human eyes of about 60 to 70 mm). Are arranged side by side. Image sensors 12Lb and 12Rb are arranged behind (on the image forming side) of the optical lenses 12La and 12Ra, respectively. The light receiving surfaces of the image pickup devices 12Lb and 12Rb are orthogonal to the optical axes of the pair of optical lenses 12La and 12Ra, and the optical axes of the pair of optical lenses 12La and 12Ra are imaged. It arrange | positions so that it may be located in the approximate center part of element 12Lb and 12Rb. Image signals generated by the image pickup devices 12Lb and 12Rb are output to the control unit 11, processed by the image processing unit 11a, and displayed on the display screen of the display unit 18 under the control of the display control unit 11c. It has become so. In this case, the display control unit 11c performs display control for 3D image display.

  Consider a case where the imaging unit 12 having such a configuration performs imaging in the situation shown in FIG. In the situation shown in FIG. 2, it is assumed that the subjects 100a and 100b are the shooting targets. It is assumed that the subject 100a is located at a distance L1 from the pair of optical lenses 12La and 12Ra, and the subject 100b is located at a distance L2 from the pair of optical lenses 12La and 12Ra.

  In this case, the left-eye image sensor 12 </ b> Rb is formed via the left-eye image lens 12 </ b> Lb through the left-eye image sensor 12 </ b> Lb and the right-eye image lens 12 </ b> Ra. A pair of image signals having parallax with the right-eye image formed on the light receiving surface can be obtained.

  The situation determination unit 13 includes various determination units for determining the situation where the camera 1 is placed, for example, a position determination unit 13a and a direction determination unit 13d using a GPS (Global Positioning System), and a year at the time of shooting. A clock unit 13b that acquires date and time information such as date, time, and the like, an underwater determination unit 13c, an inclination determination unit 13e that uses a gyro sensor, an acceleration sensor, and the like are configured. The underwater determination means by the underwater determination unit 13c uses a conventionally used technique such as a means for detecting an external pressure or a means for detecting a change in the refractive index of light.

  The clock unit 13b is not limited to the configuration example shown in FIG. 1, and may be configured to use an internal clock of a CPU or the like that constitutes a control unit provided in the control unit 11, for example. As another configuration example, a separate and independent clock circuit in the camera 1 may be provided.

  Note that the various types of shooting situation information determined by the situation determination unit 13 are recorded as accompanying information of the image data when the image data acquired by the shooting operation is recorded.

  The recording unit 14 is a part configured as a recording unit including a recording medium for recording image data and the like, and a circuit unit for driving the recording medium. As the recording medium, a general recording medium such as a semiconductor memory or a magnetic recording disk is applied. Note that the recording medium included in the recording unit 14 may be one that is fixedly arranged inside the camera 1 or one that is configured to be detachable from the camera 1 (so-called card-type recording medium or the like). Applied.

  Although not shown, the camera 1 has a buffer memory as a temporary storage area separately from the recording unit 14. The buffer memory can temporarily record various electric signals or image data handled in the camera 1, and is used as a work area when performing various signal processing, data processing, and the like. Composed. For example, the buffer memory may be provided as an internal memory in the control unit 11 in addition to a configuration provided as an independent component inside the camera 1.

  The operation unit 15 includes a plurality of operation members, operation switches, and the like provided to transmit a user's operation instructions to the control unit 11 and corresponding instruction signals when these operation members, operation switches, and the like are operated. It is a component part which comprises the user interface of the said camera 1 which consists of a circuit part etc. which generate | occur | produce. The operation members of the camera 1 include various operation members of a conventional camera, such as a shutter release button (switch), a power on / off button (switch), and an operation mode switching button (switch).

  The selection operation unit 16 is a configuration unit provided in order to issue a selection instruction by the user. For example, a touch panel provided on the display screen of the display unit 18 is applied as the selection operation unit 16. The selection operation unit 16 is controlled by the control unit 11 together with the display unit 18 and the like so that a predetermined selection operation and the like can be performed in conjunction with the display of the display unit 18 controlled by the control unit 11.

  The other components are assumed to have substantially the same configuration as that of an imaging apparatus (camera) that is an imaging apparatus that is generally put into practical use and includes an image display apparatus. Therefore, detailed description of other components will be omitted.

The effect | action at the time of recording an image using the camera 1 of this embodiment comprised in this way is demonstrated below using FIGS.
FIG. 3 is a flowchart schematically showing a main sequence of control processing of the camera 1 of the present embodiment. In this flowchart, main control related to the present invention is mainly shown, and illustration of control processing by a conventional ordinary camera is omitted.

  When the camera 1 is turned on so that the camera 1 can operate, in step S1 in FIG. 3, the control unit 11 checks whether or not the operation mode of the camera 1 is set to the shooting mode. If the shooting mode is set, the process proceeds to the next step S2. If the shooting mode is not set, that is, if an operation mode other than the shooting mode, for example, the playback mode is set, the process proceeds to step S11. The operation mode of the camera 1 includes, for example, a setting mode in addition to the shooting mode and the playback mode. In the present embodiment, the processing sequence of the operation modes other than the shooting mode and the playback mode is the present invention. The illustration is omitted because it is not directly related to. Therefore, if it is determined in step S1 that the shooting mode is not set, it is assumed that the playback mode is set, and the processing sequence after step S11 in FIG. Show.

  In step S <b> 2, the control unit 11 controls the imaging unit 12 and the display unit 18 to perform a process of displaying a live view image that sequentially displays an image based on the acquired image signal on the display screen of the display unit 18. This live view image display process is a normal process performed by a conventional camera. Thereafter, the process proceeds to step S3.

  In step S <b> 3, the control unit 11 starts executing the shooting situation determination process for determining the current situation, that is, the shooting situation, using each component of the situation determination unit 13. Examples of information relating to the shooting situation in which the camera 1 is placed include date and time information, position information (latitude and longitude, etc.), orientation information (such as the orientation in which the lens of the camera 1 faces), and posture information of the camera 1 (lens light). There is information such as the tilt of the axis, the posture of the camera 1, and the surrounding environment of the camera 1 (whether it is underwater or the like). Information regarding these shooting conditions is recorded in the recording unit 14 as the same data file or as another data file associated with the image data as information accompanying the image data obtained by the actual shooting operation. Information. This shooting state determination process is intermittently performed at predetermined time intervals, and when the camera 1 is switched to an operation mode other than the shooting mode, or when the camera 1 is turned off, these instructions are given. The control unit 11 that has received the signal ends the process.

  Subsequently, in step S4, the control unit 11 executes a similar image search process by the similar image search unit 11b. This similar image search process is already recorded in the recording unit 14 based on the shooting state information obtained in the process of step S3 described above, the image signal obtained by the imaging unit 12 and acquired by the image processing unit 11a, and the like. This is a process of searching whether there is a similar image taken at the same place and in a similar shooting target from the image data. For example, it is based on feature point extraction or pattern matching of a specific part in an image (for example, a part in the center area of the screen, a part having the same color part in the screen having a large area, a part having a high contrast, etc.), or a color determination. In addition to the search based on the image information, a part designated by the user using a touch panel or the like may be targeted. Furthermore, the composition of the image, aspect ratio information, and the like vary depending on the situation at the time of shooting. For this reason, even if the images are similar, there is a possibility that similar images cannot be detected correctly if the images are taken in landscape orientation and those in portrait orientation. Therefore, in the similar image search process, pattern matching and the like considering the screen configuration are performed by referring to the vertical information of the subject. If a similar image is found here, the process proceeds to the next step S5. If a similar image is not searched, the process proceeds to step S6.

  In a normal case, the user of the camera 1 usually performs shooting unconsciously, taking an object of interest as a shooting target, without being particularly conscious of a technique such as fixed-point shooting. As described above, when the user performs shooting according to his / her own interest, a large amount of captured image data is captured on the recording medium. A large amount of captured image data in such a recording medium may include a plurality of images that have been captured again after a certain amount of time for the same subject.

  For example, as shown in FIG. 8, when photographing the growth of a plant, the time until the plant grows from the “futaba” state shown in FIG. 8A to the “蕾” state shown in FIG. It can be considered that a relatively long time has passed from the “蕾” state in FIG. 8B to the “flowering” state shown in FIG. 8C. Therefore, a plurality of shootings related to other objects should be performed between the shooting of FIG. 8A and the shooting of FIG. 8B. In addition, it is considered that another shooting is performed in the same manner between the shooting in FIG. 8B and the shooting in FIG. 8C. If the user is shooting without being aware of the fixed point shooting method, the object (plant) 102 is small in the state shown in FIG. 8A, so the camera 1 is held in the horizontal position. Thus, shooting is performed with a composition in which the entire pot 101 including a small plant compared to the pot 101 fits in the screen. On the other hand, in the state shown in FIGS. 8B and 8C, the object (plant) 102 grows upward and grows upward. It is natural to shoot at the same time. And depending on the case, it can be considered that the composition is mainly composed of plants and the pot 101 is only partially included in the screen.

  Further, considering that the object (plant) 102 is growing between FIG. 8 (A) and FIG. 8 (B) or between FIG. 8 (B) and FIG. 8 (C), the object In order to fit the entire image of 102 in the screen, it is considered that the distance L between the camera 1 and the object 102 gradually increases. As a result, the actual size of the object 102 that has grown and changed over time from the time point of FIG. 8A through the time point of FIG. 8B to the time point of FIG. 8C has not changed. 101 is photographed so that the size of the image in the screen 101 gradually decreases.

  FIG. 9 is a diagram illustrating an example of an image as a photographing result when photographing is performed in the situation of FIG. That is, FIG. 9A shows an example of an image taken in the situation shown in FIG. FIG. 9B shows an example of an image taken in the situation shown in FIG. FIG. 9C shows an example of an image taken in the situation shown in FIG.

As can be seen from the image examples 18a, 18b, and 18c shown in FIG. 9, the image 18a in FIG. 9A is changed to the image 18c in FIG. 9C through the image 18b in FIG. 9B. Contrary to the growth of the object (plant) 102, the size of the pot 101 in the image is small.
Therefore, the images 18a, 18b, and 18c are enlarged or reduced so that the sizes of the bowls 101 in the images 18a, 18b, and 18c in FIGS. 9A, 9B, and 9C are substantially equal. When the reduction process is performed and the images 18a, 18b, and 18c are displayed in an overlapping manner, the result is as shown in FIG.

  Reference numerals (a), (b), and (c) shown in FIG. 10 denote processes for appropriately enlarging or reducing the images 18a, 18b, and 18c in FIGS. 9 (A), 9 (B), and 9 (C), respectively. It corresponds to each image after.

  In other words, when shooting with the fixed point shooting method in mind (that is, shooting with the same composition so that the size of the image that does not change in advance is the same size on the screen in consideration of growth etc.) In such a case, when the images thus captured are displayed in a superimposed manner, the growth change of the object 102 in the image can be clearly observed by the images as shown in FIG.

  However, the result is that a normal user simply collects inconsistent image data as shown in FIG. 9 for the reasons described above.

  As described above, for example, the process of changing the same subject is recorded without being conscious of the plurality of pieces of image data collected over a long period of time by the camera 1 used for normal shooting by a normal user. Searching for similar images, and even if multiple images extracted as a result of the search are displayed side by side, the subject is different from the shooting that was performed while taking into account that it was a fixed-point shooting method and a growth record at the time of shooting. It is not possible to clearly display the changes in the growth or progress of growth.

  Therefore, in the present invention, at the time of shooting, first, similar images are searched from the recorded image data. For example, when a plurality of similar images as shown in FIG. A guide display (reference display) for enabling similar combined shooting can be performed (see FIG. 11).

  That is, when a similar image is searched in the process of step S4 in FIG. 3 and the process proceeds to the next step S5, the control unit 11 controls the display unit 18 via the display control unit 11c in step S5. Then, a reference display process for displaying a reference display for guiding the photographing technique is performed by superimposing it on the display screen of the display unit 18, that is, the live view image being displayed. A display example of this reference display processing is shown in FIG.

  FIG. 11 shows a state in which the camera 1 is directed toward the object 102 and is a display example showing an enlarged screen displayed on the display unit 18 at that time.

In the screen display 18d shown in FIG. 11, a live view image is displayed on the entire screen, and a plurality of pieces of information for reference display are superimposed on a predetermined area in the screen. As information for reference display, for example, an area A for displaying search result information of recorded image data (in the display example of FIG. 11, character information such as “similar image present” is displayed)
When the search result is “present”, an area B for displaying information related to the most recent image among the search result images (in the display example of FIG. 11, information related to “azimuth” is displayed as illustration information imitating a compass magnet) )
When the search result is “present”, a region C (in the display example of FIG. 11, a reduced image (thumbnail image) of the most recent image is displayed to display other information related to the most recent image among the search result images. Yes)
Etc. Note that the display example shown in FIG. 11 is merely an example of the reference display. Various forms of reference display may be considered, which are different from this display example.

  Returning to FIG. 3, after the reference display as shown in FIG. 11 described above is made, in step S <b> 6, the control unit 11 monitors the instruction signal from the operation unit 15 or the like and confirms the presence or absence of a shooting operation instruction. . If the photographing operation instruction is confirmed here, the process proceeds to the next step S7. If the shooting operation instruction is not confirmed, the process returns to the above-described step S1, and the subsequent processes are repeated.

  In step S <b> 7, the control unit 11 controls the imaging unit 12 and the like to execute actual shooting operation processing. Here, the photographing operation process performed is a normal process performed by a conventional camera. Thereafter, the process proceeds to step S8.

  In step S <b> 8, the control unit 11 executes processing for recording the captured image data acquired in the above-described imaging operation processing in step S <b> 7 and the current situation data on a recording medium or the like of the recording unit 14. Here, the situation data is data of various types of information acquired by the situation determination unit 13, for example. These situation data may be recorded as metadata of the photographed image data, or may be recorded as a separate data file associated with the photographed image data. Thereafter, the processing returns to the above-described step S1 and the subsequent processing is repeated.

  Next, when it is determined that the operation mode of the camera 1 is set to the playback mode in the process of step S1 described above, and the process proceeds to the process of step S11, the process of the camera 1 is performed in the process after step S11. Performs playback mode operation. The processing sequence after step S11 corresponds to the operation in the image display apparatus specialized only for image reproduction, while showing the operation in the reproduction mode of the camera 1 as the photographing apparatus.

  When the camera 1 of the present embodiment operates in the playback mode, first, the control unit 11 controls the display unit 18 via the display control unit 11c to select a display form of an image on the display screen of the display unit 18. Display the menu screen. As the display form of the image, there are various kinds of display forms other than the normal display form (including single image display, thumbnail display, etc.) for displaying the image that has been recorded on the display screen of the display unit 18. In this embodiment, in particular, a similar image is searched from captured image data, and a plurality of images extracted based on the search result are displayed in a predetermined form (a plurality of images are superimposed or displayed as will be described later). Etc.) and “growth progress display”.

  When the display mode selection menu screen at the start of the playback mode is displayed on the camera 1, the user performs a desired display by touching a predetermined area (menu item display area) on the display screen. The form can be selected.

  When the camera 1 is operating in the playback mode, in step S11 of FIG. 3, the control unit 11 monitors an instruction signal from the selection operation unit 16 or the like to confirm whether or not there is an instruction to execute the growth progress display process. If the execution instruction for the growth progress display process is confirmed, the process proceeds to the next step S13. If the execution instruction for the growth progress display process is not confirmed, the process proceeds to step S12.

  In step S12, the control unit 11 executes normal display processing. This normal display process is a normal process performed by a conventional camera (detailed description of the process is omitted). Thereafter, the process proceeds to step S18.

  In step S18, the control unit 11 monitors an instruction signal from the operation unit 15 or the like and confirms whether or not the operation mode has been switched by an operation mode switching button or the like, that is, whether or not an instruction to end the reproduction mode has been issued. Do. Here, when the reproduction mode end instruction is confirmed, the processing returns to the above-described step S1, and the subsequent processing is repeated. If the playback mode end instruction is not confirmed, the process returns to the above-described step S11 and the subsequent processes are repeated.

  On the other hand, when the execution instruction of the growth progress display process is confirmed in the process of step S11 described above and the process proceeds to the process of step S13, the control unit 11 executes the growth progress display process in step S13. The details of the growth progress display process are as shown in the flowchart of FIG.

  The control unit 11 of the camera 1 enters the processing sequence of the growth progress display process when the user gives a predetermined operation instruction as desired by the user. At this time, on the display screen of the display unit 18 of the camera 1, for example, an image corresponding to the latest image data among the recorded image data is reproduced and displayed as in the normal reproduction process. On the display screen, for example, a mark indicating that the growth progress display process is being executed, an instruction display for prompting selection of the reference image, and the like are displayed superimposed on the latest image. In this state, the user first performs an operation for setting an image serving as a reference for displaying the progress of growth (hereinafter referred to as a reference image). This selection operation is, for example, the following operation.

  First, a normal operation for reproducing and displaying an image is performed, and a desired image to be a reference image is displayed on the display screen of the display unit 18. In this state, the reference image selection instruction display is touched. At this time, the control unit 11 monitors instruction signals from the operation unit 15, the selection operation unit 16, and the like.

  When the user performs the reference image selection instruction operation as described above, in step S21 in FIG. 4, the control unit 11 receives the reference image selection instruction signal and sets the currently displayed image as the reference image. A reference image selection process is executed. Thereafter, the process proceeds to step S22.

  In step S22, the control unit 11 uses the similar image search unit 11b to record the same situation information (shooting position information, azimuth information, etc.) as the reference image selected in the processing of step S21 from the recorded image data. A similar image search process for searching for similar images having () is executed. Thereafter, the process proceeds to step S23.

  In step S23, the control unit 11 confirms whether or not a similar image exists in the process of step S22 described above. That is, as a result of the search process in step S22 described above, if a similar image is not searched, the sequence of the growth progress display process is skipped and the process proceeds to step S18 in FIG. If a similar image is found, the process proceeds to step S24. Note that the result of the similar image search process is displayed on the display screen of the display unit 18 in some form. As the display form, for example, various notice forms such as a notice display using characters or the like, or displaying or blinking pictograms are conceivable. And the camera 1 will be in the state which waits for the instruction | indication by a user in the state which displayed the search result. The user performs the following operation while seeing the notice display of the search result.

  For example, if there is no similar image as a result of the similar image search process, a notification to that effect and a display for prompting whether to end the playback mode are displayed. The user operates according to the screen display.

  On the other hand, if a similar image exists as a result of the similar image search process, a notification to that effect and a display to prompt a reference portion selection instruction operation are displayed. The user operates according to the screen display. This reference part selection instruction operation is an area that exists in common with the reference image and its similar image, and an operation in which the user arbitrarily selects a reference image area (hereinafter referred to as a reference part). It is. As a specific operation, for example, an area corresponding to the reference portion is selected and designated by a touch operation on the reference image displayed on the display screen of the display unit 18 or a similar image thereof. At this time, the control unit 11 monitors instruction signals from the operation unit 15, functioning as a region specifying unit, the selection operation unit 16, and the like.

  When the user performs the reference part selection instruction operation as described above, in step S24 of FIG. 4, the control unit 11 receives the selection operation instruction signal and performs a reference part selection process. Thereafter, the process proceeds to step S25.

  In step S <b> 25, the control unit 11 executes a process of storing and recording an image currently displayed on the display screen of the display unit 18 in association with a specific area in the recording medium of the recording unit 14, such as a specific folder. In the present embodiment, the specific area is referred to as a “growth image collection” area as shown in FIG. Thereafter, the process proceeds to step S26.

  In step S26, the control unit 11 causes the display control unit 11c to display the latest image on the display screen of the display unit 18 among the images based on the similar image search results described above. Thereafter, the process proceeds to step S27.

  In step S27, the control unit 11 performs a reference part search process. The details of the reference portion search process are as shown in FIG.

  First, in step S31 in FIG. 5, the control unit 11 checks whether or not the reference unit selected and specified in step S24 in FIG. 4 is present in the next similar image by the image processing unit 11a or the like. Here, when it is confirmed that the reference part also exists in the next similar image, the process proceeds to the next step S33. If it is determined that the reference portion does not exist in the next similar image, the process proceeds to step S32.

  In step S32, the control unit 11 confirms whether or not there is a similar point in the next similar image by the image processing unit 11a or the like. If it is determined that there is no similarity, the process returns to the process sequence of FIG. 4 (return) and proceeds to the process of step S28 of FIG. If it is confirmed that there is a similar point, the process proceeds to step S33.

  In step S <b> 33, the control unit 11 performs image synthesis processing for synthesizing the reference image and the next similar image using the image processing unit 11 a and the like. Thereafter, the process proceeds to step S34.

  In step S <b> 34, the control unit 11 additionally stores the composite image generated in the process of step S <b> 33 described above in the “growth image collection”. Thereafter, the processing sequence of FIG. 4 is returned (return), and the process proceeds to step S28 of FIG.

  Returning to FIG. 4, in step S <b> 28, the control unit 11 confirms whether or not there is a next similar image that has not yet been subjected to the reference portion search among the images based on the similar image search result in step S <b> 22 described above. If it is confirmed that there is a similar image, the process returns to the above-described step S26 and the subsequent processes are repeated. If there is no similar image, the process proceeds to the next step S29.

  In step S29, the control unit 11 performs the enlargement process or the reduction process for other similar images in accordance with the size of the reference part of the latest image among the images based on the similar image search result in step S22 described above. Run by. The image processing result of the enlargement process or reduction process is as shown in FIG. 12, for example.

  Here, in the case where the image 18c shown in FIG. 12C is the latest image and is the reference image, and the bowl 101 is the reference portion in the image, the similarity shown in FIGS. 12A and 12B is obtained. Assume that an image exists. Then, it is assumed that time has passed in the order of FIG. 12A, FIG. 12B, and FIG. 12C (that is, plants as subjects are growing in that order).

  In the actual image in FIG. 12A, the entire image region is within the frame indicated by reference numeral 18a. That is, in this case, the size of the bowl 101a is so large as to occupy an almost full area of the screen. When the image 18a is reduced and processed so that the size of the bowl 101a is substantially the same as that of the bowl 101c in the case of FIG. 12C (the dimension indicated by the symbol H in FIG. 12 is matched) The image is as shown in FIG. At this time, a frame indicated by reference numeral 18Ac in FIG. 12A is a frame having the same size as the image 18c in FIG. 12C and is the size of the display screen. The same applies to the case of FIG.

  In this way, by comparing the image sizes of the similar images 18a and 18b with the reference image 18c and performing the reduction process so that the size of the reference portion is substantially the same, it becomes easy to understand the comparison of the progress of growth.

  The image processing result in the process of step S29 described above is additionally stored in the “growth image collection”. Thereafter, the process returns to the process sequence of FIG. 3 (return) and proceeds to the process of step S14 of FIG.

  Returning to FIG. 3, in step S <b> 14, the control unit 11 monitors the instruction signal from the operation unit 15, the selection operation unit 16, and the like, and confirms whether there is an instruction signal for ending the growth progress display process. If the growth progress display process end instruction signal is confirmed, the process proceeds to step S18. If the growth progress display process end instruction signal is not confirmed, the process proceeds to the next step S15.

  In step S15, the control unit 11 executes parallel display processing (subroutine in FIG. 6) by the display control unit 11c. Details of the parallel display processing are as shown in FIG.

  In step S41 of FIG. 6, the control unit 11 starts execution of the overlapping portion position determination process for the composite image displayed on the display unit 18 by the processes of FIGS.

  First, in step S42, the control unit 11 uses the image processing unit 11a or the like to check whether or not the overlapping portion position is closer to the lower part in the composite image. Here, when it is confirmed that the overlapping part position is closer to the lower part in the composite image, the process proceeds to step S43.

  In step S43, the control unit 11 controls the display unit 18 with the image processing unit 11a and the display control unit 11c to obtain the plurality of similar images, and the “growth image collection” in FIGS. 4 and 5 described above. A plurality of similar images stored and recorded on the display unit 18 are displayed side by side in the horizontal direction from the left to the right of the screen, for example, in time series on the screen of the display unit 18. Thereafter, the process proceeds to step S44.

  In step S <b> 44, the control unit 11 confirms whether each image in the “growth image collection” is a 3D image. For this confirmation, for example, metadata attached to image data may be referred to. If it is confirmed that the image is a 3D image, the process proceeds to step S45. If it is determined that the image is not a 3D image, the process proceeds to step S46.

  When it is determined that the image is a 3D image and the process proceeds to step S45, in step S45, the control unit 11 performs image processing for adjusting the parallax of the overlapping portions of the images by the image processing unit 11a and the like. Thereafter, the process proceeds to step S46.

  In step S46, the control unit 11 performs parallel display so that the overlapping portion is displayed at substantially the same position in the lower area of the screen, and the positions are aligned and arranged side by side in time series on the same screen. In this case, for example, based on the shooting date / time information, the images are displayed side by side from the left to the right in the order of the shooting date / time. Thereafter, the process returns to the process sequence of FIG. 3 (return) and proceeds to the process of step S16 of FIG.

  A display example of the display unit 18 as a result of the parallel display processing at this time is shown in FIG. In FIG. 13A, on the display screen of the display unit 18, a plurality of similar images of the above-described images 18a, 18b, and 18c are displayed side by side in order of time series from the left to the right of the screen. In the vicinity of each image, each corresponding shooting date / time information is displayed.

  On the other hand, if it is determined in step S42 described above that the overlapping portion position is not close to the lower part in the composite image, the process proceeds to step S51.

  In step S51, the control unit 11 confirms whether or not the overlapping portion position is closer to the right side or the left side in the composite image by the image processing unit 11a or the like. Here, when it is confirmed that the overlapping portion position is near the right part or the left part in the composite image, the process proceeds to step S52. On the other hand, if it is determined that the overlapping position is not close to the right or left in the composite image, the process proceeds to step S56.

  In step S56, the control unit 11 executes a warning process for displaying a warning display on the display unit 18 to the effect that the parallel display process cannot be performed by the display control unit 11c. Thereafter, the process returns to the process sequence of FIG. 3 (return) and proceeds to the process of step S16 of FIG.

  In step S52, the control unit 11 controls the display unit 18 with the image processing unit 11a and the display control unit 11c to obtain the plurality of similar images, which are “growth images” in FIGS. 4 and 5 described above. A plurality of similar images stored and recorded in the “collection” are displayed on the screen of the display unit 18 side by side, for example, in the time-series order in the vertical direction from the top to the bottom. Thereafter, the process proceeds to step S53.

  In step S <b> 53, the control unit 11 checks whether each image of the “growth image collection” is a 3D image. This confirmation is the same as the processing in step S44 described above. If it is confirmed that the image is a 3D image, the process proceeds to step S54. If it is determined that the image is not a 3D image, the process proceeds to step S55.

  When it is determined that the image is a 3D image and the process proceeds to step S54, in step S54, the control unit 11 performs image processing for adjusting the parallax of the overlapping portions of the images by the image processing unit 11a and the like (steps above). Same as the process of S45). Thereafter, the process proceeds to step S55.

  In step S55, the control unit 11 displays the overlapping portions so as to be displayed at substantially the same position in the right region or the left region of the screen, in the same position, and arranged in chronological order on the same screen. I do. In this case, for example, based on the shooting date / time information, the images are displayed in the vertical order from the top to the bottom of the shooting date / time. Thereafter, the process returns to the process sequence of FIG. 3 (return) and proceeds to the process of step S16 of FIG.

  A display example of the display unit 18 as a result of the parallel display processing at this time is as shown in FIG. The display example in FIG. 13B is substantially the same as the display example in FIG. That is, in FIG. 13B, on the display screen of the display unit 18, a plurality of similar images of the above-described images 18a, 18b, and 18c are displayed side by side in order of time series from the top to the bottom. In the vicinity of each image, each corresponding shooting date / time information is displayed.

  In the situation illustrated in FIG. 13, the growth of the plant extends toward the observer side with respect to the screen. In such a case, for example, if a 3D image is taken, an effective display can be obtained.

  Returning to FIG. 3, in step S <b> 16, the control unit 11 monitors an instruction signal from the operation unit 15, the selection operation unit 16, and the like, and confirms whether there is an instruction signal for starting execution of the animation display process. If the animation display process start instruction signal is confirmed, the process proceeds to the next step S17. When the animation display process start instruction signal is not confirmed, the process returns to the above-described step S14 and the subsequent processes are repeated.

  In step S17, the control unit 11 executes animation display processing (subroutine in FIG. 7) by the display control unit 11c. The details of this animation display process are as shown in FIG.

  In step S61 in FIG. 7, the control unit 11 includes a plurality of synthesized images (a plurality of images recorded in the “growth image collection” area of the recording medium) generated by the processes in FIGS. 4 and 5 described above. The oldest image (the image in FIG. 12A) is displayed on the display unit 18. Thereafter, the process proceeds to step S62.

Subsequently, in step S62, the control unit 11 confirms whether or not a predetermined time has elapsed with reference to the clock unit 13b. This predetermined time is the display time of the image on the display unit 18. As the predetermined time, for example, when there are a large number of images to be displayed, if images are displayed one after another at a short interval (for example, a time interval of one second or less, such as a fraction of a second), Display is possible. In addition, when the number of images is small, if the display interval is long (for example, at intervals of 1 to 2 seconds), the display becomes a so-called slide show.
If the passage of the predetermined time is confirmed in the process of step S62, the process proceeds to the next step S63.

  In step S63, the control unit 11 confirms whether there is an image to be displayed next. If it is confirmed that there is a next image, the process proceeds to the next step S64. If it is determined that there is no next image, the process proceeds to step S65.

  In step S64, the control unit 11 causes the display control unit 11c or the like to match the size and position of the common part (particularly the reference part) between the already displayed image and the next image, and displays the next image. Thereafter, the process returns to step S62, and thereafter the same process is repeated.

  On the other hand, when it is determined in step S63 described above that there is no image to be displayed next and the process proceeds to step S65, in step S65, the control unit 11 executes an end process of the animation display process. Thereafter, the processing sequence returns to FIG. 3 (return). That is, returning to the process of step S14 in FIG. 3, the same process is repeated thereafter.

  By the way, when the process of changing the same object to be photographed is taken, for example, without being aware of the fixed-point method, the part that attracts the photographer changes depending on the state of the object at the time of photographing. There is.

  For example, in the case of recording the progress of plant growth, as shown in FIG. 14A, an image taken at the time of germination after planting seeds, a small object (plant that has just germinated) and a pot It is assumed that the image 18a is a whole image.

  Thereafter, it is assumed that the plant has grown and the plant itself has grown greatly with respect to the previous germinated plant. In such a case, if the size of the plant is relatively large with respect to the size of the pot as shown in FIG. 14B, the object of interest as the object to be photographed is the plant. In the photographed image, the entire bowl may not be imprinted on the screen, and there may be an image 18b in which only a part of the bowl is imprinted.

  Furthermore, when a plant grows and the flowering time is reached, the object of interest for photography is the flower portion of the plant. In this case, as shown in FIG. 14C, the photographed image has a flower portion occupying the main portion, and as a result, an image 18c in which a pot does not fit in the screen may be acquired.

  In such a case, even in a series of a plurality of images that should be associated with each other, the image area initially set as the reference portion does not always exist in common for all images.

  Therefore, in the similar image search process, a specific image at a certain time point is compared with the immediately preceding image, and the two images are similar images depending on the presence or absence of a common part between the two images. It may be determined whether or not.

For example, between the two images in FIGS. 14A and 14B, the area indicated by reference numeral (a), that is, the image area in which a part of the bowl is reflected is treated as a common part.
In addition, between the two images in FIGS. 14B and 14C, the area indicated by reference numeral (b), that is, the image area in which a part of the plant (leaves) is reflected is handled as a common part.
In this way, a process of sequentially searching for a common part between two adjacent images in time series is performed. If a plurality of images as a result of the search are displayed so that the common portions overlap, an overall image as shown in FIG. 15 can be obtained.

Then, image processing such as reduction processing or enlargement processing is performed in order to perform parallel display processing or animation display processing on the plurality of images. At this time, the image 18a corresponding to FIG. 14A and the image 18b corresponding to FIG. 14B are subjected to image processing so that the size of a portion of the bowl having no change in size is aligned. . Thereby, images as shown in FIGS. 16A and 16B are obtained.
In addition, in the image 18b corresponding to FIG. 14B and the image 18c corresponding to FIG. 14C, image processing of each image is performed so that the sizes of a part of the plant of the common part are almost uniform. In this case, strictly speaking, a part of the plant (leaf) is completely matched in size and position with the growth of the plant, as the leaf part of the image 18b and the leaf part of the image 18c change with time. Not exclusively. The similar determination in this case may be based on the shape of a part (leaf) of the plant. Thereby, images as shown in FIGS. 16B and 16C are obtained.
FIG. 17 shows an example in which the images thus obtained are displayed in parallel. In this case, the old images are displayed in order from left to right in the same way as the display example shown in FIG. 13 described above, but with respect to the previous images as they become new images in time series. A composite image in which common parts of new images are superimposed is displayed in order. With such a display form, it is possible to display a common part to all of a plurality of images displayed in parallel.

As described above, according to the above-described embodiment, when a user selects a specific object from a plurality of image data captured on a recording medium, the selected specific object, shooting state information, and the like are selected. Search similar images, and as a result of the search, perform predetermined image processing (enlargement processing, reduction processing, composition processing, etc.) on the extracted multiple image data as appropriate, and the image processing results and various accompanying information Can be stored as a series of images and displayed on the display unit 18 in a predetermined form (parallel display, animation display, composite display, etc.).
This makes it possible to display a series of similar images taken over a long period of time without the user being aware of the fixed point shooting method in the form of images shot by the fixed point shooting method. , You can expand the enjoyment of image display.

Further, in the camera 1 of the present embodiment, at the time of shooting, image data with a similar shooting situation or the like is searched from image data recorded on a recording medium, and if a similar image is searched, the recorded image is recorded. It is possible to display a guide display (reference display) for enabling photographing according to a similar image.
As a result, the user can take a picture in a situation (composition, direction, etc.) similar to the case where the same object has been taken before. Therefore, even if the user has forgotten that the same object has been previously photographed, the guide display is displayed at the time of photographing, so that photographing according to the previous photographing situation can be easily performed. That is, even if the user is not conscious of fixed-point shooting, if the same object that has been shot before is shot, shooting by a fixed-point shooting method can be performed by the guide display.

  Further, when recording the image data thus photographed, it is automatically stored as a series of image collections. Therefore, when the series of image data is reproduced and displayed, it is displayed in a predetermined form by the above-described method. Can be easily performed and image display can be enjoyed.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications can of course be implemented without departing from the spirit of the invention. Further, the above embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if several constituent requirements are deleted from all the constituent requirements shown in the above-described embodiment, if the problem to be solved by the invention can be solved and the effect of the invention can be obtained, this constituent requirement is deleted. The configured structure can be extracted as an invention.

  The present invention is not limited to a photographing apparatus that has a photographing function such as a digital camera and is also an electronic apparatus having a display device, and is an apparatus having an image display function for displaying image data, as a single image. In addition to the display device, other types of electronic devices equipped with a display device and a photographing function, such as a mobile phone, a recording device, an electronic notebook, a personal computer, a game device, a television, a clock, and a GPS (Global Positioning System) are used. The present invention can be applied to various image display devices such as navigation devices and electronic devices with a photographing function.

DESCRIPTION OF SYMBOLS 1 ... Camera, 11 ... Control part, 11a ... Image processing part, 11b ... Similar image search part, 11c ... Display control part, 12 ... Imaging part, 13 ... Situation determination part, 14 ... Recording part, 16 ... Selection operation part , 18 ... display section

Claims (8)

In an image display device comprising a display means for displaying an image,
An area designating unit for designating a part of an image area of a reference image selected from a plurality of images recorded in the recording unit as a reference part;
A similar image search means for searching and extracting an image including an image area substantially the same as the reference portion specified by the area specifying means from a plurality of images recorded in the recording means;
Image processing is performed based on the reference portion that is an image area common to the image searched and extracted by the similar image search means and the reference image, and the resulting image is displayed on the display screen of the display means. Control means;
An image display device comprising: The area specifying means is a touch panel,
The image display apparatus according to claim 1, wherein the similar image search unit performs an image search process based on an image of the reference portion designated by the touch panel.   The said similar image search means searches for a similar image based on the image of the said reference | standard part designated by the said area | region designation | designated means, and the accompanying information contained in the said reference image, The search of the similar image of Claim 1 characterized by the above-mentioned. Image display device. The control means, for the plurality of images searched and extracted by the similar image search means and the reference image, so that the position and size in the display screen image of the display means of the reference portion substantially match. An image processing unit for performing an enlargement process or a reduction process;
The image display apparatus according to claim 1, further comprising: a display control unit that displays a plurality of processed images obtained by the image processing unit on a display screen of the display unit.   The display control unit performs parallel display processing in which a plurality of processed images obtained by the image processing unit are arranged and displayed in time series in one screen of the display screen of the display unit. The image display device described in 1.   The display control unit performs an animation display process of sequentially displaying a plurality of processed images obtained by the image processing unit on a display screen of the display unit in a time-series order at a constant time interval. 2. The image display device according to 1.   The image processing unit performs an image composition process for superimposing the plurality of images obtained by performing processing so that the position and size of the reference part substantially match so that the reference part of each image matches. The image display device according to claim 1, wherein An imaging apparatus comprising: an imaging unit that photoelectrically converts a subject image to acquire an image; a recording unit that records an image acquired by the imaging unit; and an image display device that includes a display unit that displays an image. In
An area designating unit for designating a partial image area of a reference image selected from a plurality of images recorded in the recording unit as a reference part;
A similar image search means for searching and extracting an image including an image area substantially the same as the reference portion specified by the area specifying means from a plurality of images recorded in the recording means;
Image processing is performed based on the reference portion that is an image area common to the image searched and extracted by the similar image search means and the reference image, and the resulting image is displayed on the display screen of the display means. Control means;
An imaging apparatus comprising:

Images