JP5585168B2 - DIFFERENTIAL IMAGE GENERATION DEVICE, PROGRAM, AND DIFFERENTIAL IMAGE GENERATION METHOD - Google Patents

Description

  The present invention relates to a difference image generation apparatus, a program, and a difference image generation method for generating a difference image between a photographed first image and a second image.

  Conventionally, a technique has been devised for cutting out a difference image between a first image taken together and a second image in which a subject to be cut out exists in the first image (see Patent Document 1). If this technique is used, a first image of only the background is captured, and then a second image in which the main subject to be extracted exists in the background is generated, thereby generating an image in which only the main subject is cut out. Can do.

Japanese Patent Laid-Open No. 10-21408

  However, the background at the time of shooting is not necessarily fixed, and there may be a moving subject such as a tree that swings in the wind or a clock with a minute hand in the background. In this case, when the second image in which the main subject is present in the background is taken after taking the first image of only the background, not only the main subject but also a tree swung by the wind or rotated. A moving background subject such as a minute hand exists as a difference image between the two images.

  Therefore, when the difference image is cut out from the first image and the second image, the moving background subject is cut out together with the main subject, and there is a problem that it is not possible to cut out only the target main subject.

  The present invention has been made in view of such conventional problems, and an object of the present invention is to provide a differential image generation device, a program, and a differential image generation method that can reliably cut out only a subject desired by a user in an image. To do.

In order to solve the above problem, the invention according to claim 1 acquires the first image and the second image in which the subject to be cut out exists in the first image as a set of set images. Acquisition means, display means for displaying a second image among the first image and the second image acquired as a combined image by the acquisition means, and a second image displayed on the display means An area designating unit that designates an area where the subject is present as a target area to be cut out, and an area corresponding to the target area designated by the area designating unit is set in the first image, and the target area and the set area are set. Cutting means for cutting out a difference image with respect to the area, and the area specifying means further displays the second image with an angle of view smaller than the angle of view at the time of shooting, This display control means Designating means for designating, as the area, an angle of view smaller than the angle of view of the captured second image as the area, and the clipping means includes the difference within the area designated by the designating means. The image is cut out .

The invention according to claim 2 further comprises instruction means for instructing the enlarged display in response to an operation in the invention according to claim 1 , wherein the display control means is responsive to an instruction from the instruction means. The second image displayed on the display means is enlarged and displayed .

According to a third aspect of the present invention, a computer included in a device acquires a first image and a second image in which a subject to be cut out exists in the first image as a set of combined images. An acquisition means, a display means for displaying a second image among the first image and the second image acquired as a combined image by the acquisition means, and the second image displayed on the display means An area designating unit for designating an area where a subject exists as a target area to be cut out, an area corresponding to the target area designated by the area designating unit is set in the first image, and the target area, the set area, A display control unit for displaying the second image with an angle of view smaller than the angle of view at the time of shooting, and the display control unit. By means The second image displayed in an enlarged manner is caused to function as a designating unit that designates an angle of view smaller than the angle of view at the time of shooting as the region. It is characterized by functioning to cut out the difference image .

According to a fourth aspect of the present invention, an acquisition step of acquiring a first image and a second image in which a subject to be cut out exists in the first image as a set of combined images, and the acquisition Of the first image and the second image acquired as the combined image in the step, a display step for displaying the second image on the display unit, and a second image displayed in the display step An area designating step for designating an area in which the subject exists as a target area to be cut out, and an area corresponding to the target area designated in the area designating step are set in the first image, and the target area and the set area are set. A step of cutting out a difference image with respect to the region, and the region specifying step further includes a display control step for displaying the second image with an angle of view smaller than the angle of view at the time of shooting, this A designating step of designating, as the region, an angle of view smaller than the angle of view of the second image enlarged and displayed in the display control step, and the clipping step is designated in the designating step The difference image is cut out in the region thus formed.

  According to the present invention, it is possible to reliably cut out only a subject desired by a user in an image.

1 is a block diagram of an electronic camera to which an embodiment of the present invention is applied. It is a flowchart which shows the process sequence of 1st Embodiment. It is explanatory drawing which shows the process sequence in this Embodiment. It is a flowchart which shows the process sequence of 2nd Embodiment. It is explanatory drawing which shows the process sequence in this Embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a circuit block diagram showing an electrical configuration of a digital camera 1 common to each embodiment of the present invention. The digital camera 1 has general functions such as AE (Auto Exposure), AWB (Auto White Balance), and AF (Auto Focus). That is, the lens block 11 includes an optical system such as a zoom lens 111 and a focus lens (not shown), and a drive mechanism for driving the optical system. The zoom lens 111 is driven in the optical axis direction by a zoom motor 12 provided in a drive mechanism.

  A motor driver 16 is connected to a central processing unit (CPU) 13 that controls the entire digital camera 1 via a bus 14 and a timing generator (TG) 15. The motor driver 16 drives the zoom motor 12 based on a timing signal generated by the timing generator 15 in accordance with an instruction from the CPU 13 according to a user operation. As a result, the zoom lens 111 is driven to change its lens position.

  At this time, the CPU 13 stores and manages instructions generated by itself. Therefore, the CPU 13 can acquire and manage the lens position of the zoom lens 111 that changes according to the timing signal generated by the timing generator 15 according to the command generated by the CPU 13 by storing and managing the command generated by the CPU 13. it can.

  Although not shown in the drawings, actually, a focus motor and a motor driver for driving the focus lens, a mechanical shutter, a mechanical aperture, and a drive mechanism having a motor for driving them are provided.

  The digital camera 1 also has a CCD (Charge Coupled Device) 18 as an image sensor. The CCD 18 is disposed on the optical axis of the lens block 11. The subject is imaged on the light receiving surface of the CCD 18 by the lens block 11. The CCD 18 is driven by a vertical and horizontal driver 19 based on a timing signal generated by the timing generator 15 in accordance with a command from the CPU 13, and outputs an analog imaging signal corresponding to the optical image of the subject to the unit circuit 20. The unit circuit 20 is a CDS (Correllated Double Sampling) circuit that removes noise included in the output signal of the CCD 18 by correlated double sampling, or an A / D (Analog / Digital) that converts an imaging signal from which noise has been removed to a digital signal. ) Consists of a converter and the like, and outputs a digital signal to a DSP (Digital Signal Processor) 21.

  The DSP 21 performs image processing such as pedestal clamp and auto white balance on the digital signal output from the unit circuit 20 to generate image data including a luminance (Y) signal and a color difference (UV) signal. The image data converted by the DSP 21 is sequentially stored in an SDRAM (Synchronous Dynamic Random Access Memory) 22 and a video signal every time one frame of data (image data) is accumulated in the REC (Record) live view mode. And is sent to the liquid crystal monitor 4 and displayed on the screen as a live view image.

  In the still image shooting mode, using the shutter button operation as a trigger, the CPU 13 switches the live view image shooting mode to the still image shooting mode for the CCD 18, the vertical and horizontal drivers 19, the unit circuit 20, and the DSP 21. The image data obtained by the shooting processing in the still image shooting mode and temporarily stored in the SDRAM 22 is compressed by the CPU 13 and finally recorded in the external memory 25 as a still image file of a predetermined format. The still image file recorded in the external memory 25 is read out and expanded by the CPU 13 in accordance with the user's selection operation in the PLAY mode, and expanded as image data on the SDRAM 22, and then an image based on the image data is displayed on the monitor 4. Is displayed.

  The flash memory 26 stores various programs for causing the CPU 13 to control each unit, for example, a program for controlling AE, AF, and AWB, and a program for executing processing shown in the flowcharts described later. Has been.

  The digital camera 1 also has a key input unit 27 including a plurality of operation keys and switches such as a shutter button, a zoom button, and an OK button. When any operation key is operated by the user, the key input unit 27 supplies an operation signal corresponding to the operation content to the CPU 13.

  A touch panel 23 is stacked on the monitor 4. The touch panel 23 detects the touch position of the user on the monitor 4 and supplies a signal indicating the detected touch position to the CPU 13.

(First embodiment)
Next, the operation of the digital camera 1 according to the above configuration in the first embodiment will be described. FIG. 2 is a flowchart showing a processing procedure in the difference image generation mode in the present embodiment. When the difference image generation mode is set by the user's operation on the key input unit 27, the CPU 13 executes the processing shown in this flowchart according to the program.

  That is, the CPU 13 determines whether or not the shooting mode in the difference image generation mode is set (step S101). If the shooting mode is set, it is determined whether or not the first shutter button operation has been detected (step S102). If the first shutter button operation is detected, the photographing process is executed and the image is recorded as the first image (step S103). Specifically, the CPU 13 compresses the image data stored in the S10 DRAM 22 when the shutter button is pressed, and records the compressed image data in the external memory 25 as a first image.

  Next, it is determined whether or not a second shutter button operation has been detected (step S104). If the second shutter button operation is detected, the photographing process is executed and the image is recorded as the second image in the external memory 25 in association with the first image recorded in step S103 (step S105). ).

  Subsequently, it is determined whether or not the cut-out mode in the difference image generation mode has been shifted, and the processing in steps S101 to S106 is repeated until the shift is made. Therefore, during this period, the user captures the background by the first shutter button operation, and captures the image in which the main subject to be cut out in the background exists by the second shutter button operation. Multiple sets of images are recorded in the external memory 25 with the image and the second image as a set.

  When the user sets the extraction mode in the difference image generation mode to synthesize the difference image, the process proceeds from step S106 to step S107, and any one of the plurality of sets of images recorded in the external memory 25 is one set. It is determined whether an image is selected. If a set of images composed of the associated first image and second image is selected from the external memory 25, the second image in the selected set of images is displayed on the monitor 4 ( Step S108). Therefore, if a set of images consisting of the first image P1 shown in FIG. 3A and the second image P2 shown in FIG. 3B is selected, as shown in FIG. The second image P2 is displayed on the monitor 4.

  As shown in FIG. 3A, in this example, the first image P1 is an image composed of a background in which trees exist as background subjects A and B. As shown in FIG. 3B, the second image is an image in which a main subject C to be cut out is present in a background where similar trees exist as background subjects A and B.

  Next, the CPU 13 determines whether or not an operation of a zoom button provided in the key input unit 27 is detected (step S109). If the operation of the zoom button by the user is detected, the second image P2 is enlarged in the SDRAM 22 and the image displayed on the monitor 4 is enlarged (step S110). Subsequently, it is determined whether or not an OK button operation has been detected (step S111), and the processing from step S109 is repeated until an OK button operation is detected.

  Accordingly, whenever the user operates the zoom button during this time, the second image P2 is enlarged in the SDRAM 22, and the image displayed on the monitor 4 is enlarged as shown in FIG. Step S110). As a result, the background subjects A and B are outside the screen of the monitor 4 as shown in FIG. As a result, in the screen of the monitor 4, only the enlarged partial image P3 having a small angle of view including the main subject C and including the background subjects A and B is displayed.

  In this way, if the image P3 in which the main subject C exists except for the background subjects A and B is displayed on the monitor 4, the user operates the OK button. Then, the CPU 13 advances the processing from step S111 to step S112, and calculates the magnification of the second image, that is, the magnification of the enlarged partial image P3 being displayed.

  Here, the enlargement magnification is the magnification of the enlarged partial image P3 with the angle of view of the screen of the monitor 4 calculated using the second image P2 having the angle of view at the time of shooting as a reference (1 time).

  When the enlargement magnification is calculated, the first image P1 is enlarged in the SDRAM 22 with the calculated enlargement magnification (step S113). Therefore, if the first image P1 is also displayed on the monitor 4, the background subjects A and B are enlarged to a size outside the screen of the monitor 4. As a result, the SDRAM 22 stores the first image P1 and the second image P2 that have been enlarged at the same magnification.

  Subsequently, the CPU 13 designates a region within the angle of view displayed on the monitor 4 of the enlarged second image P2 and the corresponding angle of view of the enlarged first image (step S114). ). Then, a difference image between the first image P1 and the second image P2 that has been enlarged is cut out within the designated region (step S115).

  At this time, trees are present as background subjects A and B within the angle of view of the captured first image P1 and second image P2, as shown in FIGS. Therefore, after the first image P1 having only the background is captured and then the second image P2 in which the main subject C is present in the background is captured, if the background subjects A and B swing due to the wind, the second In this image P2, not only the main subject C but also background subjects A and B that have been swung by the wind exist as differential images in both images. Therefore, if the difference image is cut out using the first image P1 and the second image P2 as they are, not only the main subject C but also the changed portions of the background subjects A and B are cut out in the difference image. End up.

  However, in the present embodiment, the background subjects A and B are moved outside the area in accordance with the enlargement process, and then the difference image between the first image P1 and the second image P2 is cut out within the area ( Step S115). Therefore, even if there is a subject other than the main subject C in the first and second images P1 and P2, as shown in FIG. 3D, only the main subject C desired by the user is included. The difference image RP can be reliably cut out.

  In addition, in the present embodiment, the second image P2 is displayed on the monitor 4, and the image displayed on the monitor 4 is enlarged in accordance with the user's operation so that the area is designated. Therefore, the user can easily exclude unnecessary background subjects A and B from the cutout target area while viewing the image displayed on the monitor 4.

  When the difference image RP of the main subject C is cut out as described above, the difference image RP is recorded in the external memory 25 in association with or without association with the first and second images (step S116). . Therefore, the user can generate a composite image by reading out the differential image RP consisting only of the main subject C from the external memory 25 and pasting it on another background image at a later date.

(Second Embodiment)
FIG. 4 is a flowchart showing a processing procedure in the second embodiment of the present invention.
When the difference image generation mode is set by the user's operation on the key input unit 27, the CPU 13 executes the processing shown in this flowchart according to the program.

  That is, the CPU 13 determines whether or not the shooting mode in the difference image generation mode is set (step S201). If the shooting mode is set, it is determined whether or not the first shutter button operation has been detected (step S202). If the first shutter button operation is detected, the photographing process is executed and the image is recorded as the first image (step S203). Specifically, the CPU 13 compresses the image data stored in the S20 DRAM 22 when the shutter button is pressed, and records the compressed image data in the external memory 25 as a first image.

  Next, it is determined whether or not a second shutter button operation has been detected (step S204). If the second shutter button operation is detected, the photographing process is executed and the image is recorded as the second image in the external memory 25 in association with the first image recorded in step S203 (step S205). ).

  Subsequently, it is determined whether or not the cut-out mode in the difference image generation mode has been shifted, and the processes in steps S201 to S206 are repeated until the shift is made. Therefore, during this period, the user captures the background by the first shutter button operation, and captures the image in which the main subject to be cut out in the background exists by the second shutter button operation. Multiple sets of images are recorded in the external memory 25 with the image and the second image as a set.

  When the user sets the extraction mode in the difference image generation mode to synthesize the difference images, the process proceeds from step S206 to step S207, and any one of the plurality of sets of images recorded in the external memory 25 is one set. It is determined whether an image is selected. If a set of images composed of the associated first image and second image is selected from the external memory 25, the second image in the selected set of images is displayed on the monitor 4 ( Step S208). Therefore, if a set of images consisting of the first image P1 shown in FIG. 5D and the second image P2 shown in FIG. 5A is selected, as shown in FIG. The second image P2 is displayed on the monitor 4.

  As shown in FIG. 5D, in this example, the first image P1 is an image composed of a background in which trees exist as background subjects A and B. Further, as shown in FIG. 5A, the second image is an image in which a main subject C to be cut out is present in a background where similar trees exist as background subjects A and B.

  Next, the CPU 13 determines whether or not a point that is a touch position of the user's pen on the monitor 4 is detected by the touch panel 23 (step S209). If the touch point is detected, the touch point is displayed on the monitor 4 on which the second image P2 is displayed (step S210). Subsequently, it is determined whether or not an OK button operation has been detected (step S211), and the processing from step S209 is repeated until an OK button operation is detected.

  Therefore, during this time, when the user touches the main subject C around the main subject C so as to remove the background subjects A and B on the monitor 4 on which the second image P2 is displayed, the touch point display processing is performed in step S210 each time. Thus, a plurality of touch points T are displayed around the main subject C as shown in FIG.

  In this way, when a plurality of touch points T surrounding the main subject C excluding the background subjects A and B are displayed on the monitor 4, the user operates the OK button. Then, the CPU 13 advances the processing from step S211 to step S212, and displays the adjacent touch points on the monitor 4 by connecting them sequentially with a line (step S212).

  Therefore, as shown in FIG. 5C by the process in step S212, the monitor 4 has a connecting line L2 that sequentially connects adjacent touch points T around the main subject C included in the second image P2. Is displayed.

  Subsequently, the CPU 13 designates the area of the image portion surrounded by the connecting line of the second image and the corresponding image portion of the first image (step S213). Therefore, by the processing in step S213, as shown in FIG. 5C, the image portion PP2 surrounded by the connection line L2 of the second image P2, and the connection line as shown in FIG. The region of the image portion PP1 of the first image P1 surrounded by the connecting line L1 having the same position and the same shape as L2 is designated. Then, a difference image is cut out from the designated area, that is, from the partial image PP1 and the partial image PP2 (step S214).

  At this time, trees are present as background subjects A and B within the angle of view of the captured first image P1 and second image P2, as shown in FIGS. Therefore, after the first image P1 having only the background is captured and then the second image P2 in which the main subject C is present in the background is captured, if the background subjects A and B swing due to the wind, the second In this image P2, not only the main subject C but also background subjects A and B that have been swung by the wind exist as differential images in both images. Therefore, if the difference image is cut out using the first image P1 and the second image P2 as they are, not only the main subject C but also the changed portions of the background subjects A and B are cut out in the difference image. End up.

  However, in the present embodiment, in step S214, a difference image is cut out from the partial images PP1 and PP2 that are within the area designated by the user. Therefore, even if there is a subject other than the main subject C in the first and second images P1 and P2, as shown in FIG. 5E, only the main subject C desired by the user is included. The difference image RP can be reliably cut out.

  Moreover, in the present embodiment, the second image P2 is displayed on the monitor 4 and is displayed on the monitor 4 according to the user's operation so that the user can specify the area. Therefore, the user can easily exclude unnecessary background subjects A and B from the cutout target area while viewing the image displayed on the monitor 4.

  When the difference image RP of the main subject C is cut out as described above, the difference image RP is recorded in the external memory 25 in association with or without association with the first and second images (step S215). .

  In the present embodiment, the present invention is applied to a digital camera. However, the present invention is not limited to a digital camera, and may be applied to various devices having a shooting function such as a mobile phone. If it is a device capable of capturing from the outside the first image taken without the image and the second image in which the subject to be cut out exists in the first image, it should be applied to other electronic devices. Is possible.

1 Digital Camera 4 Monitor 11 Lens Block 12 Zoom Motor 13 CPU
18 CCD
19 Horizontal driver 20 Unit circuit 21 DSP
22 SDRAM
23 Touch Panel 25 External Memory 26 Flash Memory 27 Key Input Unit 111 Zoom Lens A Background Subject C Main Subject RP Difference Image T Touch Point

Claims (4)

Acquisition means for acquiring a first image and a second image in which a subject to be cut out exists in the first image as a set of set images;
Display means for displaying a second image of the first image and the second image acquired as a combined image by the acquisition means;
A region designating unit for designating a region where the subject exists in the second image displayed on the display unit as a target region to be cut out;
Cutting out means for setting a region corresponding to the target region specified by the region specifying unit in the first image, and cutting out a difference image between the target region and the set region;
The area specifying means further includes:
Display control means for enlarging and displaying the second image at an angle of view smaller than the angle of view at the time of shooting;
Designating means for designating, as the area, an angle of view smaller than the angle of view of the second image enlarged and displayed by the display control means;
The difference image generation apparatus characterized in that the cutout means cuts out the difference image within an area designated by the designation means . In accordance with the operation, further comprising instruction means for instructing the enlarged display,
2. The difference image generating apparatus according to claim 1, wherein the display control unit enlarges and displays the second image displayed on the display unit in response to an instruction from the instruction unit. Acquisition means for acquiring a first image and a second image in which a subject to be cut out exists in the first image as a set of sets;
Display means for displaying a second image of the first image and the second image acquired as a combined image by the acquisition means,
A region designating unit for designating a region where the subject exists in the second image displayed on the display unit as a target region to be cut out;
A cutting means for setting a region corresponding to the target region specified by the region specifying unit in the first image, and cutting out a difference image between the target region and the set region;
Function as
The area specifying means includes:
Display control means for enlarging and displaying the second image at an angle of view smaller than the angle of view at the time of shooting;
Functioning as a designating unit for designating, as the region, an angle of view smaller than the angle of view of the second image enlarged and displayed by the display control unit;
The cutout unit causes the function to cut out the difference image in an area designated by the designation unit. An acquisition step of acquiring a first image and a second image in which a subject to be cut out exists in the first image as a set of sets;
A display step of causing the display unit to display a second image of the first image and the second image acquired as a combined image in the acquisition step;
An area designating step of designating an area in which the subject of the second image displayed in the display step is present as a target area to be cut out;
A step of setting an area corresponding to the target area specified in the area specifying step in the first image, and cutting out a difference image between the target area and the set area. The area specification step
A display control step for enlarging and displaying the second image at an angle of view smaller than the angle of view at the time of shooting;
A designating step of designating, as the region, an angle of view smaller than the angle of view of the second image enlarged and displayed in the display control step.
The difference image generation method characterized in that, in the extraction step, the difference image is extracted in the region specified in the specification step.

Images