JP2010021710A - Imaging device, image processor, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a means capable of rapidly reproducing a moving image when storing RAW data in imaging the moving image. <P>SOLUTION: This imaging device includes an imaging part, a mode changeover part, and a control part. The imaging part acquires a plurality of images continuing in a time axis direction by imaging an object. The mode changeover part changes over a first mode for recording RAW data of an image to a second mode for recording a digital development image obtained by applying at least a gradation conversion process to the RAW data, and vice versa. The control part records a moving image file by the digital development images in a storage medium together with the RAW data in the first mode. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus that acquires moving images and still images, and peripheral technologies thereof.

2. Description of the Related Art Conventionally, imaging apparatuses having functions for moving image shooting and still image shooting are known. For example, Patent Document 1 discloses an example of an imaging apparatus that performs still image shooting during moving image shooting.
JP 2001-111934 A

  By the way, when RAW data is stored in moving image shooting with an imaging apparatus, it is preferable that a moving image can be quickly reproduced.

  Also, when a still image continuous in the time axis direction is added to generate a composite image with an image flow, and a moving image and a still image based on the composite image are recorded, the remaining capacity of the storage medium is taken into consideration. It is preferable to change the recording method.

  Furthermore, in an imaging device having functions of moving image shooting and still image shooting, it may be preferable to prohibit moving image shooting depending on the still image shooting mode. On the other hand, depending on the still image capturing mode, it may be preferable to capture a moving image.

  The present invention solves any of the above-described problems of the prior art. An object of the present invention is to provide means capable of quickly reproducing a moving image when storing RAW data in moving image shooting.

  Another object of the present invention is to provide means for recording data in consideration of the remaining capacity of a storage medium when recording a moving image and a still image by a composite image.

  Another object of the present invention is to provide means for appropriately controlling moving image shooting in accordance with a still image shooting mode.

  The imaging device according to the first aspect includes an imaging unit, a mode switching unit, and a control unit. The imaging unit captures a subject and acquires a plurality of images continuous in the time axis direction. The mode switching unit switches between a first mode for recording RAW data of an image and a second mode for recording a digitally developed image obtained by performing at least gradation conversion processing on the RAW data. In the first mode, the control unit records a moving image file based on the digitally developed image on the storage medium together with the RAW data.

  In the first aspect described above, the imaging apparatus may further include an image processing unit that generates a composite image by adding at least one or more added images different in the time axis direction to each image. Then, in the first mode, the control unit associates the incidental information indicating the addition ratio of the image to be added when generating the composite image with the RAW data, and uses a plurality of composite images continuous in the time axis direction. A moving image file may be recorded on a storage medium.

  The image processing apparatus according to the second aspect includes a data reading unit, a digital development processing unit, an image processing unit, and a control unit. The data reading unit reads the RAW data and the moving image file generated in the first mode of the first mode. The digital development processing unit performs a gradation conversion process and a retouch process on the RAW data to generate a corrected image. The image processing unit adds at least one or more added images different in the time axis direction to the corrected image to generate a combined image, and generates a corrected moving image file using the combined image based on the corrected image. The control unit deletes the moving image file generated by the imaging device when the corrected moving image file is generated or before the corrected moving image file is recorded on the storage medium.

  An image processing apparatus according to a third aspect includes a data reading unit, an image processing unit, a control unit, and a recording unit. The data reading unit reads a plurality of still images continuous in the time axis direction obtained by imaging the subject. The image processing unit adds at least one or more images to be added that differ in the time axis direction to the still image to generate a composite image. A control part produces | generates the 1st image file which recorded the addition ratio of the to-be-added image at the time of producing | generating a synthesized image in association with several still images, and the 2nd image file which recorded the moving image by a synthesized image. . The recording unit records the first image file and the second image file on a storage medium. Then, when the remaining capacity of the storage medium becomes less than the threshold, the control unit stops recording the second image file and saves the data of the first image file with priority over the data of the second image file.

  An image processing apparatus according to a fourth aspect includes a data reading unit, an image processing unit, a control unit, and a recording unit. The data reading unit reads a plurality of still images continuous in the time axis direction obtained by imaging the subject. The image processing unit adds at least one or more images to be added that differ in the time axis direction to the still image to generate a composite image. A control part produces | generates the 1st image file which recorded the addition ratio of the to-be-added image at the time of producing | generating a synthesized image to several still images, and the 2nd image file which recorded the moving image by a synthesized image . The recording unit records the first image file and the second image file on a storage medium. Then, when there are a plurality of first image files in the storage medium and the remaining capacity of the storage medium is less than the threshold, the control unit sorts the second image files stored in the storage medium in ascending order of priority. In addition to deleting, the data of the first image file is saved.

  The imaging device according to the fifth aspect includes an imaging unit and a control unit. The imaging unit can capture a subject to generate moving image data, and can generate still image data in response to an imaging instruction during moving image shooting. When the still image shooting mode is any of the bracket shooting mode, bulb shooting mode, and panorama shooting mode, the control unit prohibits shooting of a moving image.

  An imaging device according to a sixth aspect includes an imaging unit and a control unit. The imaging unit images a subject and generates moving image data or still image data. When the still image shooting mode is self-timer shooting, the control unit generates moving image data during a period until a still image is shot by the self-timer.

  In addition, what expresses the operation of the image processing apparatuses of the second to fourth embodiments as an image processing method and a program for causing a computer to execute the image processing method are also effective as specific embodiments of the present invention. It is.

  In one embodiment of the present invention, a moving image file of a digitally developed image is simultaneously recorded when RAW data is recorded, and the moving image can be quickly viewed by reproducing the moving image file.

  In another embodiment of the present invention, it is possible to record an appropriate image by giving priority to recording a still image according to the remaining capacity of the storage medium.

  In another embodiment of the present invention, by controlling the moving image shooting mode according to the still image shooting mode, it is possible to record an image on the imaging device in accordance with the user's intention.

<Description of First Embodiment>
(Description of electronic camera configuration)
FIG. 1 is a block diagram showing the configuration of the electronic camera of the first embodiment. The electronic camera includes an imaging optical system 11, a lens driving unit 12, an imaging element 13, an imaging element driving circuit 14, a signal processing circuit 15, a data processing circuit 16, a first memory 17, and a display control circuit 18. The monitor 19, the compression / decompression circuit 20, the recording I / F (interface) 21, the communication I / F (interface) 22, the operation member 23, the release button 24, the second memory 26, and the control circuit. 27 and bus 28.

  Here, the data processing circuit 16, the first memory 17, the compression / expansion circuit 20, the second memory 26 and the control circuit 27 are connected to each other via a bus 28. Further, the lens driving unit 12, the image sensor driving circuit 14, the signal processing circuit 15, the display control circuit 18, the recording I / F 21, the communication I / F 22, the operation member 23, and the release button 24 are connected to the control circuit 27, respectively. (In FIG. 1, signal lines connecting the signal processing circuit 15, the display control circuit 18, and the control circuit 27 are omitted for simplicity).

  The imaging optical system 11 includes a plurality of lens groups including a zoom lens and a focusing lens, and serves to form a subject image on the imaging surface of the imaging element 13. For simplicity, the imaging optical system 11 is illustrated as a single lens in FIG.

  Each lens position of the imaging optical system 11 is adjusted in the optical axis direction by the lens driving unit 12. The lens driving unit 12 includes a lens driving mechanism and adjusts the lens position in accordance with a lens driving command from the control circuit 27. For example, the focus adjustment of the imaging optical system 11 is performed by the lens driving unit 12 driving the focus lens back and forth in the optical axis direction. Further, zoom adjustment of the imaging optical system 11 is performed by the lens driving unit 12 driving the zoom lens to advance and retreat in the optical axis direction.

  The image sensor 13 photoelectrically converts a subject image by a light beam that has passed through the imaging optical system 11 to generate an analog image signal. The image sensor 13 in the present embodiment is capable of single-shot imaging of still images, continuous shooting imaging of still images, and imaging of moving images. The output of the image sensor 13 is connected to the signal processing circuit 15. The image sensor 13 is configured by a CCD image sensor or a CMOS image sensor.

  The image sensor drive circuit 14 generates a drive signal at a predetermined timing in accordance with a command from the control circuit 27 and supplies the drive signal to the image sensor 13. Then, the image sensor driving circuit 14 controls charge accumulation (imaging) and reading of the accumulated charges of the image sensor 13 by the drive signal.

  The signal processing circuit 15 is an ASIC that performs various types of signal processing on the output of the image sensor 13. Specifically, the signal processing circuit 15 performs correlated double sampling, gain adjustment, DC regeneration, A / D conversion, and the like. Parameters such as gain adjustment in the signal processing circuit 15 are determined in accordance with a command from the control circuit 27. The signal processing circuit 15 is connected to the data processing circuit 16, and the data after the signal processing is output to the data processing circuit 16.

  The data processing circuit 16 is a circuit that performs digital signal processing on the image data output from the signal processing circuit 15. In the data processing circuit 16, for example, image processing such as pixel defect correction processing, color interpolation processing, gradation conversion processing, contour enhancement processing, and white balance adjustment is executed. The data processing circuit 16 is connected to the display control circuit 18 and the compression / decompression circuit 20, respectively. Then, the data processing circuit 16 outputs the recorded image data after the image processing to the compression / decompression circuit 20 in accordance with a command from the control circuit 27.

  Further, the data processing circuit 16 executes image resolution conversion (pixel number conversion) processing in response to a command from the control circuit 27. As an example, when displaying a reproduced image on the monitor 19, the data processing circuit 16 executes resolution conversion (pixel number conversion) processing for matching the number of pixels of the monitor 19 with respect to the data of the image to be reproduced and displayed. (Note that unless otherwise specified, when an image is displayed on the monitor 19 in this specification, the number of pixels of the display image is adjusted by the data processing circuit 16). Then, the data processing circuit 16 outputs the reproduced image data after the resolution conversion to the display control circuit 18. When performing the electronic zoom processing, the data processing circuit 16 executes resolution conversion (pixel number conversion) processing on the input image data, and compresses / decompresses the image data after the resolution conversion. Each is output to the display control circuit 18.

  The first memory 17 is a buffer memory that temporarily stores image data in the pre-process and post-process of the processing by the data processing circuit 16 or the compression / decompression circuit 20. The first memory 17 is constituted by an SDRAM which is a volatile storage medium.

  In response to a command from the control circuit 27, the display control circuit 18 performs predetermined signal processing on the image data input from the data processing circuit 16 (for example, conversion of gradation characteristics in accordance with the display characteristics of the monitor 19). And output to the monitor 19. The display control circuit 18 further performs processing for superimposing overlay image data such as a shooting menu and a cursor on the image data. By such control of the control circuit 27 and the display control circuit 18, the subject image on which the overlay image is superimposed can be displayed on the monitor 19. Note that the monitor 19 in the present embodiment may be configured with any of an electronic viewfinder having an eyepiece and a liquid crystal display panel provided on the back surface of the camera housing.

  The compression / decompression circuit 20 performs predetermined compression processing on the image data input from the data processing circuit 16 in response to a command from the control circuit 27. The compression / decompression circuit 20 is connected to the recording I / F 21, and the compressed image data is output to the recording I / F 21. The compression / decompression circuit 20 executes a decoding process that is a reverse process of the compression process on the compressed image data. Note that the compression / decompression circuit 20 of the present embodiment has a configuration capable of performing lossless compression (so-called lossless encoding).

  The recording I / F 21 has a connector for connecting the storage medium 29. The recording I / F 21 writes / reads data to / from the storage medium 29 connected to the connector. The storage medium 29 includes a small hard disk, a memory card incorporating a semiconductor memory, an optical disk such as a DVD, and the like. In FIG. 1, a memory card is illustrated as an example of the storage medium 29. The storage medium 29 may be built in the electronic camera or may be detachably attached to the electronic camera. An external storage medium electrically connected via the communication I / F 22 may be used as a storage medium for reading and writing image data.

  Here, when recording data of an image captured in a photographing mode which is one of the operation modes of the electronic camera, the control circuit 27 displays a reproduced image corresponding to the recorded image on the monitor 19. The recorded image in this specification means a still image corresponding to still image data obtained by imaging and finally recorded on the storage medium 29 (or recorded on the storage medium 29). .

  When non-compressed recording of a recorded image is instructed to the control circuit 27 by the user's operation using the operation member 23, the compression / decompression circuit 20 does not perform compression processing and records the recorded image data as a recording I / F 21. Output to. Also in the case of the above uncompressed recording, the control circuit 27 causes the monitor 19 to display a reproduced image corresponding to the recorded image.

  In the playback mode, which is one of the operation modes of the electronic camera, the control circuit 27 causes the monitor 19 to display a playback image based on the image data stored in the storage medium 29. In this playback mode, the recording I / F 21 reads data of an image to be played back from the storage medium 29 in response to a command from the control circuit 27. Then, the compression / decompression circuit 20 performs a decoding process on the image data to be reproduced, and sends the decoded image data to the data processing circuit 16. Thereafter, the data processing circuit 16 and the display control circuit 18 execute the above-described processing on the decoded image data, so that the reproduced image is displayed on the monitor 19. When uncompressed image data is read from the storage medium 29, the compression / decompression circuit 20 sends the image data to the data processing circuit 16 without performing the decoding process.

  The communication I / F 22 controls data transmission / reception with an external device 30 (for example, a personal computer or an external storage medium) in accordance with the specifications of a known communication standard such as wired or wireless. Communication between the electronic camera and the external device 30 is performed via a wired or wireless communication line.

  The operation member 23 includes, for example, a command dial, a cross-shaped cursor key, a zoom operation button, a determination button, and the like. The operation member 23 receives various inputs of the electronic camera from the user. The operation member 23 may be provided with a setting member for setting the number of frames for generating an interpolation image, which will be described later.

  As an example, when receiving an input from the zoom operation button, the control circuit 27 outputs a lens drive command for the zoom lens and causes the lens drive unit 12 to drive the zoom lens forward and backward. As a result, the subject image formed on the imaging surface of the imaging device 13 is enlarged or reduced, and optical zoom adjustment is performed by the imaging optical system 11.

  When the control circuit 27 further receives an input from the zoom operation button, the control circuit 27 outputs a command to the data processing circuit 16 to change the conversion ratio of the resolution conversion processing for the image data in accordance with the user's operation. Thereby, the image displayed on the monitor 19 is enlarged or reduced, and electronic zoom adjustment is performed (electronic zoom). The conversion ratio of the resolution conversion process corresponds to the electronic zoom magnification. When the data processing circuit 16 changes the conversion ratio in the direction of increasing the electronic zoom magnification, a part of the reproduced image is enlarged and displayed on the monitor 19 (while the enlargement ratio is increased, the display range of the reproduced image is narrowed). . On the other hand, when the data processing circuit 16 changes the conversion ratio in the direction of decreasing the electronic zoom magnification, the enlargement ratio of the reproduced image displayed on the monitor 19 is lowered, but the display range of the reproduced image is widened. In the above-described shooting mode, captured image data corresponding to the display image on the monitor 19 can be recorded in the storage medium 29.

  In the shooting mode, the release button 24 receives from the user an instruction input for starting an autofocus (AF) operation before shooting by a half-press operation and an instruction input for starting an imaging operation by a full-press operation.

  The control circuit 27 executes a known contrast detection AF operation in response to a half-press operation of the release button 24. In this AF, a signal corresponding to a focus detection area set in advance in the shooting screen is used among image signals read from the image sensor 13. Specifically, the control circuit is configured to maximize the integrated value (so-called focus evaluation value) of the high frequency component for the data corresponding to the focus detection area among the data of the image signal processed by the signal processing circuit 15. 27 sends a lens drive command for the focus lens to the lens drive unit 12. The position of the focus lens that maximizes the focus evaluation value is a focus position that eliminates blurring of the edge of the subject image captured by the image sensor 13 and maximizes the contrast of the image (increases sharpness).

  The second memory 26 is a non-volatile storage medium such as a flash memory. The second memory 26 stores various setting data and the like. The second memory 26 stores a program executed by the control circuit 27.

  The control circuit 27 is a processor that comprehensively controls the operation of the electronic camera. For example, the control circuit 27 obtains the brightness of the object scene from the signal output from the image sensor 13. Then, the control circuit 27 executes a known AE calculation based on the above brightness information, and the imaging conditions in the imaging mode (the charge accumulation time of the imaging device 13, the aperture value of the aperture (not shown), the image signal) The degree of amplification).

  In addition, the control circuit 27 executes a program stored in the second memory 26 to perform digital development processing of RAW data, retouch processing of a developed image, generation processing of an interpolated image, and time-direction image. A synthesis process by addition or the like is executed (these processes will be described later).

(Explanation of shooting mode of the first embodiment)
Hereinafter, an example of the shooting mode of the electronic camera of the first embodiment will be described. The shooting mode of the electronic camera of the present embodiment is broadly divided into a still image mode that performs still image shooting and a moving image mode that performs moving image shooting. Then, the control circuit 27 switches between the still image mode and the moving image mode in accordance with the input from the operation member 23.

  In the still image mode, the control circuit 27 drives the image pickup device 13 in accordance with the full pressing operation of the release button 24 to pick up a recorded image. The image signal output from the image sensor 13 is subjected to predetermined processing by the signal processing circuit 15 and the data processing circuit 16. The recorded image data is finally recorded in the storage medium 29.

  In addition, the electronic camera according to the first embodiment has “bracket photography”, “bulb photography”, “panoramic photography”, and “self-timer photography” as setting items relating to still image photography.

  When “bracket shooting” is selected from the above setting items, the control circuit 27 records a plurality of frames by one shooting instruction while changing shooting conditions (exposure, white balance, focus position, etc.). Take an image. In a state where “bulb photography” is selected, the control circuit 27 continues to accumulate charges in the image sensor 13 while the release button 24 is pressed, and generates a recorded image of long-time exposure.

  In addition, in a state where “panoramic shooting” is selected, the control circuit 27 executes processing suitable for shooting a panoramic image (a continuous image obtained by joining images captured by dividing a subject).

  Specifically, the control circuit 27 at the time of panoramic shooting drives the imaging element 13 at predetermined intervals during shooting standby to perform a through image imaging operation, and displays the through image on the monitor 19 as a moving image. When a recorded image is captured during panorama shooting, the control circuit 27 generates a guide image corresponding to the joint portion of the panorama image from the immediately preceding frame and displays the guide image on the through image of the monitor 19 as an overlay display. . Accordingly, the user can easily perform panoramic shooting by performing framing so that the subject overlaps the through image and the guide image.

  In “self-timer shooting”, the control circuit 27 captures a recorded image after a predetermined time has elapsed since the release button 24 was fully pressed. At this time, the control circuit 27 can execute moving image shooting until the recording image is shot immediately before the release button 24 is fully pressed, and then can capture the recording image as the main image. In general, since self-timer shooting is used for commemorative shooting or the like, convenience for the user is improved by leaving a video record including the situation before shooting of the main image.

  Next, the operation of the electronic camera in the moving image mode will be described. In the moving image mode, the electronic camera captures a moving image during a period from the moving image shooting start instruction to the moving image shooting end instruction. Here, the control circuit 27 in the moving image mode determines the imaging condition so that each frame constituting the moving image can withstand viewing as a still image. For example, the control circuit 27 performs control so as to obtain a blur-free image by increasing the shutter speed.

  Further, the control circuit 27 in the moving image mode executes a still image (recorded image) acquisition operation when a full pressing operation of the release button 24 is accepted during moving image shooting. For example, the control circuit 27 performs marking on the frame of the moving image corresponding to the pressing timing of the release button 24, and extracts the marked moving image frame as a still image. Alternatively, the control circuit 27 may suspend moving image shooting and capture a still image.

  In this embodiment, when the setting item in the still image mode is “bracket shooting”, “bulb shooting”, or “panoramic shooting”, the control circuit 27 disables the moving image mode selection operation, Prohibit shooting. This is because in the case of “bracket shooting”, “bulb shooting”, and “panoramic shooting”, moving images cannot be shot, and the shooting mode settings conflict.

  Furthermore, the electronic camera according to the first embodiment includes “RAW data recording” and “image addition processing” as setting items relating to moving image shooting.

  Here, in the “RAW data recording” item, the user can set to the control circuit 27 whether or not to record RAW data of each frame. The RAW data in this embodiment refers to raw data obtained by performing pixel defect correction processing and color interpolation processing on digital image data output from the image sensor 13. When “RAW data recording” is set to OFF, the control circuit 27 performs normal image data obtained by performing gradation conversion processing, white balance correction, color correction, edge enhancement, noise removal processing, and the like on the RAW data. Generate a video file using.

  In the “image addition processing” item, the user can set the control circuit 27 whether or not to generate a composite image with image flow by performing image addition for each frame of the moving image.

  Hereinafter, an operation example of the electronic camera in the moving image mode of the present embodiment will be described with reference to the flowchart of FIG. In the example of FIG. 2, the description will be made on the assumption that the items “RAW data recording” and “image addition processing” are both turned on. For simplicity, the example in FIG. 2 will be described assuming that still image shooting is not performed during moving image shooting.

  Step S101: When receiving a moving image shooting start instruction from the operation member 23, the control circuit 27 drives the imaging device 13 to start moving image shooting. For example, the image sensor 13 at the time of moving image shooting outputs an image signal at a frame rate of 15 fps. The image signal of each frame output from the image sensor 13 is subjected to predetermined processing by the signal processing circuit 15 and the data processing circuit 16. At this time, the control circuit 27 temporarily stores RAW data corresponding to each frame and normal image data (digitally developed image data) obtained by subjecting the RAW data to gradation conversion processing or the like to the first memory 17. Remember me. The resolution (number of pixels) of the moving image data is set in accordance with the resolution standard of the playback apparatus.

  Note that, during moving image shooting in the moving image mode, the data processing circuit 16 performs resolution conversion processing on the normal image data and generates through image data in accordance with the number of pixels of the monitor 19. Then, the data processing circuit 16 sequentially supplies the through image data of each frame to the monitor 19 via the display control circuit 18. Therefore, during operation in the moving image mode, through images corresponding to the respective frames are sequentially displayed on the monitor 19 under the control of the display control circuit 18. Thereby, the user can perform framing while checking the state of the object scene by viewing the through image on the monitor 19.

  Step S102: The control circuit 27 obtains a motion vector of the subject between frames for two frames (S101) of normal image data adjacent in the time axis direction. For example, the control circuit 27 extracts the motion of the subject between frames in units of 16 × 16 pixel macroblocks, and obtains the motion vector of the subject between frames from this result. Note that the control circuit 27 in S102 calculates the motion vector between all frames at the time of moving image shooting.

  Step S103: The control circuit 27 obtains the maximum value Z of the magnitude of the motion vector among the motion vectors between the frames of each normal image data (obtained in S102). Then, the control circuit 27 records the maximum value Z of the magnitude of the motion vector in the first memory 17.

  Step S104: The control circuit 27 determines whether or not the maximum value Z of motion vector magnitudes (obtained in S103) is a “0” value between all frames. If the above requirement is satisfied (YES side), the process proceeds to S114. On the other hand, if the above requirement is not satisfied (NO side), the process proceeds to S105. Note that the YES side in S104 corresponds to, for example, a case where a subject with no change in brightness and color is photographed.

  Step S105: The control circuit 27 selects one specific image from the frame of the normal image data. This specific image is an image serving as a reference when generating the above-described composite image. In S105 in the present embodiment, the control circuit 27 sequentially selects specific images frame by frame from the frames of normal image data. In this selection, it is assumed that all the frames arranged in time series are selected without skipping the middle.

  Step S106: The control circuit 27 determines whether or not the current specific image selected in S105 is the first frame. If it is the first frame (YES side), the process proceeds to S112. On the other hand, if it is not the first specific image (NO side), the process proceeds to S107.

  Step S107: The control circuit 27 reads a corresponding value between the current specific image and the next specific image from the maximum value Z of the magnitude of the motion vector obtained in S103. Then, the control circuit 27 determines whether or not the maximum value Z of the magnitude of the read motion vector is less than the threshold Th (Z <Th). If the above requirement is satisfied (YES side), the process proceeds to S108. On the other hand, if the above requirement is not satisfied (NO side), the process proceeds to S109.

  Step S108: The control circuit 27 sets the number of frames to be added at the time of generating the composite image to “3” and determines the addition ratio between the specific image and the added image.

  Specifically, the control circuit 27 includes, in the normal image data, one frame of the previous (past) and subsequent (future) in the time axis direction with respect to the specific image (that is, the past 1 with reference to the specific image). Frame and one future frame) are selected as images to be added. Then, the control circuit 27 sets the addition ratio of the specific image to 0.5. Further, the control circuit 27 sets the addition ratio of each added image to 0.25. That is, when expressed in the time axis direction, the image addition ratio in S108 is “0.25: 0.5: 0.25” (see FIG. 3A). Thereafter, the control circuit 27 proceeds to S110.

  Step S109: The control circuit 27 sets the number of frames to be added at the time of generating the composite image to “5” and determines the addition ratio between the specific image and the added image.

  Specifically, in the normal image data, the control circuit 27 includes two frames, that is, the front and rear frames in the time axis direction with respect to the specific image (that is, the past two frames and the future two frames based on the specific image). ) Are selected as images to be added. Then, the control circuit 27 sets the addition ratio of the specific image to 0.4. In addition, the control circuit 27 sets the addition ratio of the first frame before and after the specific image among the respective images to be added to 0.2. Further, the control circuit 27 sets the addition ratio of the second and the second frames before and after the specific image among the images to be added to 0.1. That is, when expressed in the time axis direction, the image addition ratio in S109 is “0.1: 0.2: 0.4: 0.2: 0.1” (see FIG. 3B).

  Step S110: The control circuit 27 temporarily stores, in the first memory 17, associated information indicating the number of addition frames and the addition ratio obtained in S108 or S109 in association with the RAW data.

  Step S111: The control circuit 27 adds the added image to the specific image based on the number of addition frames and the addition ratio obtained in S108 or S109 to generate a composite image. Then, the control circuit 27 temporarily stores the generated composite image in the first memory 17.

  Here, FIG. 4A shows an example of the specific image. FIG. 4B schematically shows a composite image using the specific image of FIG. 4A as a display nucleus. In the example of FIG. 4B, the number of addition frames is “5”, and the image addition ratio is “0.1: 0.2: 0.4: 0.2” when expressed in the time axis direction. : "Is shown as an example. In the composite image, the past image and the future image are added in the time axis direction to the specific image serving as the nucleus of the display, so that an image flow occurs on the image. For this reason, when a plurality of synthesized images are viewed as a moving image, the flow of motion is smooth and preferable. Note that the number of added frames and the addition ratio when generating a composite image are changed by the control circuit 27 according to the maximum value (Z) of the magnitude of the motion vector between recorded images taken continuously. Therefore, in the present embodiment, the image flow of the composite image can be adjusted to a preferable amount in accordance with the movement of the subject in the captured scene.

  Step S112: The control circuit 27 determines whether or not the determination has been completed from the last frame to the frame one frame before the normal image data. If the above requirement is satisfied (YES side), the process proceeds to S113. On the other hand, if the above requirement is not satisfied (NO side), the control circuit 27 returns to S105 and repeats the above operation.

  Step S113: The compression / decompression circuit 20 generates a first image file by performing lossless compression processing on the RAW data of each frame in accordance with a command from the control circuit 27. Note that the RAW data of the first image file in S113 is associated with each frame of additional information (S110) indicating the number of added frames and the added ratio for each specific image. As a result, when the RAW data of the first image file is digitally developed in the post-processing step, the control circuit 27 can easily generate a composite image by referring to the number of additional frames and the addition ratio of the auxiliary information. Become.

  In addition, the compression / decompression circuit 20 generates a second image file by compressing a series of synthesized images (S111) in the MPEG format in accordance with a command from the control circuit 27. As a result, by reproducing the second image file, it is possible to quickly reproduce the moving image without performing development processing of the RAW data. Further, since each frame of the second image file is composed of a composite image, a preferable moving image having a smooth flow of motion can be reproduced on the monitor 19.

  Then, the first image file and the second image file are finally recorded in the storage medium 29. Thereafter, the control circuit 27 ends the processing in the moving image mode.

  Step S114: The compression / decompression circuit 20 generates a first image file by performing lossless compression processing on the RAW data of each frame in accordance with an instruction from the control circuit 27. Note that in the case of S114, since the composite image is not generated, the above-mentioned incidental information is not included in the first image file. The compression / decompression circuit 20 generates a second image file by compressing a series of normal image (S101) data in the MPEG format in accordance with a command from the control circuit 27. In the case of S114 as well, by reproducing the second image file, it is possible to quickly reproduce the moving image without performing the RAW data development processing. Above, description of the flowchart of FIG. 2 is complete | finished.

  When the item “RAW data recording” is set to the on state and the item “image addition processing” is set to the off state, the control circuit 27 adds the addition ratio and the like as in S114. Without generating the first image file in which the RAW data is recorded and the second image file based on the normal image.

(Description of retouch processing in the first embodiment)
Next, a case where the RAW data (first image file generated in S113) obtained in the moving image mode is retouched with the electronic camera will be described with reference to the flowchart of FIG.

  Step S201: The control circuit 27 reads the first image file and the second image file from the storage medium 29. Then, the compression / decompression circuit 20 executes a decoding process of the RAW data in the first image file in accordance with a command from the control circuit 27. Note that the decoded data of each image is temporarily stored in the first memory 17.

  Step S202: The control circuit 27 displays an image included in the first image file or the second image file on the monitor 19, and accepts designation of an image to be a retouch process from the user. Here, the retouching process is a process in which a user manually processes a RAW data image in a post-process after photographing. As an example, the contents of the retouch process include brightness correction of the entire image (brightness process), image noise removal, image saturation adjustment and color correction, contrast correction, tone curve correction, contour enhancement process, unsharp mask Processing, shadow point processing (processing for specifying the range of RGB values to be darkest in the image data), highlight point processing (processing for specifying the range of RGB values to be brightest in the image data), and the like.

  Step S203: The control circuit 27 performs retouch processing of the content specified by the user on the RAW data to be subjected to retouch processing. Then, the control circuit 27 performs various image processing including gradation conversion processing on the data after the retouch processing to generate corrected image data. Note that the control circuit 27 may compress the retouched corrected image in the JPEG format and record the compressed image in the storage medium 29 as independent still image data.

  Step S204: The control circuit 27 generates composite image data from the RAW data of the first image file.

  Specifically, first, the control circuit 27 performs various kinds of image processing including gradation conversion processing on the RAW data of the first image file to generate digitally developed images having the same resolution as the above-described corrected image. Next, the control circuit 27 uses the corrected image (S203) and the digitally developed image as specific images, and adds the images to be added back and forth in the time axis direction to generate a composite image. At this time, the control circuit 27 uses the incidental information (S110) of the first image file to determine the number of addition frames and the addition ratio when the corrected image is a specific image. Further, the control circuit 27 in S204 may change the number of addition frames and the addition ratio based on the user input. The control circuit 27 temporarily stores the generated composite image in the first memory 17.

  Step S205: In response to a command from the control circuit 27, the compression / decompression circuit 20 compresses the series of synthesized image data generated in S204 in MPEG format to generate a corrected moving image file. In this corrected moving image file, a composite image is generated using the corrected image, so that the content of the retouch processing in S203 is reflected in the moving image reproduction.

  Step S206: The control circuit 27 records the corrected moving image file (S205) in the storage medium 29 and deletes the second image file (generated in S113) stored in the storage medium 29. As a result, the second image file is replaced with a corrected moving image file reflecting the state after retouching, so that the recording capacity of the storage medium 29 can be effectively utilized. Above, description of the flowchart of FIG. 5 is complete | finished.

<Description of Second Embodiment>
Next, with reference to FIG. 6, an example of an operation in the moving image mode in the electronic camera in the second embodiment will be described. Here, the configuration of the electronic camera in the second embodiment is the same as that of the electronic camera of the first embodiment shown in FIG.

  The operation example shown in FIG. 6 will be described on the assumption that the items “RAW data recording” and “image addition processing” are both turned on. In addition, it is assumed that the second image file header stores number-of-times data indicating the number of times of reproduction of moving images and protect data indicating prohibition of image erasure.

  Step S301: Upon receiving a moving image shooting start instruction from the operation member 23, the control circuit 27 drives the imaging device 13 to start moving image shooting. And the control circuit 27 produces | generates a 1st image file and a 2nd image file by the operation | movement (S101-S114) similar to the moving image mode of said 1st Embodiment. Then, the data of the first image file and the data of the second image file are sequentially stored in the storage medium 29 by the control circuit 27.

  Therefore, when the maximum value Z of the motion vector size is not “0” in the moving image, the additional information indicating the number of added frames and the added ratio for each specific image is recorded together with the RAW data in the first image file. . On the other hand, a moving image composed of a plurality of composite images is recorded in the second image file. Note that the control circuit 27 in S301 sets a flag indicating the generation state of the second image file to an on state.

  Step S302: The control circuit 27 determines whether or not the remaining capacity of the storage medium 29 has become equal to or less than a threshold during shooting. If the above requirement is satisfied (YES side), the process proceeds to S303. In this case (YES side of S302), the control circuit 27 may output a warning display indicating that the remaining capacity of the storage medium 29 has decreased or a display indicating the shootable time to the monitor 19 or the like. Good. Note that the remaining capacity and the shootable time of the storage medium 29 may be increased by stopping the recording of the second image file described later or deleting the second image file. On the other hand, if the above requirement is not satisfied (NO side), the process proceeds to S308.

  Step S303: The control circuit 27 determines whether or not the flag indicating the generation state of the second image file is on (whether the second image file is being generated). If the above requirement is satisfied (YES side), the process proceeds to S304. On the other hand, if the above requirement is not satisfied (NO side), the process proceeds to S305.

  Step S304: The control circuit 27 stops the recording of the second image file and changes the flag indicating the generation state of the second image file to an off state. Then, the data of the second image file recorded in the storage medium 29 is overwritten with the data of the first image file generated thereafter. Further, the control circuit 27 in S304 may output a display indicating the recording stop of the second image file to the monitor 19 or the like. The control circuit 27 thereafter proceeds to S308.

  Step S305: The control circuit 27 determines whether or not another second image file generated by the previous shooting exists in the storage medium 29. If the above requirement is satisfied (YES side), the process proceeds to S306. On the other hand, if the above requirement is not satisfied (NO side), the process proceeds to S307.

  Step S306: The control circuit 27 deletes the other second image files in descending order of priority. For example, the control circuit 27 sets the other second image file having no protected data in the header as an erasure candidate image file. Next, the control circuit 27 refers to the number-of-times data in the header of the deletion candidate image file, and designates the image file with the smallest number of reproductions as the deletion target. When there are a plurality of image files having the same number of reproductions, the control circuit 27 sets the image file having the oldest shooting date and time as an object to be deleted.

  Then, the control circuit 27 changes the image file to be erased (another second image file) to a state where it can be overwritten. As a result, the image file to be erased is overwritten with the data of the first image file generated thereafter. Note that the control circuit 27 repeats the above-described steps to sequentially erase the deletion candidate image files from the storage medium 29. The control circuit 27 thereafter proceeds to S308.

  Step S307: The control circuit 27 determines whether or not moving image shooting cannot be continued due to insufficient storage medium 29 capacity. When the above requirement is satisfied (YES side), the control circuit 27 displays on the monitor 19 or the like that the storage medium 29 has no capacity, and ends the shooting operation in the moving image mode. On the other hand, if the above requirement is not satisfied (NO side), the process returns to S302 and the control circuit 27 repeats the above operation.

  Step S308: The control circuit 27 determines whether or not a moving image shooting end instruction has been received from the operation member 23. When the above requirement is satisfied (YES side), the control circuit 27 ends the shooting operation in the moving image mode. On the other hand, if the above requirement is not satisfied (NO side), the process returns to S302 and the above operation is repeated. Above, description of the flowchart of FIG. 6 is complete | finished.

  In the second embodiment, when the remaining capacity of the storage medium 29 is reduced, the recording of the second image file generated in the moving image mode is stopped (S304). Therefore, it is possible to record the RAW data of the first image file for a longer time. Even in this case, it is possible that a composite image can be generated from the RAW data of the first image file after the photographing is completed, so that it is considered that no major problem occurs.

  In the second embodiment, when the remaining capacity of the storage medium 29 decreases, the other second image files generated by the previous shooting are deleted in ascending order of priority (S306). Therefore, it is possible to record the RAW data of the first image file for a longer time. Even in this case, since the second image file can be restored from the first image file corresponding to the other second image file after the photographing is finished, it is considered that no serious problem occurs.

<Supplementary items of the embodiment>
(1) In the first embodiment and the second embodiment described above, the example of recording RAW data at the time of moving image shooting has been described. However, the first embodiment and the second embodiment can be similarly applied to the case of RAW data obtained by continuous shooting of still images, for example.

  (2) When generating the composite image shown in the first embodiment, an interpolated image for interpolating between frames of a moving image is generated, and this interpolated image may be used as an added image or a specific image. Good.

  (3) In the second embodiment, the example in which the first image file and the second image file are recorded during shooting in the moving image mode has been described. However, the operation of the electronic camera in the second embodiment can also be applied when, for example, the remaining capacity of the storage medium 29 is reduced when performing the retouching process after shooting.

  (4) In the above embodiment, an example in which the control circuit 27 of the electronic camera performs various processes has been described. For example, the external device 30 (such as a personal computer) connected to the electronic camera is connected to the control circuit 27 of the above embodiment. The process may be executed. In addition, in the above embodiment, a part of the processing executed by the control circuit 27 of the electronic camera is borne by the external device 30, and the electronic camera and the external device 30 cooperate with each other to perform the first embodiment or the second embodiment. You may implement | achieve the algorithm demonstrated by the form.

  (5) In the retouching process of the first embodiment described above, the example in which the second image file stored in the storage medium 29 is deleted when the corrected moving image file is generated has been described. However, in the retouching process of the first embodiment, the control circuit 27 may delete the second image file in the storage medium 29 in advance before the corrected moving image file is stored in the storage medium 29. .

  The present invention can be implemented in various other forms without departing from the spirit or the main features thereof. Therefore, the above-described embodiment is merely an example in all respects and should not be interpreted in a limited manner. The present invention is shown by the scope of claims, and the present invention is not limited to the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

The block diagram which shows the structure of the electronic camera of 1st Embodiment. A flowchart showing an operation example of the electronic camera in the moving image mode in the first embodiment. The figure which shows the image and addition ratio which comprise a composite image (A) A diagram showing an example of a specific image, (b) a diagram schematically showing a composite image with FIG. 4 (a) as a display nucleus. A flowchart for explaining an operation example of retouch processing in the first embodiment. The flowchart which shows the operation example of the moving image mode in the electronic camera in 2nd Embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 13 ... Image sensor, 19 ... Monitor, 21 ... Recording I / F, 22 ... Communication I / F, 23 ... Operation member, 24 ... Release button, 27 ... Control circuit, 29 ... Storage medium, 30 ... External device

Claims (8)

An imaging unit that captures a subject and obtains a plurality of continuous images in the time axis direction;
A mode switching unit that switches between a first mode for recording RAW data of the image and a second mode for recording a digitally developed image obtained by performing at least a gradation conversion process on the RAW data;
A controller that records a moving image file of the digitally developed image on a storage medium together with the RAW data in the first mode;
An imaging apparatus comprising: The imaging device according to claim 1,
An image processing unit that generates a composite image by adding at least one or more images to be added different in the time axis direction to each of the images;
In the first mode, the control unit associates incidental information indicating an addition ratio of an image to be added when generating the composite image with the RAW data, and a plurality of continuous information in a time axis direction. An image pickup apparatus that records the moving image file based on a composite image on the storage medium. A data reading unit that reads RAW data and a moving image file generated in the first mode of the imaging apparatus according to claim 1;
A digital development processing unit that performs gradation conversion processing and retouch processing on the RAW data to generate a corrected image;
An image processing unit that generates a composite image by adding at least one or more images to be added that differ in the time axis direction to the correction image, and generates a corrected moving image file using the composite image based on the correction image When,
A controller that deletes the moving image file generated by the imaging device when the corrected moving image file is generated or before the corrected moving image file is recorded on the storage medium;
An image processing apparatus comprising: A data reading unit for reading a plurality of still images continuous in the time axis direction obtained by imaging a subject;
An image processing unit that generates a composite image by adding at least one image to be added different in the time axis direction to the still image;
Generating a first image file in which the addition ratio of the image to be added when generating the composite image is recorded in association with the plurality of still images, and a second image file in which a moving image of the composite image is recorded; A control unit;
A recording unit for recording the first image file and the second image file on a storage medium,
The control unit stops recording the second image file when the remaining capacity of the storage medium becomes less than a threshold, and prioritizes the data of the first image file over the data of the second image file. An image processing apparatus characterized by storing. A data reading unit for reading a plurality of still images continuous in the time axis direction obtained by imaging a subject;
An image processing unit that generates a composite image by adding at least one image to be added different in the time axis direction to the still image;
Generating a first image file in which the addition ratio of the image to be added when the composite image is generated is associated with a plurality of the still images and a second image file in which a moving image of the composite image is recorded; A control unit;
A recording unit for recording the first image file and the second image file on a storage medium,
The control unit prioritizes the second image file stored in the storage medium when there are a plurality of the first image files in the storage medium and the remaining capacity of the storage medium is less than a threshold value. An image processing apparatus that deletes data in the descending order and stores data of the first image file.   A program for causing a computer to execute processing of the image processing apparatus according to any one of claims 3 to 5. An imaging unit that captures a subject to generate moving image data, and generates still image data in response to an imaging instruction during shooting of the moving image;
When the setting of the still image shooting mode is one of a bracket shooting mode, a bulb shooting mode, and a panorama shooting mode, a control unit that prohibits shooting of the moving image;
An imaging apparatus comprising: An imaging unit that images a subject and generates at least one of moving image data and still image data;
When the setting of the still image shooting mode is self-timer shooting, a control unit that generates moving image data during a period until the still image is shot by the self-timer;
An imaging apparatus comprising:

Images