JP6665019B2 - Imaging device - Google Patents

Description

The present invention relates to an imaging device capable of recording both information before and after processing.

  2. Description of the Related Art In recent years, portable devices with a photographing function (photographing devices) such as digital cameras have become widespread. Many of these types of photographing devices are designed so that various image processing can be enjoyed. For example, a device capable of performing time-lapse photography that continuously captures images under specific conditions such as a predetermined time interval and generates a moving image having a reproduction time shorter than the photography time has been commercialized. For example, in time-lapse photography, a scene fixed on a tripod is used to photograph the scenery of the day to observe changes in the scenery in a short time. Various uses are possible, such as observing over time.

  By the way, conventionally, in a photographing apparatus, after performing a predetermined image signal processing on a captured image obtained by an image sensor, the image is converted into a standardized general-purpose image format, for example, a JPEG format, and recorded or transmitted. ing. However, in an image in the JPEG format (JPEG image), part of the image information obtained from the image sensor is lost due to the JPEG data compression process. Further, an image having an image quality different from the image quality desired by the user may be generated by the image signal processing on the information obtained from the image sensor.

  Therefore, image data obtained by performing only minimal correction such as lens distortion correction and pixel defect correction on an image captured from the image sensor is subjected to reversible conversion compression processing to create RAW data, Some photographing devices adopt a method of recording and transmitting the RAW data as it is without converting it to the JPEG format. In this case, no information is lost, but since the data is not compressed, a load is imposed on image data recording and transfer. As described above, various types of image processing such as image compression have different characteristics before and after the processing.

  Patent Document 1 discloses a device that includes a display image switching unit that switches image data displayed on an image display unit from a pre-compression image to a post-compression image, and confirms a difference in image quality.

JP 2004-113321 A

  However, when only one of the information before and after the processing is recorded, each information has its own merits and demerits, and it has not been possible to easily reproduce data in accordance with various needs.

SUMMARY OF THE INVENTION An object of the present invention is to provide an imaging apparatus capable of recording information before and after processing and selecting such information in accordance with a predetermined switching condition to reproduce data according to needs.

In addition, the imaging device according to one embodiment of the present invention includes an imaging unit that acquires an image, a processing unit that processes the image, and an image that is not processed by the processing unit and an image that has been processed by the processing unit. A recording unit that records the image in association with, a determination unit that determines whether a predetermined switching condition has been satisfied, and reads and outputs the processed image from the recording unit, and the predetermined switching condition has been satisfied. When it is indicated, the control unit reads and outputs the image before processing from the recording unit in place of the image after processing from the recording unit, and a signal processing unit that performs signal processing. The unit is capable of performing time-lapse photography to repeatedly image, the recording unit records both the pre-processing and post-processing images obtained by the time-lapse photography, and the signal processing unit performs the time-lapse photography. A time-lapse moving image is generated using the processed image, and the switching condition is satisfied when the signal processing unit attempts to perform image processing on the time-lapse moving image. At the time of processing, an image before processing is supplied to the signal processing unit as an image to be processed instead of the image after processing.

  According to the present invention, by recording information before and after processing and selecting such information according to a predetermined switching condition, there is an effect that data can be reproduced according to needs.

FIG. 2 is a block diagram showing a circuit configuration of the information terminal device according to the first embodiment of the present invention. FIG. 9 is an explanatory diagram for explaining an association process performed by an association unit when a resize process is performed by a resize unit; FIG. 4 is an explanatory diagram showing an AVI file 30. 5 is a flowchart for explaining the operation of the first embodiment. FIG. 6 is a block diagram showing a second embodiment of the present invention. 5 is a flowchart showing the operation of the imaging device 40. 5 is a flowchart showing the operation of the information terminal device 50. FIG. 4 is an explanatory diagram illustrating an example of an appearance of an imaging device 40 and a display on a display screen. FIG. 3 is an explanatory diagram for explaining a time-lapse movie. FIG. 3 is an explanatory diagram for explaining a time-lapse movie. FIG. 4 is an explanatory diagram illustrating an example of an image displayed on a display screen of a display unit 57 of the information terminal device 50. FIG. 4 is an explanatory diagram illustrating an example of an image displayed on a display screen of a display unit 57 of the information terminal device 50. FIG. 4 is an explanatory diagram illustrating an example of an image displayed on a display screen of a display unit 57 of the information terminal device 50. FIG. 4 is an explanatory diagram illustrating an example of an image displayed on a display screen of a display unit 57 of the information terminal device 50. FIG. 4 is an explanatory diagram illustrating an example of an image displayed on a display screen of a display unit 57 of the information terminal device 50. FIG. 4 is an explanatory diagram illustrating an example of an image displayed on a display screen of a display unit 57 of the information terminal device 50. FIG. 4 is an explanatory diagram illustrating an example of an image displayed on a display screen of a display unit 57 of the information terminal device 50. FIG. 4 is an explanatory diagram illustrating an example of an image displayed on a display screen of a display unit 57 of the information terminal device 50. FIG. 4 is an explanatory diagram illustrating an example of an image displayed on a display screen of a display unit 57 of the information terminal device 50.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First Embodiment)
FIG. 1 is a block diagram showing a circuit configuration of the information terminal device according to the first embodiment of the present invention. Note that the information terminal device in FIG. 1 is an example of an imaging device as a device that acquires audio information or image information.
Here, RAW data and JPEG data will be described as an example of an image before and after processing, but the RAW data is not standardized, and unlike JPEG data, it is difficult to use it on an external device. Further, the RAW data has a relatively large data amount, and it is difficult to use the RAW data in consideration of a communication band. For this reason, there has been a problem that a JPEG format image file may be used for recording and transmission at the expense of missing image information. Note that not only images but also audio collected by a microphone or the like may be recorded and transmitted as standardized compressed audio data instead of uncompressed audio data in terms of data capacity and the like. There is a similar problem as in the case. Due to the diversification of values, the processing of information and data is required to be capable of discerning reproduction. However, according to the technology of this embodiment, non-compressed information or lossless An information terminal device, an imaging device, and an image processing device that can record both compressed information and select these pieces of information according to predetermined switching conditions to obtain convenient image output and audio output with high image quality or high sound quality. An information processing method can be provided.

  The imaging device, which is the information terminal device according to the present embodiment, can not only record information such as images and audio in a lossy compression manner, but also record in an uncompressed state or record in a lossless compression manner. . Examples of information to be recorded without compression (non-compression information) or information to be recorded by lossless compression (hereinafter, lossless compression information) include RAW data (RAW image) obtained by an imaging device. The RAW image may be an image of the output of the image sensor itself, or may be an image obtained by performing only a minimum correction such as lens distortion correction or pixel defect correction on an image captured from the image sensor. Further, it may also refer to an image obtained by performing a reversible conversion compression process on such an image. By subjecting such a RAW image to predetermined signal processing and JPEG compression, JPEG data (JPEG image) in JPEG format is obtained.

  For audio, uncompressed information includes, for example, WAV, and lossless compressed information includes, for example, MPEG-4ALS. The lossy compression information includes, for example, JPEG data and MP3 data. In the following, for the sake of simplicity, an image will be mainly described, but sound can be processed in the same manner.

  Of the recorded RAW images and JPEG images, JPEG images are often used during transmission from the viewpoint of data capacity and ease of handling. However, when a recorded image is output after being subjected to some kind of image processing, a desired image effect may not be obtained even if a predetermined effect is applied to a JPEG image. In the present embodiment, a predetermined switching condition is determined, and when the switching condition is satisfied, a desired image processing effect is obtained by using a RAW image for image effect processing instead of a JPEG image. . Further, the present embodiment provides a highly versatile image output by recording and outputting an image after image processing in a JPEG format.

  In FIG. 1, an imaging device 10 has an imaging unit 12a including an imaging device such as a CCD or a CMOS sensor. The imaging unit 12a is provided with a lens (not shown), and an optical image of a subject is guided through the lens. The lens may have a function of zooming and focusing. The imaging unit 12a obtains a captured image by converting an optical image of a subject into an electric signal using an imaging element. This captured image is supplied to the signal processing unit 13.

  The imaging device 10 is provided with a control unit 11, and the control unit 11 controls each unit of the imaging device 10. The control unit 11 may be constituted by a processor such as a CPU (not shown) or may operate according to a program stored in a memory (not shown). The imaging device 10 includes an operation unit 17 including a shutter button, a function button, various switches for setting a shooting mode, and the like provided on a housing of the imaging device 10. The operation unit 17 receives a user operation and outputs an operation signal based on the user operation to the control unit 11. The control unit 11 can control the imaging control unit 13a and the sound collection control unit 13b in the signal processing unit 13 based on an operation signal from the operation unit 17. The imaging control unit 13a is controlled by the control unit 11, generates a focus signal and a zoom signal, and drives the lens of the imaging unit 12a to perform zooming, focusing, and the like.

  The imaging device 10 is provided with a sound pickup unit 12b. The sound collection unit 12b is configured by, for example, a microphone, and collects surrounding sounds. The sound collection control unit 13b of the signal processing unit 13 can be controlled by the control unit 11 to drive the sound collection unit 12b to control sound collection.

  The control unit 11 can supply a captured image (hereinafter, also referred to as an original image) from the imaging unit 12a and a sound (hereinafter, also referred to as an original sound) from the sound collection unit 12b to the signal processing unit 13. The signal processing unit 13 performs predetermined signal processing, for example, color adjustment processing, matrix conversion processing, noise removal processing, and other various signal processing on the read captured image. Further, the signal processing unit 13 performs predetermined signal processing on the read voice.

  The recording device 14 is provided in the imaging device 10. The signal processing unit 13 is provided with a recording control unit 13f. The recording control unit 13f is controlled by the control unit 11 to control recording and reproduction on the recording unit 14. The control unit 11 can provide the recording unit 14 with the picked-up image and the sound pickup sound that have been subjected to the signal processing by the signal processing unit 13 so as to be recorded.

  The signal processing unit 13 includes a lossless compression processing unit 13c and a lossy compression processing unit 13d. The reversible compression processing unit 13c is controlled by the control unit 11 to reversibly compress the captured image and output a RAW image. Note that the reversible compression processing unit 13c may be controlled by the control unit 11 to output a captured image as a RAW image without compression. Note that the RAW image may have been subjected to only minimum correction such as lens distortion correction and pixel defect correction in the signal processing unit 13. Further, the reversible compression processing section 13c is controlled by the control section 11, and outputs the collected voice without compression.

  The irreversible compression processing unit 13d is controlled by the control unit 11 to irreversibly compress the captured image and the collected sound and outputs compressed data. For example, the irreversible compression processing unit 13d performs JPEG compression on the RAW image and outputs a JPEG image in the JPEG format. Further, for example, the irreversible compression processing unit 13d compresses non-compressed audio data such as PCM data using MP3, and outputs MP3 audio in MP3 format.

  The recording control unit 13f is capable of giving both the output of the lossless compression processing unit 13c and the output of the lossy compression processing unit 13d to the recording unit 14 for recording. In this case, in the present embodiment, the associating unit 13e controls the recording control unit 13f to determine the output of the lossless compression processing unit 13c and the output of the irreversible compression processing unit 13d for the image having the common original image. Are recorded in the recording unit 14 in association with each other. For example, the associating unit 13e assigns a file name having only a different extension to the output of the lossless compression processing unit 13d and the output of the lossy compression processing unit 13d corresponding to the same original image or original audio. Record.

  Although FIG. 1 shows only one recording unit 14, for example, a double-slot card interface capable of simultaneously recording data on two memory cards may be employed as the recording unit 14. In this case, the output of the lossless compression processing unit 13c may be recorded via one of the double slots, and the output of the lossy compression processing unit 13d may be recorded via the other of the double slots.

  Further, the signal processing unit 13 is provided with a display control unit 13h. The display control unit 13h performs various processes related to display. The display control unit 13h controls the display on the display unit 16 under the control of the control unit 11. The display unit 16 has a display screen such as an LCD, and displays an image given from the display control unit 13h. For example, the control unit 11 can control the display control unit 13h to display a through image or a reproduced image of a captured image on the display screen of the display unit 16.

  A touch panel as the operation unit 17 may be provided on the display screen of the display unit 16. The touch panel can generate an operation signal corresponding to a position on the display screen pointed by a user with a finger. This operation signal is supplied to the control unit 11. Thereby, when the user touches or slides on the display screen, the control unit 11 can detect the operation. That is, the control unit 11 can detect various operations such as a user's touch position, an operation of closing and separating fingers (pinch operation), a slide operation, a position reached by a slide operation, a slide direction, and a touching period. It is possible to execute a process corresponding to a user operation.

  The signal processing section 13 is also provided with a resizing section 13g. The resizing section 13g can reduce the taken image to obtain a resized image. For example, when an electronic zoom process is instructed by the control unit 11, the resizing unit 13g cuts out a designated range in the captured image to create a resized image.

  In the present embodiment, when clipping is performed by the resizing unit 13g, the associating unit 13e adds information on the clipping range to the recording file and records it.

  FIG. 2 is an explanatory diagram for explaining the associating process by the associating unit 13e when the resizing process is performed by the resizing unit 13g. FIG. 2 shows a set of files 21 associated by the association unit 13e. One set of files 21 includes a JPEG image file 22 and a RAW image file 23. The JPEG image file 22 and the RAW image file 23 may each be composed of a plurality of files.

  In the example of FIG. 2, the file 22 of the JPEG image includes cut-out range information 22a of the original image. The original image indicates an output image of the image sensor of the imaging unit 12a, and an image portion of a cut-out range of a part of the original image is acquired as a resized image by the resizing unit 13g by the electronic zoom processing. This resized image is converted into the JPEG format to obtain a JPEG image. A JPEG image file 22 is composed of the JPEG image and the clipping range information 22a that is information on the clipping range.

  Note that the RAW image file 23 is composed of a RAW image of an original image that has not been compressed, a RAW image obtained by performing a predetermined correction process on the original image, or a RAW image obtained by irreversibly compressing the original image. . That is, the RAW image includes substantially the same image information of the same size as the original image output from the image sensor. It is possible to know which part of the RAW image of the file 23 corresponds to the JPEG image of the file 22 by the clipping range information included in the file 22.

  In the example of FIG. 2, the example in which the cutout range information is provided in the file 22 including the JPEG image has been described. However, it is apparent that the cutout range information may be provided in the file 23 including the RAW image.

  In addition, the control unit 11 controls the imaging unit 12a to perform time-lapse photography in which photography is performed at predetermined time intervals and reproduction is performed in a time shorter than the photography time. In addition, the control unit 11 can also provide the recording unit 14 with a plurality of captured images acquired from the imaging unit 12a acquired in the time-lapse photography mode and record them. Even during this time lapse shooting, the control unit 11 can provide both the RAW image and the JPEG image to the recording unit 14 and record them.

  Further, the control unit 11 can also generate a time-lapse moving image by arranging the time-lapse captured image recorded in the recording unit 14 on a timeline. The control unit 11 can store the generated time-lapse moving image in the recording unit 14 as, for example, a motion JPEG format AVI file.

  FIG. 3 is an explanatory diagram showing the AVI file 30 in this case. The AVI file 30 is a file in which a plurality of JPEG images J1, J2,... Obtained by time-lapse shooting are arranged along a predetermined timeline. In the present embodiment, the corresponding RAW images R1, R2,... Having a common original image are associated with each JPEG image by the association unit 13e. In FIG. 3, a pair of a corresponding JPEG image and RAW image K1, K2,.

  Although the description has been given mainly of the example of the JPEG image and the RAW image, the non-compressed (or losslessly compressed) information relating to the sound and the PCM sound and the MP3 sound, which are the lossy compression information, are similarly associated with each other.

  The signal processing unit 13 is provided with an effect processing unit 13i. The effect processing unit 13i can perform predetermined image effect processing or sound effect processing on the image and sound recorded in the recording unit 41.

  In the present embodiment, the control unit 11 can use a JPEG image as an output image among the captured images recorded in the recording unit 14. For example, in the example of FIG. 2, a JPEG image of the file 22 is used as an output image, and in the example of FIG. 3, an AVI file 30 including JPEG images J1, J2,.

  On the other hand, in the present embodiment, in the image effect processing in the effect processing unit 13i, a corresponding RAW image having a common original image can be selectively used instead of a JPEG image. Further, in the sound effect processing in the effect processing unit 13i, for example, the corresponding uncompressed sound having a common original sound can be selectively used instead of the MP3 sound. In the present embodiment, for this selection, the control unit 11 is provided with a switching condition determination unit 11a and a switching control unit 11b.

  The switching condition determination unit 11a determines whether a switching condition for selecting a RAW image instead of a JPEG image is satisfied, and whether a switching condition for selecting uncompressed audio instead of MP3 audio is satisfied. For example, the presence or absence of a switching operation by the user can be set as the switching condition. Further, for example, a predetermined scene may be determined from a captured image, and the fact that a predetermined scene has been reached may be used as the switching condition. Further, for example, the fact that the luminance distribution of the captured image is within a predetermined range may be set as the switching condition. The switching condition determination unit 11a may determine whether the switching condition has been satisfied by image analysis of an image acquired by the imaging unit 12a or audio analysis of audio acquired by the sound collection unit 12b. Good.

  When the switching condition determination unit 11a indicates that the switching condition is satisfied, the switching control unit 11b selectively reads a corresponding RAW image having a common original image from the recording unit 14 instead of the JPEG image, A corresponding uncompressed audio having a common original audio is selectively read from the recording unit 14 instead of the MP3 audio. Further, in the present embodiment, the control unit 11 converts the image after the image effect processing performed on the RAW image by the effect processing unit 13i into a JPEG file, and While updating the recording unit 14 as an image, the effect processing unit 13i converts the sound after the sound effect processing applied to the uncompressed sound into an MP3 file, and converts the sound into a corresponding MP3 sound having the same original sound. The recording unit 14 is updated.

  Next, the operation of the embodiment configured as described above will be described with reference to FIG. FIG. 4 is a flowchart for explaining the operation of the first embodiment.

  The control unit 11 of the imaging device 10 determines whether or not a power-off operation has been performed in step S1 of FIG. When a power-off operation is performed, the control unit 11 turns off the power in step S3 and returns the process to step S1.

  The control unit 11 displays a through image in step S2. That is, the control unit 11 gives the imaging signal acquired by the imaging unit 12a to the signal processing unit 13. The display control unit 13h causes the captured image after the signal processing to be displayed on the display screen of the display unit 16 as a through image. Thus, the through image acquired by the imaging unit 12a is displayed on the display screen of the display unit 16.

  The control unit 11 determines whether or not the photographing mode is designated in step S4. Now, it is assumed that the shooting mode is specified. In this case, in the next step S5, the control unit 11 determines whether recording of not only a JPEG image but also a RAW image is specified. When the recording of only the JPEG image is designated, the mode shifts to the normal photographing mode, and the photographed image is recorded in the recording unit 14 as the JPEG image according to the photographing operation.

  When the simultaneous recording of the JPEG image and the RAW image is specified, the control unit 11 determines whether or not a shooting control operation has been performed in the next step S6. When the user performs a setting operation of a photographing parameter, an image processing parameter, and the like, the control unit 11 changes the photographing mode and the parameter in step S7.

  Here, it is assumed that the user operates a release button (not shown) to instruct the start of shooting. When detecting the shooting start operation in step S8, the control unit 11 shifts the processing to step S9 and instructs the imaging unit 12a to perform imaging. The captured image from the imaging unit 12a is supplied to the signal processing unit 13. After performing predetermined signal processing, the signal processing unit 13 gives the processed image to the lossless compression processing unit 13c and the lossy compression processing unit 13d. The lossless compression processing unit 13c performs a lossless compression process or a non-compression process to generate a RAW image. The irreversible compression processing unit 13d generates a JPEG image by irreversible compression processing, for example, JPEG compression.

  The control unit 11 controls the recording unit 14 to simultaneously record the RAW image from the lossless compression processing unit 13c and the JPEG image from the lossy compression processing unit 13d. In this case, the associating unit 13e stores the RAW image and the JPEG image having the same original image in association with each other (Step S10). When the original image has been resized by the resizing unit 13g, the associating unit 13e records clipping range information indicating a clipping range in the resizing process in association with the RAW image and the JPEG image.

  In the present embodiment, it has been described that a RAW image and a JPEG image are simultaneously recorded. For example, for audio, irreversible compression information such as MP3 is always stored, and uncompressed only for a predetermined period. A PCM audio as information or an audio file of reversible compression information may be recorded. When the imaging device 10 is used for bird observation or the like, for example, it is unknown when a bird's cry can be picked up. Therefore, it is better to record irreversible compression information and non-compression information or reversible compression information at the same time. Good. In addition, it can be said that a specific effect can be immediately enjoyed by simple handling by processing such as compression, and that the handling is difficult before processing.

  Here, it is assumed that the user confirms the workmanship of the captured image obtained by shooting, and performs image effect processing depending on the workmanship. When the user specifies the playback mode, the control unit 11 shifts the processing from step S4, S11 to step S12, and reads and displays the JPEG image recorded in the recording unit 14. For example, in the example of FIG. 2, when one set of files 21 is specified by the user, the control unit 11 reads out the file 22 including the JPEG image in the one set of files 21 from the recording unit 14 and reproduces it. It controls the recording control unit 13f. The recording control unit 13f gives the read JPEG image to the irreversible compression processing unit 13d to perform expansion processing, and the display control unit 13h gives the expanded image to the display unit 16 for display. Thus, an image based on the JPEG image included in the file 22 is displayed on the display screen of the display unit 16.

  In the present embodiment, the switching condition determination unit 11a of the control unit 11 determines whether or not the original image satisfies the switching condition using the common RAW image instead of the JPEG image. In the example of FIG. 4, as the switching condition, it is detected whether or not the user has performed an operation for careful editing and confirmation.

  For example, it is assumed that a user performs predetermined image effect processing such as luminance adjustment on a captured image. Also in this case, image adjustment to some extent is possible by image processing on the JPEG image. However, the original image information is slightly lost in the JPEG image due to the irreversible compression process, and it is considered that the image effect process satisfying the user cannot always be given. Thus, in the present embodiment, when editing with such specialty, by using a RAW image instead of a JPEG image, it is possible to perform image effect processing that satisfies the user.

  That is, in step S13, the switching condition determination unit 11a of the control unit 11 determines whether or not there is an operation for discerning editing and confirmation by the user, and when the user performs this operation, the switching condition is satisfied. As a matter of course, the switching control unit 11b shifts the processing to step S14, and selects and reads a corresponding RAW image in which the JPEG image displayed in step S12 and the original image are common. As described above, the pre-processing data has a large degree of freedom when performing processing different from the original purpose, and it is easier to properly use the data as needed to meet diversifying user needs. That is, the collection unit that obtains the image or audio data, the processing unit that processes the image or audio data, and the data before and after the processing by the processing unit are associated with each other. A recording unit that records the data, a determination unit that determines whether a predetermined switching condition is satisfied, and reads and outputs the processed data from the recording unit, and that the predetermined switching condition is satisfied. If indicated, a corresponding portion of the data before the processing (for example, one obtained under the same imaging conditions such as the same timing and the same position) is read from the recording unit instead of the data after the processing. By providing an information terminal device having a control unit for outputting the information, a device capable of freely performing discrimination processing as needed can be provided.

  Note that, in the example of FIG. 4, an example is shown in which the JPEG image and the corresponding RAW image in which the original image is common are selectively read out only when the user desires careful editing and confirmation. Instead of the image, a corresponding RAW image may be selected and read.

  Further, the switching condition determination unit 11a may automatically determine the switching condition based on the captured image and the collected sound. For example, in the case where the imaging device 10 is used for bird observation or the like, it is better to reproduce high-quality sound such as uncompressed information for a portion such as a bird squeal included in the collected sound. In this case, at the time of reproducing the audio signal, the switching condition determining unit 11a determines the portion of the bird's singing or the like by voice processing, and automatically reproduces the irreversible compression information at the time of reproducing the relevant portion. The audio may be reproduced by reading the uncompressed information or the lossless compressed information. That is, as the switching condition, not only the time of editing by signal processing, but also an appropriate condition such as output, reproduction, display and the like can be set.

  In step S15, the effect processing unit 13i performs image processing (editing) based on a user operation or the like. The image after the image processing is converted into JPEG by the recording control unit 13f. When the recording of the editing result is instructed by the user, the control unit 11 gives the edited and JPEG-converted image to the recording unit 14 and records it (step S17). That is, when the RAW image corresponding to the JPEG image is edited, the recording unit 14 is updated with the JPEG image based on the edited RAW image. For example, the file 23 of the RAW image corresponding to the JPEG image of the file 22 in FIG. 2 is read out by careful editing, and if an editing process is performed on the image based on the RAW image, the file 23 is edited based on the edited JPEG image. Overwrite 22.

  As described above, in the present embodiment, when a predetermined switching condition at the time of the image effect processing or the like is satisfied, the lossless compression information such as the corresponding RAW image or the like is substituted for the irreversible compression information that is the JPEG image. The non-compressed information is read and processed. In the lossless compression information or the non-compression information, the image information of the original image is not missing as compared with the lossy compression information, and more effective image processing can be performed. Further, after the image effect processing, lossless compression information or non-compression information is converted into lossy compression information and recorded. When outputting an image, for example, a standardized JPEG image can be output. This makes it possible to record information that is easy to use in an external device and that has a reduced data amount.

  In the above description, audio is hardly mentioned, but it is clear that audio can be processed in the same manner as video.

(Second embodiment)
FIG. 5 is a block diagram showing a second embodiment of the present invention. In the present embodiment, the same function as the imaging device 10 of FIG. 1 is achieved by both the imaging device and the information terminal device. For example, in the case of performing time-lapse photography, it is often the case that the imaging device is fixed to a tripod, and photography is often performed. There is convenience in such a case. That is, the first embodiment and the second embodiment provide the same functions except for the operability.

  In the following description, an example of an apparatus capable of performing time-lapse imaging will be described. However, the present embodiment is not limited to this, and an apparatus that performs various other signal processing on recorded information is described. Is also applicable.

  As shown in FIG. 5, this embodiment includes an imaging device 40 and an information terminal device 50. The imaging device 40 can be configured by an imaging device such as a digital camera. The imaging device 40 has an imaging unit 42a configured by an imaging device such as a CCD or a CMOS sensor. The imaging unit 42a is provided with a lens (not shown), and an optical image of a subject is guided through the lens. The lens may have a function of zooming and focusing. The imaging unit 42a converts an optical image of a subject into an electric signal by an imaging element to obtain an imaging signal.

  The imaging device 40 is provided with a control unit 41, and the control unit 41 controls each unit of the imaging device 40. The control unit 41 may be configured by a processor such as a CPU (not shown) or may operate according to a program stored in a memory (not shown). The imaging device 40 has an operation unit 44 including a shutter button (not shown), a function button, various switches for setting a shooting mode, and the like, which are provided on a housing of the imaging device 40. The operation unit 44 receives a user operation and outputs an operation signal based on the user operation to the control unit 41. The control unit 41 generates a focus signal and a zoom signal based on an operation signal from the operation unit 44, and drives the lens of the imaging unit 42a to perform zooming, focusing, and the like. Further, the control unit 41 can output a drive signal of the image pickup device to the image pickup unit 42a to control a shutter speed, an exposure time, and the like.

  The image pickup device 40 is provided with a sound collecting unit 42b, and the sound collecting unit 42b is configured by a microphone (not shown) or the like, and converts surrounding sounds into audio signals. The control unit 41 can supply the sound collected by the sound collection unit 42b to the signal processing unit 48 based on the operation signal from the operation unit 44. For the sake of simplicity, the following description will focus on images, but audio can be processed in the same manner.

  The control unit 41 can read a captured image from the imaging unit 42a and supply the read image to the signal processing unit 48. The signal processing unit 48 performs predetermined signal processing, for example, color adjustment processing, matrix conversion processing, noise removal processing, and other various signal processing on the read captured image.

  The signal processing section 48 includes a lossless compression processing section 48a and a lossy compression processing section 48b. The reversible compression processing section 48a is controlled by the control section 41 to reversibly compress the captured image and output a RAW image. Note that the reversible compression processing unit 148a may be controlled by the control unit 41 to output a captured image as a RAW image without compression. The reversible compression processing section 48a is controlled by the control section 41, and outputs the collected voice without compression. Note that the RAW image may have been subjected to only minimum correction such as lens distortion correction and pixel defect correction in the signal processing unit 48.

  The irreversible compression processing unit 48b is controlled by the control unit 41 to irreversibly compress a captured image or a collected sound and outputs compressed data. For example, the lossy compression processing unit 48b performs JPEG compression on the RAW image and outputs a JPEG image in the JPEG format. For example, the irreversible compression processing unit 48b compresses non-compressed audio data such as PCM data using MP3 and outputs MP3 audio in MP3 format.

  The recording device 43 is provided in the imaging device 40. The control unit 41 can give the captured image subjected to the signal processing by the signal processing unit 48 to the recording unit 43 and record it. In the present embodiment, the control unit 41 can provide both the output of the lossless compression processing unit 48a and the output of the irreversible compression processing unit 48b to the recording unit 43 for recording. In this case, in the present embodiment, the control unit 41 associates the output of the lossless compression processing unit 48a and the output of the lossy compression processing unit 48b with respect to the image having the common original image and records the output in the recording unit 43. . For example, the control unit 41 records the output of the reversible compression processing unit 48a and the output of the irreversible compression processing unit 48b, which share the original image, with file names that differ only in the extension.

  Although only one recording unit 43 is shown in FIG. 5, for example, a double-slot card interface capable of simultaneously recording data on two memory cards may be employed as the recording unit 43. In this case, the output of the lossless compression processing section 48a may be recorded via one of the double slots, and the output of the lossy compression processing section 48b may be recorded via the other of the double slots.

  Further, the signal processing unit 48 may have the same function as the resizing unit 13g and the effect processing unit 13i in FIG. For example, the signal processing unit 48 can perform a predetermined image effect process on an image reproduced from the recording unit 43. In addition, the signal processing unit 48 can perform a resizing process on the captured image. When the signal processing unit 48 performs the resizing process, the control unit 41 adds the cutout range information to one of the non-compressed or lossless-compressed information and the lossy-compressed information at the time of association recording. I have. Thus, in the recording unit 43, for example, as shown in FIGS. 2 and 3, a file of a set of non-compressed or lossless compressed information and irreversible compressed information is recorded.

  Further, the control unit 41 has functions of the switching condition determination unit 11a and the switching control unit 11b of the control unit 11 of FIG. That is, the control unit 11 sets a predetermined switching condition, determines whether or not the switching condition is satisfied, and when the switching condition is satisfied, the recording unit 43 replaces the irreversible compression information with the original image, The non-compressed information or the reversible compressed information to be read can be read and supplied to the signal processing unit 48 and the like.

  Note that the control unit 41 can also control the imaging unit 42a to perform shooting at a predetermined time interval and perform time-lapse shooting in which playback is performed in a time shorter than the shooting time. Further, the signal processing unit 48 is also controlled by the control unit 41 and can arrange a captured image recorded in the recording unit 43 on a timeline to generate a time-lapse moving image.

  In the present embodiment, the imaging device 40 can be remotely controlled by the information terminal device 50. A communication unit 46 is provided in the imaging device 40 for this remote operation. The communication unit 52 is also provided in the information terminal device 50. The communication unit 46 is controlled by the communication control unit 47 to send and receive information to and from the communication unit 52 of the information terminal device 50 via a predetermined transmission path. The control unit 41 is configured such that commands and parameters are transferred from the information terminal device 50 via the communication unit 46, and each unit can be controlled in accordance with the transferred commands and parameters.

  The communication control unit 47 is controlled by the control unit 41 to transmit an image captured by the imaging device 40 to the communication unit 52 of the information terminal device 50 via the communication unit 46.

  The information terminal device 50 can be configured by a smartphone, a tablet PC, or the like. The information terminal device 50 has a control unit 51, and the control unit 51 controls each unit of the information terminal device 50. The control unit 51 may be configured by a processor such as a CPU (not shown) or may operate according to a program stored in a memory (not shown). The information terminal device 50 has an operation unit 53 including a switch (not shown) and the like provided on a housing of the information terminal device 50. The operation unit 53 can be constituted by a software keyboard or the like obtained by a program executed by the control unit 51, or can be constituted by a touch panel (not shown) provided on a display unit 57 described later.

  The operation unit 53 receives a user operation and outputs an operation signal based on the user operation to the control unit 51. The control unit 51 controls each unit based on an operation signal from the operation unit 53.

  The information terminal device 50 is provided with a recording unit 54, and the recording unit 54 can record information given from the control unit 51. For example, the image and various parameters transmitted from the imaging device 40 are given to the recording unit 54 and recorded. The recording unit 54 also records information based on a user operation of the operation unit 53 and information initialized by the control unit 51.

  The information processing device 50 is provided with a signal processing unit 55. The signal processing unit 55 performs various signal processings under the control of the control unit 51.

  The display control unit 56 executes various processes related to display. The display control unit 56 is controlled by the control unit 51 to control the display on the display unit 57. The display unit 57 has a display screen such as an LCD, and displays an image given from the display control unit 56. For example, the control unit 51 controls the display control unit 56 to display each captured image used for generating the time-lapse moving image on the display screen of the display unit 57 and change the time-lapse moving image to a desired moving image. Therefore, a display (hereinafter, referred to as an edit display) for performing a predetermined image effect process on the captured image obtained by the time-lapse imaging of the imaging device 40 can be displayed.

  On the display screen of the display unit 57, a touch panel as the operation unit 53 may be provided. The touch panel can generate an operation signal corresponding to a position on the display screen pointed by a user with a finger. This operation signal is supplied to the control unit 51. Accordingly, when the user touches or slides on the display screen, the control unit 51 can detect the operation. That is, the control unit 51 can detect various operations such as a user's touch position, an operation of closing and separating fingers (pinch operation), a slide operation, a position reached by a slide operation, a slide direction, and a touching period. It is possible to execute a process corresponding to a user operation.

  The control unit 51 can transmit a control signal based on an operation signal generated by a user operation on the edit display to the control unit 41 of the imaging device 40 via the communication unit 52. In this case, the control unit 41 of the imaging device 40 controls each unit including the signal processing unit 48 based on a control signal from the information terminal device 50.

  Next, an operation of the embodiment configured as described above will be described with reference to FIGS. 6 to 9 and FIGS. 10A to 10I. 6 and 7 are flowcharts showing the operation of the imaging device 40 or the information terminal device 50, respectively. In FIG. 6, the same steps as those in FIG. 4 are denoted by the same reference numerals, and description thereof is omitted. FIG. 8 is an explanatory diagram showing an example of the appearance of the imaging device 40 and a display on a display screen. 9A and 9B are explanatory diagrams for explaining a time-lapse moving image. 10A to 10I are explanatory diagrams illustrating an example of an image displayed on the display screen of the display unit 57 arranged on the front side of the housing 50a of the information terminal device 50.

  FIG. 8 shows an example of the appearance of the imaging device 40 and a display on a display screen. In the example of FIG. 8, a display unit is provided on the back side of the imaging device 40, and a display screen 49a constituting the display unit is exposed on the back surface of the housing 40a of the imaging device 40. A lens barrel (not shown) is provided on the front surface of the housing 40a, and the lens unit of the imaging unit 42a is housed in the lens barrel. Further, the imaging device 40 may be provided with a finder section 61 on the upper surface of the housing 40a. Further, an eye sensor 63 for detecting that the user has brought the eyes close to the finder section 61 may be provided near the finder section 61. The eye sensor 63 detects that the user has brought his / her eyes close to the finder section 61 and outputs a detection result to the control section 41. In this case, the control unit 41 may be configured to be able to switch the output destination of the image between the display screen 49a of the display unit and the LCD constituting the finder unit 61 according to the detection result of the eye sensor 63. .

  A shutter button 64 and two dial operation units 65 and 66 are provided on the upper surface of the housing 40a as the operation unit 44, and a PICT button 68 is provided on the rear surface of the housing 40a. While holding the imaging device 40, the user can not only press the shutter button 64 with the index finger while looking through the finder unit 61, for example, but also operate the dial operation unit 66 with the right thumb and the dial operation with the index finger. The part 65 can be easily operated. By operating the PICT button 68, a predetermined effect process can be instructed. The signal processing unit 48 performs an image effect process on a captured image or the like according to a user operation.

  Note that the example of FIG. 8 illustrates an example in which the user operates the operation unit 53 of the information terminal device 50 to instruct reproduction of a time-lapse moving image. A time lapse moving image 71 is displayed on the display screen 49a.

  In step S31 in FIG. 7, the control unit 51 of the information terminal device 50 controls the display control unit 56 to display a function selection display. FIG. 10A shows an example of a function selection display displayed on the display screen 57a of the display unit 57 of the information terminal device 50. In the example of FIG. 10A, a selection button 31a for selecting an SNS (social network service) function, an external camera button 31b for selecting an external camera function, and a mail button for selecting a mail function are displayed on the display screen 57a. 31c and a telephone button 31d for selecting a telephone function are displayed.

  Here, it is assumed that the user touches the external camera button 31b to select the external camera function, as indicated by the thick frame in FIG. 10A. The control unit 51 determines whether or not the external camera function has been selected in step S32. If the external camera function has not been selected, the control unit 51 determines in step S51 whether or not there is a request from the external camera. The control unit 51 determines whether the request is for activating another function when the request is satisfied (step S52), and activates another function when the request is for activating another function (step S52). Step S53). When there is no request from the external camera, the control unit 51 receives an image from the external camera in step S54, displays the received image, and performs temporary recording (step S55). In step S56, the control unit 51 determines whether or not an editing or confirmation operation described below has been performed by the user. If the editing or confirmation operation has been performed, the process proceeds to step S61. It returns to step S31.

  When the external camera function is selected, the control unit 51 shifts the processing from step S32 to step S33, and determines whether an external camera has already been designated. If an external camera has not been designated, in step S34, a camera designation operation such as wireless connection setting, ID, and password setting is accepted, and the designated information is recorded in the recording unit 54.

  If the control unit 51 determines in step S33 that an external camera has already been specified, in step S35, the control unit 51 causes the communication unit 52 to establish communication with the communication unit 46 of the imaging device 40 and specifies the external camera. Access the imaging device 40, which is a camera.

  On the other hand, the control unit 41 of the imaging device 40 is in a communication standby state in step S21 of FIG. The control unit 41 determines whether or not power-off communication has occurred in step S1. When there is power-off communication, the control unit 41 turns off the power in step S3 and returns the process to step S21.

  In step S22, the control unit 41 determines whether the destination of the live view transmission has been set. If the setting has been completed, the control unit 41 determines whether or not the shooting mode has been designated in step S4. If the shooting mode has been set, the control unit 41 transmits a through image to a communication partner (step S26). ). That is, the control unit 41 provides the image signal acquired by the imaging unit 42a to the signal processing unit 48, and transmits the captured image after the signal processing from the communication unit 46 to the information terminal device 50 as a through image.

  When the control unit 51 of the information terminal device 50 determines that the communication has been established in step S36, the communication unit 52 receives the through image. Then, the control unit 51 proceeds from step S38 to step S39, and gives the live view image received via the communication unit 52 to the display control unit 56. Thus, the through image acquired by the imaging device 40 is displayed on the display screen 57a of the display unit 57. If it is determined in step S36 that communication has not been established, the control unit 51 issues a warning such as a warning display after a predetermined time has elapsed (step S37).

  In the imaging device 40, when the partner of the through image transmission has not been set, the control unit 41 determines in step S23 whether or not the operation of setting the partner of the through image transmission has been performed. When the setting operation of the through image transmission partner is performed, the control unit 41 performs the setting procedure in step S24, records the setting information in the recording unit 43 (step S25), and shifts the processing to step S21. return.

  In step S6, the control unit 41 of the imaging device 40 determines whether communication for imaging control has been performed. When a photographing parameter, a parameter for image processing, and the like are received from the information terminal device 50, the control unit 41 changes the photographing mode and the parameter in step S7. In step S7, the information terminal device 50 may be able to set parameters relating to time-lapse photography.

  Here, it is assumed that the user performs an operation of starting shooting. The shooting operation can be performed not only by the operation unit 44 of the imaging device 40 but also by the operation unit 53 of the information terminal device 50. For example, the control unit 51 of the information terminal device 50 can control the display control unit 56 to display the mode selection screen shown in FIG. 10B. In the example of FIG. 10B, a communication setting button 32a for selecting a communication setting mode, a shooting button 32b for selecting a shooting mode, a playback button 32c for selecting a playback mode, and a function selection screen are displayed on the display screen 57a. A return button 32d for returning to is displayed.

  Here, it is assumed that the user touches the shooting button 32b to select a shooting mode. The control unit 51 determines whether or not an operation for imaging control has been performed in step S44, and determines whether or not a shooting operation has been performed in step S46. When an operation for designating photographing is performed, the control unit 51 shifts the processing to step S <b> 47 and transmits the operation result to the imaging device 40.

  When receiving the control signal based on the user's shooting start operation in step S8, the control unit 41 of the imaging device 40 shifts the processing to step S9 and instructs the imaging unit 12a to perform imaging. The captured image from the imaging unit 42a is supplied to the signal processing unit 48. After performing predetermined signal processing, the signal processing unit 48 gives the processed image to the lossless compression processing unit 48a and the lossy compression processing unit 48b. The lossless compression processing unit 48a performs a lossless compression process or a non-compression process to generate a RAW image. The irreversible compression processing unit 48b generates a JPEG image by, for example, JPEG compression, which is irreversible compression processing.

  The control unit 41 controls the recording unit 43 to simultaneously record the RAW image from the lossless compression processing unit 48a and the JPEG image from the lossy compression processing unit 48b. In this case, the control unit 41 stores the RAW image and the JPEG image having the same original image in association with each other (Step S10). When the original image has been resized, the control unit 41 records cutout range information indicating a cutout range in the resize processing in association with the RAW image and the JPEG image.

  The control unit 41 transmits the recorded photographing result to the information terminal device 50 (Step S27). When the photographing operation is to instruct the normal photographing, for example, the file 22 including the JPEG image of FIG. 2 is output. When the shooting operation is to instruct time-lapse shooting, for example, each of the JPEG images J1, J2,... Shown in FIG. 3 is output. When a time-lapse movie is generated by arranging the captured images obtained by time-lapse photography on a timeline, the control unit 41 stores the time-lapse movie in the recording unit 43 as a motion JPEG format AVI file. In this case, for example, the AVI file 30 shown in FIG. 3 may be transmitted to the information terminal device 50.

  Now, it is assumed that time-lapse photography is performed by the imaging device 40, and each captured image obtained by the time-lapse photography is transferred to the information terminal device 50. Here, it is assumed that the user wishes to confirm the quality of the time-lapse shooting and to edit each captured image used for the time-lapse moving image.

  FIG. 10C shows a setting menu screen for checking and editing the captured image displayed on the display screen 57a of the information terminal device 50. In the example of FIG. 10C, on the display screen 57a, a captured image display 33a, a time lapse button 33b for selecting to confirm and edit a time lapse moving image, and a selection button 33c for selecting an image to be edited are provided. , A processing button 33d for instructing processing of the selected image capturing, and a return button 33e are displayed.

  Here, it is assumed that the user wants to confirm and edit the time-lapse moving image. In this case, the user operates the time lapse button 33b.

  FIG. 10D shows a setting menu screen for checking and editing the time-lapse moving image displayed on the display screen 57a of the information terminal device 50 in this case. In the example of FIG. 10D, on the display screen 57a, a time-lapse moving image display 34a, a playback button 34b for instructing rewind, reproduction, and fast-forward of the time-lapse moving image, and for editing and confirming each captured image by time-lapse shooting. An edit button 34c and a return button 34d are displayed.

  Here, it is assumed that the user wants to edit the captured image used for the time-lapse moving image with reference to the time-lapse moving image display 34a. In this case, the edit button 34c is operated. Then, a menu screen for instructing editing or confirmation is displayed.

  FIG. 10E shows this menu screen. In the example of FIG. 10E, on the display screen 57a, a time-lapse display 35a, a confirmation button 35b for confirming each captured image by time-lapse photography, and an edit for editing (discussed editing) each captured image by time-lapse photography. A button 35c and a return button 35d are displayed. Here, it is assumed that the user wants to select and edit the captured image used for the time-lapse moving image. In this case, the user operates the edit button 35c. Then, a special editing screen is displayed.

  FIG. 10F shows this particular editing screen. In the example of FIG. 10F, on the display screen 57a, an image display 36a of each captured image obtained by time-lapse shooting, a range selection button 36b for designating a range of the captured image to be edited, and an instruction for adjusting image quality of the captured image are provided. An image quality adjustment button 36c and a return button 36d are displayed.

  In the example of FIG. 10F, the image display 36a is a thumbnail display of each captured image obtained by time-lapse shooting, and shows an example in which six captured images obtained by one time-lapse shooting are displayed. By sliding the image display 36a, for example, left and right, each captured image (thumbnail image) is sequentially displayed according to the shooting order of the time-lapse shooting.

  Here, it is assumed that the user adjusts the image quality of the captured image used for the time-lapse moving image. In this case, the user operates the image quality adjustment button 36c. Then, an image quality adjustment screen is displayed.

  FIG. 10G shows this image quality adjustment screen. In the example of FIG. 10G, on the display screen 57a, an image display 37a of each captured image obtained by time-lapse shooting, a gentle button 37b for making a time-lapse moving image smoothly change as editing content, and a gradation expression of the captured image are corrected. A tone adjustment button 37c and a return button 36d are displayed.

  Also in FIG. 10G, the image display 37a is a thumbnail display of each captured image obtained by time-lapse shooting, and shows an example in which six captured images obtained by one time-lapse shooting are displayed. By sliding the image display 37a to the left and right, for example, each captured image (thumbnail image) is sequentially displayed in the order of time-lapse photography.

  In addition, in the example of FIG. 10G, the captured images obtained by the time-lapse photography include not only a mode for editing each captured image one by one, but also a mode for continuously editing a plurality of captured images in a certain range. When the tone adjustment button 37c is operated after performing a touch operation on one captured image in the image display 37a of FIG. 10G, the captured image subjected to the touch operation becomes an edit target image, and editing work becomes possible. When a touch operation is performed on one captured image in the image display 37a and then a touch operation is performed on another captured image, the captured images specified by the two touch operations and the captured images while the captured images are captured are captured. The captured image obtained by the operation is selected, and by operating the gentle button 37b in this state, a series of a plurality of captured images selected by two touch operations becomes the edit target image.

  Also in the present embodiment, a RAW image is used for an image to be edited instead of a JPEG image. Now, for example, it is assumed that JPEG images J1, J2,... And RAW images R1, R2,... Shown in FIGS. 9A and 9B are stored in the recording unit 43 of the imaging device 40 as time-lapse captured images. . These RAW images R1, R2,... Are sequentially obtained by time-lapse shooting. As shown in FIG. 3, the AVI file 30 of the time-lapse moving image is formed by the JPEG images J1, J2,. It constitutes.

  The JPEG images J1, J2, ... and the RAW images R1, R2, ... are time-lapse shots of a landscape including a mountain in the center, a relatively dark forest before the mountain, and a relatively bright sky behind the mountain, It shows the change of the landscape from morning to night. However, in a time-lapse moving image using JPEG images J1, J2,..., It is assumed that, for example, the luminance change of the JPEG images J3 to J6 is not smooth, and the state of the change in the scenery is not what the user desires. In this case, the user operates the gentle button 37b after touching the thumbnail image corresponding to the JPEG images J3 and J6 in the image display 37a of FIG. 10G.

  In step S48, the control unit 51 determines whether or not the image recorded in the recording unit 43 has been transferred from the imaging device 40 and the reproduction thereof has been instructed. The process moves to step S49. The control unit 51 generates a control signal for selecting a specified image by a touch operation on a thumbnail image corresponding to the JPEG images J3 and J6 on the display screen 57a. Further, when an operation for editing / confirmation is performed in step S50, that is, the operation of the gentle button 37b is performed, the control unit 51 determines that the predetermined switching condition is satisfied, and in step S61, the control unit 51 transmits A control signal based on these operations is transmitted.

  Accordingly, the control unit 41 of the imaging device 40 reads the RAW images R3 to R6 corresponding to the JPEG images J3 to J6 from the recording unit 43 and outputs the RAW images R3 to R6 to the signal processing unit 48 (step S14). Further, the control unit 41 controls the signal processing unit 48 to use the read RAW images R3 to R6 and the JPEG images J1, J2, J7, and J8 used for the time-lapse moving image to generate portions of the JPEG images J3 to J6. Generates a time-lapse moving image in which the luminance changes gradually. The signal processing unit 48 performs correction for adjusting the average luminance of each of the RAW images J3 to J6 based on information on a change in the luminance distribution of the RAW images J3 to J6 and the JPEG images before and after the RAW images (step S15). In this manner, the signal processing unit 48 performs the luminance distribution correction processing on the RAW images R3 to R6, and thereby the time lapse moving image in which the luminance changes smoothly according to the JPEG images J1 and J2, the corrected RAW images R3 to R6 and the JPEG images J7 and J8. Generate

  The signal processing unit 48 converts the corrected RAW images R3 to R6 into JPEG, and overwrites and records them in the recording unit 43 as JPEG images J3 to J6 (step S17). The control unit 41 may transmit the updated JPEG images J3 to J6 to the information terminal device 50.

  It is assumed that the user operates the confirmation button 35b while the menu screen of FIG. 10E is displayed. Then, a confirmation screen shown in FIG. 10H is displayed.

  In the example of FIG. 10H, on the display screen 57a, similarly to FIG. 10E, a time-lapse display 39, a confirmation button 35b for confirming each captured image by time-lapse photography, and editing of each captured image by time-lapse photography. , An edit button 35c and a return button 35d are displayed. The time lapse display 39 on the confirmation screen is displayed using a RAW image instead of a JPEG image.

  That is, when the confirmation button 35b is operated, the control unit 51 requests the imaging device 40 to transfer a RAW image corresponding to the JPEG image used for the time lapse display 35a. The control unit 41 of the imaging device 40 reads out the requested RAW image from the recording unit 43 and transmits it to the information terminal device 50. The control unit 51 of the information terminal device 50 generates a thumbnail image based on the received RAW image and displays the thumbnail image as the time lapse display 39.

  If the electronic zoom processing is performed at the time of shooting, the JPEG image is a part of the original image. On the other hand, the RAW image includes all image portions of the original image. Therefore, when creating a thumbnail image based on a RAW image, the control unit 51 determines an image range to be displayed by using clipping range information included in a JPEG image file.

  In the time lapse display 39, the user can check all image portions of the original image. In the example of FIG. 10H, the time lapse display 39 includes a mountain image 39a at the center of the original image and a forest image 39b at the near side. Here, it is assumed that the user slides the time lapse display 39 with the finger 38. Then, the control unit 51 changes the cutout range of the original image according to the slide operation of the user, and generates a thumbnail image of the RAW image.

  FIG. 10I shows a confirmation screen in this case. The time lapse display 39 is obtained by moving the image of FIG. 10H to the upper right side of the screen. Contains a lake image 39c. By referring to the time lapse display 39, the user can confirm what image portion is included in the original image.

  As described above, in the present embodiment, similarly to the first embodiment, for example, when a predetermined switching condition such as editing or display is satisfied, a corresponding RAW image is adopted instead of a JPEG image. Even if the image desired by the user cannot be obtained by the signal processing on the JPEG image, by performing the signal processing on the RAW image substantially retaining the image information of the original image, the user can easily obtain the desired image.

  For example, taking time-lapse photography as an example, in time-lapse photography, it can be said that the user enjoys changing images during a predetermined photography time. In this case, even if each image is an image having no problem, in a time-lapse moving image, for example, an image during a certain period may not have a satisfactory change in luminance. In such a case, correction by image processing may be performed. Even in this case, correction for a JPEG image is possible. However, in general, the amount of information of a JPEG image is smaller than that of a RAW image before compression, and it is considered that an ideal image is easily obtained by performing image processing on the RAW image before compression. Can be In the present embodiment, for example, a case where a user intends to perform image processing on a time-lapse moving image is set as a switching condition, and image processing is performed using a RAW image instead of a JPEG image in a section to be edited. Thereby, a desired image processing effect can be obtained. Then, the processed RAW image is converted into a JPEG image and replaced with the original JPEG image. Thereby, it is possible to obtain a time-lapse moving image in which image editing desired by the user has been performed.

  Note that, in the above-described embodiment, an example in which non-compression or lossless compression information is used in place of lossy compression information when it is determined that the switching condition is satisfied when the user performs a discretionary button operation for editing. Although shown, the switching condition may be automatically determined by analyzing images and sounds. For example, in the above-described time-lapse moving image, a gradual change in luminance may be expected. The change of the average luminance of each captured image forming the time-lapse moving image is determined, and the automatic determination of the switching condition is possible as a condition for satisfying the switching condition that it is detected that a gradual change in the luminance is not obtained. Further, in this case, automatic image processing can be performed so that a change in luminance is not caused.

  In the present embodiment, by using uncompressed or losslessly compressed information in the image processing in this case, it is possible to automatically obtain a time-lapse moving image in which an image change desired by the user can be obtained. Time lapse can be used to write fast forwards and quickly confirm the process, while raw data, uncompressed or losslessly compressed information is useful for discerning observation and appreciation. For example, among specialists, there are users who can perform observation, diagnosis, and checking more easily by performing specific signal processing, and can respond to such diversification and special needs.

  Note that it is obvious that the same processing as the editing processing for the time-lapse moving image in the second embodiment can also be realized in the imaging device in the first embodiment.

  Further, in the above-described embodiment, the description has been made mainly on the image, but the present invention can be similarly applied to the sound.

  The present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the gist thereof at the stage of implementation. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the above embodiments. For example, some components of all the components shown in the embodiment may be deleted.

For example, as a system for obtaining information such as a user's favorite image and sound, the system is not limited to a consumer imaging system, but may be used for obtaining an image for observation, diagnosis, and examination by a user. It can be applied to other systems. The process for obtaining the user's favorite image may be expressed as an adjustment process on a report or a moving image for evidence. For medical use, it can be applied to endoscopes, etc., and it can be used for digesting only the necessary treatment process and necessary diseased part.For scientific use, it is most suitable for cell culture. It can be utilized when creating a moving image or the like in which the feature has appeared. It is possible to control observation and photographing while confirming a change in a moving image.
In other words, according to the present invention, by recording information before and after processing, such as recording both non-compressed information or both lossless-compressed information and irreversible-compressed information, and selecting these information according to predetermined switching conditions For example, an easy-to-use image output or audio output with high image quality or high sound quality can be obtained, and the data can be reproduced according to needs.
Such a concept can be applied not only to compression of image frames but also to compression between frames. Also, in the case of audio, if the presence or absence and type of processing are different, such as the presence or absence of compression, sampling characteristics, dynamic range characteristics, and presence or absence of noise cancellation, different characteristics appear before and after processing. In the simplest examples of images, there are many examples in which the usability differs between the original data and the processed data, such as a colored image and a black-and-white image, and in the processing of images and sounds, various examples can be given without being limited to compression. . In other words, the configuration is effective for the purpose of continuously acquiring data and quickly diagnosing and observing the data, and for the purpose of carefully examining specific data therefrom. Therefore, application to medical devices, observation devices, and inspection devices is effective. In some applications, such as a robot or an inspection device, the imaging result is repeatedly output while changing the position, and the inspection result is output. However, for each position, both the image data before and after the processing are recorded in association with each other. In this case, when it is indicated that the predetermined switching condition is satisfied, the control unit reads and outputs a corresponding portion of the pre-processing information from the recording unit instead of the post-processing information. By using the information terminal device, the observation device, and the observation method, it is possible to perform both the discerning inspection and the careful inspection.

  It should be noted that even if the operation flow in the claims, the specification, and the drawings is described using “first”, “next”, etc. for convenience, it is essential that the operation be performed in this order. It does not mean. In addition, it is needless to say that each step constituting the operation flow can be omitted as appropriate for portions that do not affect the essence of the invention.

  In addition, among the techniques described here, the control and functions mainly described in the flowchart can be set in many cases by a program, and the above-described control and functions can be realized by reading and executing the program by a computer. it can. The program can be recorded or stored in whole or in part on a portable medium such as a nonvolatile memory such as a flexible disk or a CD-ROM, or a storage medium such as a hard disk or a volatile memory as a computer program product. It can be distributed or provided at the time of product shipment or via a portable medium or a communication line. The user downloads the program via a communication network and installs the program on a computer, or installs the program on a computer from a recording medium, so that the information terminal device, the imaging device, and the information processing method according to the present embodiment can be easily implemented. Can be realized.

    Reference Signs List 10: imaging device, 11: control unit, 11a: switching condition determination unit, 11b: switching control unit, 12a: imaging unit, 12b: sound collection unit, 13: signal processing unit, 13a: shooting control unit, 13b: sound collection Control unit, 13c lossless compression processing unit, 13d lossy compression processing unit, 13e association unit, 13f recording control unit, 13g resize unit, 13h display control unit, 13i effect processing unit, 14 recording unit , 16 ... display unit, 17 ... operation unit.

Claims (2)

An imaging unit for acquiring an image,
A processing unit for processing the image;
A recording unit that records both the image before processing and the image after processing by the processing unit in association with each other,
A determining unit that determines whether a predetermined switching condition is satisfied;
The processed image is read from the recording unit and output, and when it is indicated that the predetermined switching condition is satisfied, the image before processing is replaced by the recording unit in place of the processed image. A control unit that reads and outputs the
A signal processing unit for performing signal processing;
With
The imaging unit is capable of performing time-lapse imaging for repeatedly imaging,
The recording unit records both the pre-processing and post-processing images obtained by time-lapse photography,
The signal processing unit generates a time-lapse movie using the processed image obtained by the time-lapse shooting ,
The switching condition is satisfied when the signal processing unit attempts to perform image processing on the time-lapse moving image,
The image pickup apparatus, wherein the control unit supplies an image before the processing to the signal processing unit instead of the processed image as an image to be processed during image processing on the time-lapse moving image. 2. The imaging apparatus according to claim 1 , wherein the recording unit records the cut-out range information of the processed image with respect to the image acquired by the imaging unit in association with the image before the processing.