WO2018043585A1

WO2018043585A1 - Endoscope device, information processing device, and program

Info

Publication number: WO2018043585A1
Application number: PCT/JP2017/031220
Authority: WO
Inventors: 渡辺　浩之; 岳仁栗原
Original assignee: Hoya株式会社
Priority date: 2016-08-31
Filing date: 2017-08-30
Publication date: 2018-03-08
Also published as: JPWO2018043585A1; CN109310300A

Abstract

To allow a desired scene to be easily reproduced and displayed in an endoscope device. In an endoscope device (100) that can record, reproduce, and display moving images, when an operator carries out still picture saving operation during the recording of a moving image, a frame image (chapter image) according to a scene during the operation is recorded in a moving image file as metadata. When a moving image reproducing mode is set, an editing screen that allows a moving image reproduction starting scene to be selected is displayed on an LCD (136). At the time, metadata corresponding to a chapter image is extracted from a moving image file and displayed as a list on the editing screen. When the operator selects a desired chapter image, the reproduction of the moving image is started from the scene of the chapter image.

Description

Endoscope apparatus, information processing apparatus, and program

The present invention relates to an endoscope apparatus, an information processing apparatus, and a program for imaging a body organ or the like and performing a treatment or the like as necessary, and particularly relates to a moving image recording and reproducing process.

In the endoscope apparatus, it is possible to record an image taken by a scope as a moving image during observation or treatment / surgery. When the operator presses a button provided on the scope operation unit or the front panel of the processor, the moving image data is sent to an external recording device connected to the processor or a memory inside the processor and recorded as a moving image file (for example, Patent Document 1).

When moving image recording is performed over the entire period from the start of work such as surgery to the end of work, it is desirable to efficiently extract a desired scene during moving image playback. For this reason, a moving image portion of a specific scene is extracted from the recorded moving image data, and can be reproduced and displayed (see Patent Document 2).

Specifically, when a change occurs in the captured image, such as when a scope is inserted from the outside into the body, the change is detected, an edit point is added to the moving image recording data as meta information, and the image type, The usage status of peripheral devices such as an electric knife device is recorded in association with meta information.

In the editing screen before moving image playback, an item indicating the type of image and an item indicating the usage status of the peripheral device are displayed on the menu screen together with a series of thumbnail (reduced) images to which editing points are given. When the operator selects and sets a scene item to be reproduced, an edit point image of the scene is displayed, and the moving image portion is reproduced by performing an execution operation.

Japanese Patent Laid-Open No. 2015-85029 JP 2016-7444 A

When extracting a scene that automatically starts playback (separation) on the endoscope apparatus side, the operator does not know which scene was extracted during moving image recording. For this reason, it is difficult to correlate items displayed as character information with a series of extracted video scenes during moving image playback, and it is difficult to intuitively understand which scene is the extracted scene. Difficult to find out.

Therefore, it is required that an operator easily finds a desired scene and reproduces and displays a moving image.

An endoscope apparatus according to the present invention selects a recording processing unit that records a real-time moving image in a memory as a moving image file, a reproduction processing unit that reads the moving image file and reproduces and displays the moving image on a monitor, and selects a reproduction start scene An editing processing unit that displays a possible editing screen on the display unit, and the recording processing unit converts a frame image (hereinafter referred to as a chapter image) according to an operator-specified scene into a moving image file as metadata during moving image recording. The editing processing unit extracts the metadata corresponding to the chapter image, displays the chapter image on the editing screen, and the reproduction processing unit from the chapter image scene selected by the operator or the scene before and after the chapter image, Start playback of moving images. Here, the “front and back scenes” represent frame images in a range that can be regarded as substantially the same subject, representing frame images that are a front or back of a predetermined number of frames (for example, several frames) from the chapter image.

For example, the recording processing unit can save a chapter image corresponding to a scene at the time of still image saving operation by an operator as a metadata in a moving image file. The edit processing unit can extract a frame image in the metadata as a still image file.

For example, the recording processing unit may store at least one of the scope type and the image processing setting content as metadata together with the chapter image in the moving image file. The editing processing unit displays the top frame image corresponding to the top scene in the recorded moving image data together with the chapter image, and when the playback processing unit selects the top frame image by the operator, the top frame is displayed. It is possible to start playback of a moving image from an image.

For example, the processor of the endoscope apparatus can be connected to an external peripheral device capable of displaying an endoscope work-related image and outputting a video signal. The recording processing unit can store the endoscope work-related still image displayed during the still image storage operation by the operator in a moving image file as metadata. The reproduction processing unit may extract metadata corresponding to the endoscope work-related still image, and display the endoscope-related still image during moving image reproduction.

On the other hand, the processor of the endoscope apparatus can be connected to the filing apparatus. The playback processing unit can simultaneously display real-time moving images and recorded images (including both still images and moving images) recorded on the filing device, and the recording processing unit can play back when still images are stored by an operator. A playback frame image corresponding to a scene can be stored in a moving image file as metadata.

In another aspect of the present invention, a recording / playback method for an endoscope apparatus records a real-time moving image as a moving image file in a memory, reads the moving image file, displays the moving image on a monitor, and selects a reproduction start scene. This is a method of displaying a possible editing screen on the display unit. During moving image recording, a frame image (hereinafter referred to as a chapter image) corresponding to an operator-specified scene is saved as a metadata in a moving image file, and according to the chapter image. The metadata is extracted, the chapter image is displayed on the editing screen, and the reproduction of the moving image is started from the chapter image scene selected by the operator or the scene before and after the chapter image.

On the other hand, the processor of the endoscope apparatus derived from the viewpoint of metadata recording can be connected to an external peripheral device that displays an endoscope work-related video and can output a video signal, and a real-time moving image is a moving image file. A recording processing unit for recording in the memory. Then, the recording processing unit stores the endoscope work-related still image displayed during the still image storing operation by the operator during moving image recording in a moving image file as metadata. A processor of an endoscopic device having similar technical characteristics can be connected to a filing device, and a recording processing unit that records a real-time moving image in a memory as a moving image file, and a reproduction processing unit includes a real-time moving image, The recording image recorded on the filing device can be displayed simultaneously, and the recording processing unit stores a playback frame image corresponding to a playback scene at the time of still image storage operation by the operator as a metadata in a moving image file.

An information processing apparatus according to another aspect of the present invention provides a first acquisition unit that acquires a moving image including a moving image and metadata about a chapter image cut out at a predetermined time during shooting of the moving image, A display unit that displays a list of the chapter images extracted from the moving image file acquired by the first acquisition unit, a reception unit that receives a selection from the chapter images displayed by the display unit as a list, and the reception unit that receives the selection And a playback unit that plays back the moving image based on the time corresponding to the chapter image.

In the information processing apparatus according to another aspect of the present invention, the moving image is an image taken using an endoscope, and the first acquisition unit in the first half is operated by a button provided on the endoscope A moving image file including metadata about the chapter image cut out at the time is acquired.

The information processing apparatus according to another aspect of the present invention provides the moving image file based on an endoscope connecting unit to which the endoscope is connected, a moving image acquired through the endoscope connecting unit, and a time. And a recording unit for recording.

An information processing apparatus according to another aspect of the present invention includes a second acquisition unit that acquires a captured real-time image, and the display unit includes the real-time image acquired by the second acquisition unit and the chapter image or the moving image. Are displayed at the same time.

The program according to another aspect of the present invention acquires a moving image file including a moving image and metadata about a chapter image cut out at a predetermined time during shooting of the moving image, and extracts the acquired moving image file from the acquired moving image file. A list of the chapter images is displayed, a selection from the chapter images displayed as a list is received, and a process of reproducing the moving image based on a time corresponding to the chapter image for which the selection has been received is executed by a computer.

According to the present invention, a desired scene can be easily reproduced and displayed in the endoscope apparatus.

1 is a block diagram of an endoscope apparatus that is a first embodiment. FIG. It is the flowchart which showed the moving image recording process. It is the figure which showed the extraction frame image of the moving image data recorded. It is a flowchart of the edit process before moving image reproduction. It is the figure which showed the edit screen displayed on LCD of a front panel. It is a flowchart of a moving image reproduction start process. It is the figure which showed the simultaneous display screen of the live image and recorded image in 2nd Embodiment. It is the figure which showed the edit screen displayed on LCD of the front panel in 3rd Embodiment. It is a flowchart of the image reproduction process in 3rd Embodiment. It is a block diagram of the endoscope apparatus which is 4th Embodiment. It is explanatory drawing explaining the display screen in 4th Embodiment. It is a flowchart which shows the flow of a process of 4th Embodiment.

[First Embodiment]
Hereinafter, an endoscope apparatus according to an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram of the endoscope apparatus according to the first embodiment.

The endoscope apparatus 100 includes a video scope 110 and a processor 120, and the video scope 110 can be detachably connected to the processor 120. In addition, a monitor 150 and a keyboard 170 are connected to the processor 120, and a filing device 140 and a biological information monitoring device 160 are connected. The video scope 110 is an example of an endoscope used when observing the inside of the digestive tract or the like using an image sensor 111 as described later.

The processor 120 includes a light source 124 such as a xenon lamp, and light emitted from the light source 124 is incident on an incident end of a light guide 117 provided in the video scope 110 via a condenser lens 128. The light emitted from the light guide 117 is emitted from the scope distal end portion 110T toward the subject (observation target) via the light distribution lens 119A. A diaphragm 129 is provided between the light source 124 and the light guide 117, and the amount of illumination light is adjusted by opening and closing the diaphragm 129.

The illumination light reflected from the subject is imaged by the objective lens 119B provided at the distal end portion 110T of the scope, and the subject image is formed on the light receiving surface of the image sensor 111. An image sensor 111 constituted by a CMOS sensor or a CCD is driven by a drive circuit 113, and pixel signals for one field or one frame are sent from the image sensor 111 at a predetermined field / frame interval (for example, 1/60 seconds or 1/60 second). Read at intervals of 30 seconds). On the light receiving surface of the image sensor 111, a color filter array (not shown) in which color filters such as Cy, Ye, G, Mg or R, G, B are arranged is arranged.

The pixel signal read from the image sensor 111 is subjected to amplification processing or the like in the analog signal processing circuit, and then sent to the signal processing circuit 125 of the processor 120. The signal processing circuit 125 performs image signal processing such as color conversion processing and gamma correction processing on the digital pixel signal. As a result, R, G, and B color image signals are generated.

The R, G, B image signals are temporarily stored in an image memory (not shown) such as a RAM, and then sent to the subsequent signal processing circuit 135. In the post-stage signal processing circuit 135, an outline emphasis process, a superimpose process, an editing process for simultaneously displaying a plurality of images, and the like accompanying an input operation by the operator are performed. By outputting the video signal to the monitor 150, a real-time observation image is displayed on the monitor 150 as a moving image.

The system control circuit 121 including a CPU (Central Processing Unit) outputs control signals to the timing controller 127, the motor 130, the post-stage signal processing circuit 135, and the like, and controls the operation of the processor 120. The operation control program is stored in the memory 123 in advance.

The scope controller 115 controls the timing controller 114 and sends an operation signal to the system control circuit 121 in response to a button operation from the freeze button 118 or the like. When the video scope 110 is connected to the processor 120, scope related data stored in the memory 116 of the video scope 110 is read and stored in the RAM 122 of the processor 120. Specifically, data such as the scope model name is read.

While the observation image is displayed by imaging with the video scope 110, the operator can record a moving image by operating the keyboard 170 or the like. When a moving image recording operation is performed, the image data of each frame generated in the signal processing circuit 125 is compressed in the system control circuit 121 and recorded as a moving image file in the RAM 122. Further, it is output to the filing device 140 as necessary. Here, moving image compression processing according to the MPEG system is performed. As will be described later, when the operator operates the freeze button 118 during moving image recording, the frame image data of the scene at the time of the operation is recorded as metadata.

The front panel 126 of the processor 120 is provided with an LCD 136 as a display unit, and a touch panel 138 is installed in accordance with the screen frame of the LCD 136. The operator touches the displayed contour emphasis mark and brightness level mark, thereby executing image processing, brightness adjustment, and the like. Further, as will be described later, an edit screen capable of selecting a moving image reproduction start scene can be displayed during moving image reproduction.

The filing device 140 includes a device main body 141 and a filing monitor 142, and the device main body 141 includes a controller 144 and a memory 145. The controller 144 receives image data or the like sent from the processor 120, files it, and records it in the memory 145. When there is an instruction to read a moving image file from the processor 120, the target moving image file is read from the memory 145 and moving image data is transmitted to the processor 120. The biological information monitoring device 160 displays a heart rate, a pulse, a pulse waveform thereof, and the like on the dedicated monitor 165 based on signals from various sensors attached to the patient.

FIG. 2 is a flowchart showing the moving image recording process. FIG. 3 is a diagram showing an extracted frame image of moving image data to be recorded. With reference to FIGS. 2 and 3, still image storage during moving image recording will be described.

When the moving image recording process is started by the operation of the operator (S101), the operator moves the scope distal end portion 110T while operating the video scope 110. When the observation image of the desired scene is displayed, the freeze button 118 is pressed. If it is determined by the operator that the freeze button 118 has been operated to store a still image (S102), the frame image at the time of the operation is recorded in the memory 123 or RAM 122 as metadata (S103).

FIG. 3 shows a frame image when the operator operates the freeze button 118 three times for recording a still image during moving image recording. The J-th frame image AJ after time T1 has elapsed from the start of recording corresponds to an image when the scope distal end portion 110T reaches the observation target region and enters the visual field range. The operator saves a frame image as an image serving as a moving image reproduction start point (index). Specifically, metadata relating to the frame image AJ is assigned to acquire an annotation.

At this time, information on the scope type and image processing setting contents (such as contour emphasis) is added as metadata along with the frame number or time information from the start of the moving image, and is recorded in the moving image file. Further, in the present embodiment, the frame image AJ is added as metadata m1 and recorded in the moving image file. That is, the frame image AJ itself is handled as the metadata m1.

In the moving image file, audio data, compressed moving image data, audio, moving image synchronization data, and the like are stored in a container serving as a main storage area, and metadata is stored in a predetermined area. Further, in the metadata, in addition to the description metadata such as the time from the start of recording, the frame number, and the like, a metadata area in which the user can arbitrarily define the format is defined, and the frame image specified by the operator operation Data is stored as metadata. As the moving image recording system, any system (for example, MP4, MXF format, etc.) can be adopted.

The frame image AM of the Mth frame after the time T2 has elapsed from the start of recording is a frame image corresponding to the scene in which the video scope 110 captures the lesion, and the frame image AM is operated by the operator operating the freeze button 118. Is added as metadata m2 and recorded in the moving image file. The frame image AN of the Nth frame is a frame image corresponding to the scene when moving to a different part, is given as metadata m3, and is recorded in a moving image file. Steps S102 to S104 are repeatedly executed until a moving image recording end operation is performed using the keyboard 170 or the like (S104).

FIG. 4 is a flowchart of the editing process before moving image playback. FIG. 5 is a diagram showing an editing screen displayed on the LCD 136 of the front panel 126. FIG. 6 is a flowchart of the moving image reproduction start process. The selection and setting of the moving image playback start scene will be described with reference to FIGS.

When the moving image playback mode is set by the operator's operation with the keyboard 170 or the like, an editing screen as shown in FIG. 5 is displayed. When a moving image file to be reproduced is retrieved by the operator (S201), the first frame image is displayed on the LCD 136 as a reduced image (hereinafter referred to as a thumbnail image) (S202). At this time, in addition to the chapter image, the scope type and image processing setting contents recorded as metadata can be displayed as character information. Then, it is determined whether or not frame image data exists in the metadata written in the searched moving image file (S203).

If the frame image data exists as metadata, the metadata is extracted and displayed on the LCD 136 as an image for editing (hereinafter referred to as a chapter image) (S204). By executing steps S203 and S204 for all metadata (S205), a list of chapter images included in the metadata is displayed. However, when the frame image data is not included in the metadata, only the thumbnail image is displayed, and character information indicating that it is not included is displayed.

In FIG. 5, the thumbnail image M0 of the moving image first scene and the extracted chapter images C1, C2, and C3 are displayed on the editing screen. The chapter images C1, C2, and C3 correspond to the frame images AJ, AM, and AN of FIG.

Here, the thumbnail image M0 of the first scene and the chapter images C1 to C3 are images of the same size, and the chapter images C1 to C3 are displayed in the order of the still image saving operation. The operator can select a series of chapter images C1 to C3 as moving image playback start points by operating the touch panel 138.

Note that the size of the thumbnail image M0 and the chapter images C1 to C3 may be different. By displaying the two images in different sizes, the operator can easily distinguish the thumbnail image M0 from the chapter images C1 to C3. The sizes of the chapter images C1 to C3 may be different from each other.

When the operator selects a thumbnail image (see S301 in FIG. 6), the moving image starts to be reproduced on the monitor 150 from the first scene of the moving image file (S302). On the other hand, when a chapter image is selected by the operator, a moving image is started to be reproduced from a scene corresponding to the selected chapter image (S303, S304). Specifically, based on the time from the start of recording or the frame number (for example, time T1, frame number J) recorded as metadata together with the chapter image, reproduction is started from the frame image corresponding to the chapter image.

As described above, according to the present embodiment, in the endoscope apparatus 100 capable of moving image recording and reproduction display, when an operator performs a still image saving operation during moving image recording, a frame corresponding to the scene at the time of operation is displayed. Images (chapter images) are recorded as metadata in a moving image file. When the moving image playback mode is set, an editing screen on which a moving image playback start scene can be selected is displayed on the LCD 136. At this time, metadata corresponding to the chapter image is extracted from the moving image file and displayed in a list on the editing screen. When the operator selects a desired chapter image, the moving image starts to be reproduced from the scene of the chapter image.

Since the still image data of one scene during moving image recording is recorded according to the operator's intention, it is possible to immediately grasp which scene is the chapter image displayed on the editing screen before playback display. Further, since frame image data is recorded as metadata in such a manner that it is embedded in a moving image file, it is possible to record moving images and still images without making them into separate files. Furthermore, since still image data is recorded as metadata in association with moving image data, it is easy to grasp what situation still image is recorded. In particular, it is possible to display a list of chapter images simply by searching for and extracting image data in the metadata, so that the display of the editing screen is not complicated.

On the other hand, since the moving image file is recorded not in the filing device 140 but in the memory in the processor 120, there is no need for a dedicated connection cable or command transmission. Further, since the scope type and the image processing setting content are also recorded as metadata, it becomes possible to grasp later the work situation at the time of moving image recording.

It should be noted that the moving image playback may be started not from the frame image scene corresponding to the chapter image but from the preceding and succeeding scenes. For example, when a moving image recording process is performed in the operation of feeding the scope tip portion 110T to the observation target portion in the endoscope operation and observing the portion while gradually pulling back the scope tip portion 110T, a frame image several seconds before the chapter image is displayed. The reproduction can be started from the beginning so that the desired scene is not missed. In this case, the playback / display start scene may be corrected according to a predetermined number of shift frames or time with respect to the recording time or frame number metadata associated with the chapter image.

Regarding frame image data extraction from metadata, frame image data may be extracted as a still image file from data of the entire moving image file instead of extracting still image data for each metadata. . In this case, a step of still image file creation processing is executed instead of steps S203 to S205 of FIG.

Real-time moving images (live images) and recorded moving images may be displayed on the monitor 150 at the same time. For example, the simultaneous display mode can be set by the operator operating the keyboard 170. Further, the frame image data may be recorded at a timing designated by the operator by a method other than the operator's scope button operation. The edit screen may be displayed on a display unit other than the processor, and may be configured to be displayed on the monitor 150, for example.

When the operator performs a specific input operation using the keyboard 170 or operates a foot switch, the frame image data of the scene at the time of the operation may be recorded in the moving image file as metadata. The metadata may be data other than the frame image data.

[Second Embodiment]
Next, an endoscope apparatus according to the second embodiment will be described with reference to FIG. In the second embodiment, image data of the biological information monitoring device and filing device can be recorded, reproduced, and displayed as metadata.

FIG. 7 is a diagram showing a simultaneous display screen of live images and recorded images in the second embodiment.

In the second embodiment, when the operator performs a still image saving operation, not only the frame image data corresponding to the scene at the time of the operation but also the biological information displayed at the time of operation on the dedicated monitor 165 of the biological information monitoring device 160. A display image (still image) is recorded as metadata. When receiving the stored data transmission request from the processor 120, the biological information monitoring device 160 transmits the biological information display screen data displayed during the operation to the processor 120 as still image data. The processor 120 records the transmitted still image data as metadata in a moving image file. Note that a capture board (not shown) provided in the processor 120 may be used to capture still image data as a video signal.

When a chapter image is selected on the editing screen shown in the description of the first embodiment, a biometric information display image recorded along with the chapter image is displayed together with the start of moving image recording / playback.

FIG. 7 shows a real-time moving image I1, a recording / reproducing moving image I2, and a biological information display image I3 displayed on the monitor 150 during moving image reproduction. The operator can compare the medical condition of the current part with the past medical condition by starting reproduction and display from a scene in which the same part as the real-time moving image is displayed in the moving image file recorded in the past. At the same time, by confirming the past biometric information display image on the monitor 150, it can be compared with the biometric information currently displayed on the dedicated monitor 165.

Further, when the operator performs a still image saving operation, the frame image data of the real-time image displayed at that time is recorded as metadata, and the frame image (reproduced frame image) of the recorded / reproduced moving image displayed at that time is recorded. ) Are also recorded as metadata. In the filing device 140, the playback frame image data corresponding to the operation scene of the moving image data restored by the controller 144 is transmitted to the processor 120. Note that a still image may be acquired from the filing device, the still image may be reproduced and displayed simultaneously with the real-time moving image, and the still image may be recorded as metadata.

FIG. 7 shows real-time frame image data m5 and reproduction frame image data m6 to be recorded. Here, real-time frame image data m5 and reproduction frame image data m6 are recorded as one metadata. At this time, character information CM of a predetermined comment (please confirm progress observation, etc.) may be recorded as an attached form. Thereby, the present and past states of the affected area can be compared and observed. In addition, about a comment, a user may input a comment arbitrarily and preserve | save comment data and may record a comment in a required scene.

Note that the recording and reproduction display as the metadata of the biological information image data may be applied to the reproduction display of only the recorded moving image as in the first embodiment. In the second embodiment, the reproduction and display of the reproduced moving image and the real-time moving image may be performed without recording and reproducing the biological information image data as metadata. In these cases, it may be recorded as metadata regardless of the frame image data of the real-time moving image. The present invention can also be applied to a device capable of outputting a video signal with an external peripheral device other than the biological information monitoring device.

[Third Embodiment]
The present embodiment relates to an endoscope apparatus 100 that repeatedly reproduces a moving image around a time corresponding to a chapter selected by an operator. Description of parts common to the first embodiment is omitted.

FIG. 8 is a diagram showing an editing screen displayed on the LCD 136 of the front panel 126 according to the third embodiment. The operation when the thumbnail image M0 or the chapter images C1 to C3 is single-tapped by the operator is the same as that in the first embodiment.

FIG. 8 shows an example of an image displayed on the LCD 136 when the chapter image C1 is double-tapped by the operator. The selected chapter image C1 is surrounded by a thick frame 31 to indicate that it is being selected.

Inside the thick frame 31, a moving image in a predetermined repetition time range is repeatedly reproduced and displayed around the time when the chapter image C1 is recorded. The repetition time is several seconds, for example, about 3 seconds. The area inside the thick frame 31 is an example of the reproducing unit of the present embodiment.

The time bar 32 is displayed at the bottom of the screen. The total length of the time bar 32 indicates the repetition time. A cursor 33 is displayed on the time bar 32. The cursor 33 indicates the time when the image displayed in the thick frame 31 is captured within the range of the repetition time. Note that it may be possible for the operator to change the repetition time and the reproduction speed as appropriate by operating the keyboard 170 or the touch panel.

In endoscopy, when an operator finds an important part such as a lesion, the operator operates the freeze button 118 to record a still image. As described in the first embodiment, when the freeze button 118 is operated, metadata such as a chapter image is recorded in the moving image file.

However, the optimal image for diagnosis or the like is not always displayed at the time when the operator presses the freeze button. Further, in some cases, a moving image is easier to grasp, for example, a three-dimensional shape of a lesioned part than a still image.

As shown in FIG. 8, the operator can easily re-observe the part of interest at the time of endoscopy by repeatedly reproducing a short time moving image before and after the chapter image.

FIG. 9 is a flowchart of image reproduction processing in the third embodiment. Steps S301 and S302 are the same as the processing of the first embodiment described with reference to FIG.

In the present embodiment, the chapter image is selected in step S303 by single tap or double tap of the chapter image. Instead of single tap or double tap, any operation such as swipe, simultaneous tap with a plurality of fingers, drag and drop to a predetermined area, or keyboard operation can be used.

When a chapter image is selected by a single tap (single tap in step S303), similarly to step S304 of the first embodiment, a moving image starts to be reproduced from a scene corresponding to the selected chapter image (step S304). ).

When the chapter image is selected by double tap (double tap in step S303), the thick frame 31, the time bar 32, and the cursor 33 are displayed on the front panel 126 as described with reference to FIG. A moving image before and after the time when the selected chapter image is stored is repeatedly reproduced in the thick frame 31 (step S305). After the end of step S304 or step S305, the process ends.

The moving image may be displayed in a greatly enlarged manner. The moving image may be displayed on a display device different from the front panel 126, for example, the monitor 150.

Note that the processor 120 may return to step S301 in parallel while performing the moving image reproduction process of step S302, step S304, or step S305, and accept the next operation by the operator.

According to the present embodiment, it is possible to provide an endoscope apparatus 100 that repeatedly reproduces a moving image of a short time before and after a chapter image. By using the endoscope apparatus 100 according to the present embodiment, the operator can easily re-observe the part of interest during the endoscopy.

The flowchart described using FIG. 6 or FIG. 9 may be executed by an information processing apparatus such as a general-purpose personal computer or tablet capable of reading a moving image file including metadata recorded during endoscopy. For example, video files including metadata are recorded in a HIS (Hospital Information System) recording device connected via a network and installed in a room different from the endoscopy room It can be played back using a personal computer.

[Fourth Embodiment]
The present embodiment relates to an endoscope apparatus 100 including a processor 120 that mainly operates based on software processing. Description of parts common to the first embodiment is omitted.

FIG. 10 is a block diagram of an endoscope apparatus according to the fourth embodiment. The endoscope apparatus 100 includes a video scope 110, a processor 120, and a display device 90. The internal configuration of the video scope 110 is the same as that of the video scope 110 of Embodiment 1 described with reference to FIG.

The display device 90 includes a display unit 91 and an input unit 92. The display unit 91 is a liquid crystal display panel, for example. The input unit 92 is a touch sensor that is disposed on the surface of the display unit 91 and constitutes a touch panel in conjunction with the display unit 91. The input unit 92 may be a keyboard or a mouse.

The processor 120 includes a CPU 11, a main storage device 12, an auxiliary storage device 13, a communication unit 14, a display I / F (Interface) 15, an input I / F 16, a scope I / F 17, a light source 124, and a bus. The processor 120 according to the present embodiment is an example of an information processing apparatus that operates when the CPU 11 executes a program.

The CPU 11 is an arithmetic control device that executes a program according to the present embodiment. As the CPU 11, one or a plurality of CPUs or a multi-core CPU is used. The CPU 11 is connected to each hardware part constituting the processor 120 via a bus.

The main storage device 12 is a storage device such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), flash memory or the like. The main storage device 12 temporarily stores information necessary during the processing performed by the CPU 11 and a program being executed by the CPU 11.

The auxiliary storage device 13 is a storage device such as SRAM, flash memory, hard disk or magnetic tape. The auxiliary storage device 13 stores a program to be executed by the CPU 11, a moving image file, and various types of information necessary for executing the program. The moving image file may be stored in an external mass storage device connected to the processor 120 via a network such as HIS.

The communication unit 14 is an interface that performs communication with a network. The display I / F 15 is an interface that connects the display unit 91 and the bus. The input I / F is an interface that connects the input unit 92 and the bus. The scope I / F 17 is an interface that connects the video scope 110 and the bus. The scope I / F 17 is an example of an endoscope connection unit to which an endoscope is connected. The analog signal processing circuit 112, the timing controller 114, and the scope controller shown in FIG. 1 are connected to the bus via the scope I / F 17, respectively.

FIG. 11 is an explanatory diagram for explaining a display screen in the fourth embodiment. The screen shown in FIG. 11 is displayed on the display unit 91. The screen shown in FIG. 11 includes a thumbnail image M0, chapter images C1, C2, and C3, a real-time moving image I1, and a recording / reproducing moving image I2. Note that the chapter images may be displayed in a list in two or more rows.

In the real-time moving image I1, an image taken by the image sensor 111 of the video scope 110 is displayed in real time. The thumbnail image M0 and the chapter images C1, C2, and C3 include, for example, a thumbnail image and a chapter image extracted from a moving image file when a patient undergoing a similar endoscopic examination in the past. Is displayed. The thumbnail image C2 selected by the operator is surrounded by a thick frame 31.

In the recorded / reproduced image I2, a reproduced image reproduced from the time when the chapter image surrounded by the thick frame 31 is recorded is displayed. In the recording / reproducing image I2, a still image taken at the time when the chapter image was recorded may be displayed. A chapter image surrounded by a thick frame 31 may be enlarged and displayed on the recording / reproducing image I2. The area where the recording / reproducing image I2 is displayed is an example of the reproducing unit of the present embodiment.

Note that the moving image file may be selected from moving image files stored in the auxiliary storage device 13 based on an instruction from the operator. The moving image file may be acquired from the HIS based on an instruction from the operator and temporarily recorded in the auxiliary storage device 13. The moving image file may be automatically acquired from the HIS and recorded in the auxiliary storage device 13 based on the ID (Identifier) of the patient under examination. For example, in the case of a patient who performs follow-up observation by periodic endoscopy, the CPU 11 can select a moving image file recorded by the previous endoscopy based on the patient ID.

The outline of the procedure for displaying the screen shown in FIG. 11 will be described. Before starting the endoscopic examination, the operator selects a moving image file to be reproduced during the endoscopic examination or inputs information such as a patient ID necessary for the CPU 11 to select the video file.

CPU11 extracts the chapter image from the metadata included in the moving image file and displays it. When the operator starts the endoscopy, the CPU 11 displays the real-time moving image I1 based on the signal acquired from the video scope 110 via the scope I / F.

The operator selects a chapter image to be compared with the real-time moving image I1 by an operation such as a single tap. The CPU 11 displays the recording / reproducing image I2 based on the chapter image that has received the selection.

Thus, for example, the operator can compare the change of the lesion site between the previous endoscopy and the current endoscopy. The operator can operate the video scope 110 to shoot a lesion from the same direction as the recorded / reproduced image I2, and compare the real-time moving image I1 thus captured with the recorded / reproduced image I2. The presence or absence of changes in the lesion can be observed in detail.

The operator may select, for example, a moving image file obtained by capturing an endoscopic image of a healthy person. The operator can compare the endoscopic image of a healthy person displayed in the recording / reproducing image I2 with the real-time moving image I1.

FIG. 12 is a flowchart showing a process flow of the fourth embodiment. CPU11 acquires a moving image file (step S311). CPU11 implement | achieves the function of the 1st acquisition part which acquires a moving image file by step S311. The CPU 11 extracts a chapter image from the moving image file (step S312). The CPU 11 displays a list of the extracted chapter images as described with reference to FIG.

The CPU 11 displays the real-time moving image I1 based on the signal acquired from the video scope 110 via the scope I / F (step S314). Note that the CPU 11 continues to update the real-time moving image I1 in parallel with the processing described below. CPU11 implement | achieves the function of the 2nd acquisition part which acquires a real-time image by step S314.

The CPU 11 determines whether or not selection of any chapter image has been received from the chapter images displayed as a list (step S315). CPU11 implement | achieves the function of the reception part which receives selection of a chapter image by step S315. If it is determined that it has been accepted (YES in step S315). The CPU 11 displays the recorded / reproduced image I2 (step S316). When the recording / reproduction image I2 is a moving image, the CPU 11 continues to reproduce the recording / reproduction image I2 until selection is accepted again.

The CPU 11 determines whether or not to end the process (step S317). For example, when the endoscopy ends, the CPU 11 determines to end the process. If it is determined that the process is to be ended (YES in step S317), the CPU 11 ends the process.

If it is determined that the process is not terminated (NO in step S317), the CPU 11 returns to step S315.

According to the present embodiment, it is possible to provide the endoscope apparatus 100 that displays the real-time image I1 and the recording / reproduction image selected by the operator side by side during the endoscopic examination.

For example, part of processing performed by the CPU, such as image processing, may be executed by a dedicated circuit board connected to the bus.

The CPU 11 may automatically select a chapter image similar to the real-time image I1 by processing such as image pattern matching.

The technical features (components) described in each embodiment can be combined with each other, and new technical features can be formed by combining them.
The embodiment disclosed this time is to be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims rather than the meanings described above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

11 CPU
12 Main storage device 13 Auxiliary storage device 14 Communication unit 15 Display I / F
16 input I / F
17 Scope I / F 31 Thick frame 32 Time bar 33 Cursor 100 Endoscope device 110 Video scope (video scope, endoscope)
120 processor 121 system control circuit (editing processing unit, reproduction processing unit)
135 Subsequent signal processing circuit (editing processing unit, reproduction processing unit, recording processing unit)
136 LCD (display unit)
150 monitors

Claims

A recording processing unit for recording a real-time moving image in a memory as a moving image file;
A reproduction processing unit that reads out the moving image file and reproduces and displays a moving image on a monitor;
An edit processing unit that displays an edit screen on which a playback start scene can be selected is displayed on the display unit,
The recording processing unit stores a frame image (hereinafter referred to as a chapter image) corresponding to a scene designated by an operator during moving image recording as metadata in the moving image file,
The editing processing unit extracts metadata corresponding to the chapter image, displays the chapter image on the editing screen,
The endoscope apparatus, wherein the reproduction processing unit starts reproduction of a moving image from a scene of a chapter image selected by an operator or a scene before and after the chapter image.
The endoscope apparatus according to claim 1, wherein the recording processing unit stores a chapter image corresponding to a scene at the time of a still image saving operation by an operator as metadata in the moving image file.
The endoscope apparatus according to claim 1 or 2, wherein the editing processing unit extracts a frame image in the metadata as a still image file.
4. The recording apparatus according to claim 1, wherein the recording processing unit stores at least one of a scope type and image processing setting contents as metadata together with a chapter image in the moving image file. Endoscopic device.
The editing processing unit displays a head frame image corresponding to a head scene in the recorded moving image data together with a chapter image,
The endoscope apparatus according to any one of claims 1 to 4, wherein when the first frame image is selected by an operator, the reproduction processing unit starts reproduction of a moving image from the first frame image.
The processor of the endoscope apparatus can be connected to an external peripheral device capable of displaying an endoscope work-related video and outputting a video signal,
The endoscopic work-related still image displayed during the still image saving operation by the operator is saved in the moving image file as metadata by the recording processing unit. Endoscope device.
The endoscope apparatus according to claim 6, wherein the reproduction processing unit extracts metadata corresponding to an endoscope work-related still image, and displays the endoscope-related still image during moving image reproduction. .
A processor of the endoscopic device is connectable to a filing device;
The reproduction processing unit can simultaneously display a real-time moving image and a recorded image recorded in the filing device,
8. The recording apparatus according to claim 1, wherein the recording processing unit stores a playback frame image corresponding to a playback scene at the time of a still image storage operation by an operator as metadata in the moving image file. Endoscopic device.
Record real-time moving images as video files in memory,
Read the moving image file and display the moving image on the monitor,
A method for displaying an edit screen on which a playback start scene can be selected is displayed on a display unit,
During moving image recording, a chapter image corresponding to the scene specified by the operator is saved as metadata in the moving image file,
Extract the metadata corresponding to the chapter image, display the chapter image on the editing screen,
A recording / reproducing method for an endoscope apparatus, wherein reproduction of a moving image is started from a chapter image scene selected by an operator or a scene before and after the chapter image scene.
A first acquisition unit that acquires a moving image and a moving image file including metadata about a chapter image cut out at a predetermined time during shooting of the moving image;
A display unit for displaying a list of the chapter images extracted from the moving image file acquired by the first acquisition unit;
A receiving unit for receiving selection from the chapter images displayed in a list by the display unit;
An information processing apparatus comprising: a reproduction unit that reproduces the moving image based on a time corresponding to the chapter image for which the reception unit has received selection.
The moving image is an image taken using an endoscope,
The information processing apparatus according to claim 10, wherein the first-term first acquisition unit acquires a moving image file including metadata about a chapter image cut out at a time when a button provided on the endoscope is operated.
An endoscope connecting portion to which the endoscope is connected;
The information processing apparatus according to claim 11, further comprising: a recording unit that records the moving image file based on a moving image and time acquired through the endoscope connection unit.
A second acquisition unit for acquiring a captured real-time image;
The information processing apparatus according to any one of claims 10 to 12, wherein the display unit simultaneously displays the real-time image acquired by the second acquisition unit and the chapter image or the moving image.
Obtaining a moving image file including a moving image and metadata about a chapter image cut out at a predetermined time during shooting of the moving image;
List the chapter images extracted from the acquired video file,
Accept selection from the chapter images displayed in a list,
A program that causes a computer to execute a process of reproducing the moving image based on a time corresponding to a chapter image for which selection has been accepted.