JP4472631B2 - Image processing apparatus and image processing method in the image processing apparatus - Google Patents

Abstract

The invention provides an image processing device which implements efficiency of an observation by a user and an image processing method in the image processing device. The image processing device is equipped with an image signal input means for inputting image signals from a plurality of images captured by a medical device with an imaging feature, an image segmentation means for dividing each of the plurality of images into a plurality of regions, a characteristic amount calculating means for calculating the amount of characteristics in each of the plurality of regions, a cluster sorting means for sorting each of a plurality of clusters generated in a characteristic space into one of a plurality of classes from the amount of the characteristics and the occurrence frequency of the amount of the characteristics, a subset generating means for generating a plurality of subsets by using the plurality of images,; and a sorting criterion calculating means for calculating sorting criteria when sorting the images included in a subset from the distribution state of the characteristic amount in the characteristic space of each image included in a subset.

Description

  The present invention relates to an image processing apparatus and an image processing method in the image processing apparatus, and in particular, an image processing apparatus capable of excluding an image in which an image of a biological mucous membrane surface is not taken well and image processing in the image processing apparatus It is about the method.

  Conventionally, in the medical field, observation using image pickup devices such as X-rays, CT, MRI, ultrasonic observation apparatuses and endoscope apparatuses has been widely performed. Among such imaging devices, the endoscope apparatus has, for example, an elongated insertion portion that is inserted into a body cavity as a living body, and is imaged by an objective optical system disposed at the distal end portion of the insertion portion. The image of the body cavity is picked up by an image pickup means such as a solid-state image pickup device and output as an image pickup signal, and the image and image of the body cavity are displayed on a display means such as a monitor based on the image pickup signal. . Then, based on the image of the image in the body cavity displayed on the display unit such as a monitor, the user observes an organ or the like in the body cavity, for example. In addition, the endoscope apparatus can directly capture an image of the digestive tract mucosa. Therefore, the user can comprehensively observe various findings such as the color of the mucous membrane, the shape of the lesion, and the fine structure of the mucosal surface.

  In recent years, for example, a capsule endoscope apparatus has been proposed as an imaging apparatus that can be expected to have substantially the same usefulness as the above-described endoscope apparatus. In general, a capsule endoscope device is placed in a body cavity when a subject swallows it from the mouth, and the capsule endoscope device transmits an image of the imaged body cavity to the outside as an imaging signal. A receiver for storing the received imaging signal after receiving the captured imaging signal outside the body cavity, and an observation device for observing an image of the image in the body cavity based on the imaging signal stored in the receiver The

  Since the capsule endoscope that constitutes the capsule endoscope apparatus advances by peristaltic movement of the digestive tract, for example, it takes several hours to be discharged from the anus after being inserted into the body cavity from the mouth. It is common. And since the capsule endoscope keeps outputting the imaging signal to the receiver almost constantly until it is discharged after being put in the body cavity, for example, in the moving image for several hours, The number of still images as accumulated frame images is enormous. Therefore, in terms of improving the efficiency of observation by the user, for example, an image processing method for detecting a predetermined image including a lesion site such as a bleeding site from the accumulated image is performed. There is a demand for a proposal that reduces the data amount of an image by performing a process of not displaying or storing an image other than the first image.

  As an image processing method as described above, for example, there is a method described in Patent Document 1. The method for detecting colorimetric abnormality in vivo described in Patent Document 1 focuses on the difference in color tone between the normal mucous membrane and the bleeding site, and in a feature space in which the color tone is set as a feature amount. This is an image processing method including a method for detecting a bleeding site for each divided region of an image based on a distance from each average value, that is, a method for detecting a predetermined image including a bleeding site as a lesion site. .

PCT WO 02/073507 A2

  However, the image processing method described in Patent Document 1 has the following problems.

  In general, in the digestive tract, only the image of the surface of the living body mucosa is not always captured. For example, the image of a foreign body such as stool, foam, mucus or food residue and the image of the surface of the living body mucosa are mixed. In this state, imaging is performed. Therefore, in the image processing method described in Patent Document 1 that does not consider the presence of foreign matter as described above, for example, the possibility that the normal mucous membrane may be erroneously detected as a bleeding site by the foreign matter, and the foreign matter There is a possibility of detecting an image in which most of the image is occupied. As a result, there is a problem that the efficiency of observation by the user cannot be achieved.

  The present invention has been made in view of the above-described points. For example, an image of the gastric mucosa and villus as an image of the surface of a living mucosa and an image of stool and bubbles as images of a foreign body are provided for each small region of the image. By classifying and classifying images, it is possible to easily exclude images that do not have a good image of the surface of the biological mucosa, such as images of foreign matter occupying many small areas of the image, as images that do not require observation. Further, an image processing apparatus capable of specifying an imaged organ based on the classification result and, as a result, improving the efficiency of observation by a user, and an image processing method in the image processing apparatus are provided. The purpose is that.

  In addition, according to the present invention, for each region classified as an image on the surface of a living mucosa, for example, based on the feature amount of each region, a normal mucosa image and a lesion site image are classified. An object of the present invention is to provide an image processing apparatus capable of improving the detection accuracy of a lesion site by using a simple image processing method and an image processing method in the image processing apparatus.

  The first image processing apparatus according to the present invention is based on an image signal input unit that inputs image signals based on a plurality of images captured by a medical device having an imaging function, and an image signal input in the image signal input unit, Image dividing means for dividing each of the plurality of images into a plurality of areas; feature quantity calculating means for calculating a feature quantity in each of the plurality of areas divided by the image dividing means; and the feature quantity and the feature quantity Based on the occurrence frequency, a plurality of clusters are generated in the feature space, and each of the plurality of images is captured using a cluster classification unit that classifies each of the plurality of clusters into one of a plurality of classes, and the plurality of images. Subset generating means for generating a plurality of subsets based on the determined timing, and one sub-generation generated by the subset generating means A classification criterion for classifying the images included in the one subset into any of the plurality of classes, based on a distribution state of the feature values in the feature space of each of the images included in the network. And a classification standard calculation means.

  The second image processing apparatus according to the present invention is the first image processing apparatus, and further, the classification criterion calculating means is configured to classify the class in which the feature amount does not occur in one subset of the plurality of classes. The classification criterion is not calculated.

  The third image processing apparatus according to the present invention is characterized in that, in the first or second image processing apparatus, the plurality of classes include at least a class relating to a biological mucosa and a class relating to a non-biological mucosa.

  According to a fourth image processing apparatus of the present invention, in the third image processing apparatus, a classification unit that classifies each region in one image included in the one subset based on the classification criterion; When the one image has a region classified as a class related to the biological mucous membrane based on the classification result of one image, a lesion detection unit that performs a process for detecting a lesion location for each region It is characterized by having.

According to a fifth image processing apparatus of the present invention, an image signal input unit that inputs image signals based on a plurality of images captured in time series by a medical device having an imaging function, and an image input by the image signal input unit Based on the signal, an image dividing unit that divides the plurality of images into a plurality of regions, a feature amount calculating unit that calculates a feature amount in each of the plurality of regions divided by the image dividing unit, and a feature amount An image region classifying unit that classifies each of the plurality of regions into one of a plurality of classes, and a representative value calculating unit that calculates a representative value of at least one class of feature values in each of the plurality of images; And a fluctuation detecting means for detecting the fluctuation of the representative value in time series .

  The sixth image processing apparatus according to the present invention further includes smoothing means for applying smoothing in a time series direction to each of the representative values in the fifth image processing apparatus, and the fluctuation detecting means is , Based on the representative value smoothed by the smoothing means, detecting either an image or a time at which the fluctuation of the representative value is maximized.

  The seventh image processing apparatus according to the present invention is characterized in that, in the fifth or sixth image processing apparatus, the variation of the representative value is caused based on a change in an observation site in a living body.

  The eighth image processing apparatus according to the present invention is characterized in that, in the fifth or sixth image processing apparatus, the variation of the representative value is caused based on the presence of feces in a living body.

  According to a ninth image processing apparatus of the present invention, in the fifth or sixth image processing apparatus, the variation of the representative value is caused based on the presence of bile in a living body.

  The tenth image processing apparatus according to the present invention is characterized in that, in the fifth or sixth image processing apparatus, the variation of the representative value is caused based on the presence of a lesion site in a living body.

In an eleventh image processing apparatus according to the present invention, in the fifth to tenth image processing apparatuses, the representative value is an average value of feature amounts in each of the plurality of classes.
According to a twelfth image processing apparatus of the present invention, in the fourth image processing apparatus, the lesion detection means is provided in the medical device.
In a thirteenth image processing apparatus according to the present invention, in the fifth image processing apparatus, the variation detecting means detects either an image or a time at which the variation of the representative value detected in time series becomes a maximum. It is characterized in that the processing is performed.

  According to a first image processing method of the present invention, an image dividing step for dividing each of the images into a plurality of regions based on an image signal input based on an image captured by a medical device having an imaging function; A feature amount calculating step for calculating a feature amount in each of the plurality of regions divided by the dividing step; generating a plurality of clusters in a feature space based on the feature amount and the frequency of occurrence of the feature amount; and A cluster classification step of classifying each of the clusters into a plurality of classes, a subset generation step of generating a plurality of subsets based on the timing at which each of the plurality of images is captured using the plurality of images, and the subset generation Each of the images included in one subset generated by the step in the feature space Based on the distribution state of the feature, characterized by comprising a classification criterion calculating step of calculating a classification criteria for classifying each of the images included in the subset of the one.

  According to a second image processing method of the present invention, in the first image processing method, the classification reference calculation step further includes a step in which the feature quantity does not occur in one subset of the plurality of classes. The classification criterion is not calculated.

  According to a third image processing method of the present invention, in the first or second image processing method, the plurality of classes include at least a class related to a biological mucosa and a class related to a non-biological mucosa.

  According to a fourth image processing method of the present invention, in the third image processing method, a classification step of classifying each region in one image included in the one subset based on the classification criterion; A lesion detection step of performing a process for detecting a lesion location for each of the regions when the one image has a region classified as a class related to the biological mucous membrane based on the classification result of the one image It is characterized by having.

According to a fifth image processing method of the present invention, an image dividing step of dividing the image into a plurality of regions based on an image signal input based on an image captured by a medical device having an imaging function; A feature amount calculating step for calculating a feature amount in each of the plurality of regions divided by the dividing step; and an image region classification step for classifying each of the plurality of regions into one of a plurality of classes based on the feature amount; Each of the plurality of images, comprising: a representative value calculating step of calculating a representative value of at least one class of feature quantity; and a fluctuation detecting step of detecting a fluctuation of the representative value in time series. To do.

  According to a sixth image processing method of the present invention, in the fifth image processing method, the image processing method further includes a smoothing step of applying smoothing in a time-series direction to each of the representative values. Based on the representative value after smoothing, a process for detecting either an image or a time at which the fluctuation of the representative value is maximized is performed in the fluctuation detecting step.

  According to a seventh image processing method of the present invention, in the fifth or sixth image processing method, the variation in the representative value is caused based on a change in an observation site in a living body.

  An eighth image processing method according to the present invention is characterized in that, in the fifth or sixth image processing method, the variation of the representative value is caused based on the presence of feces in a living body.

  According to a ninth image processing method of the present invention, in the fifth or sixth image processing method, the change in the representative value is caused based on the presence of bile in a living body.

  The tenth image processing method according to the present invention is characterized in that, in the fifth or sixth image processing method, the variation of the representative value is caused based on the presence of a lesion site in a living body.

An eleventh image processing method according to the present invention is characterized in that, in the fifth to tenth image processing methods, the representative value is an average value of feature amounts in each of the plurality of classes.
According to a twelfth image processing method of the present invention, in the fourth image processing method, the lesion detection step is performed in the medical device.
According to a thirteenth image processing method of the present invention, in the fifth image processing method, the variation detection step detects either an image or a time at which the variation of the representative value detected in time series becomes a maximum. It is characterized in that the processing is performed.

  According to the image processing apparatus and the image processing method in the image processing apparatus of the present invention, it is not necessary to observe an image on which the surface of the biological mucosa is not satisfactorily captured, such as a foreign object occupying many small areas of the image. Can be easily excluded, and further, the imaged organ can be identified based on the classification result, and as a result, the efficiency of observation by the user can be improved.

  Further, according to the image processing device and the image processing method in the image processing device of the present invention, for each region classified as an image of the biological mucosal surface, for example, based on the feature amount of each region, By using an image processing method that classifies the image of a normal mucous membrane and an image of a lesion site, the detection accuracy of the lesion site can be improved.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is an external front view showing the external appearance of an image processing apparatus and peripheral devices that perform an image processing operation according to the first embodiment of the present invention. FIG. 2 is an enlarged cross-sectional view showing a main part of a capsule endoscope that generates predetermined image information to be processed in the image processing apparatus of the present embodiment. FIG. 3 is a block diagram showing a schematic internal configuration of a capsule endoscope apparatus that supplies predetermined image information to the image processing apparatus of the present embodiment. FIG. 4 is a diagram showing an example of use of the capsule endoscope apparatus that supplies predetermined image information to the image processing apparatus of the present embodiment. FIG. 5 is a timing chart showing an example of signals output from the capsule endoscope shown in FIG. FIG. 6 is an explanatory view illustrating position detection of the capsule endoscope shown in FIG. FIG. 7 is an enlarged cross-sectional view showing a main part of the antenna unit when the capsule endoscope apparatus shown in FIG. 3 is used. FIG. 8 is an explanatory diagram for explaining a shield jacket when the capsule endoscope apparatus shown in FIG. 3 is used. FIG. 9 is an explanatory diagram illustrating a state in which the external device of the capsule endoscope apparatus illustrated in FIG. 3 is attached to the subject. FIG. 10 is a block diagram showing an electrical configuration of the capsule endoscope shown in FIG. FIG. 11 is a flowchart showing an image processing operation according to the present embodiment. FIG. 12 is a diagram illustrating an example of a histogram in the feature space created by processing performed by the control unit. FIG. 13 is a diagram illustrating an example of a cluster in the feature space created by processing performed by the control unit. FIG. 14 is a diagram illustrating a state in which clusters having an area or volume in the feature space that is less than a predetermined threshold are deleted from the clusters illustrated in FIG. FIG. 15 is a flowchart illustrating an example of processing for integration or separation determination of two or more clusters in contact with each other, which is processing performed by the control unit. FIG. 16 is a graph showing the change in the time direction for each subset of the feature value μgp of the regions classified into the gastric mucosa class and the villi class. FIG. 17 is a graph when smoothing processing in the time direction is performed on the feature value μgp shown in the graph of FIG. 16. FIG. 18 is a flowchart illustrating an example of a lesion detection process that is a process performed by the control unit.

  As shown in FIG. 3, a capsule endoscope apparatus 1 that supplies predetermined image information to an image processing apparatus according to an embodiment of the present invention includes a capsule endoscope 3, an antenna unit 4, and an external apparatus 5. The main part is configured.

  The capsule endoscope 3 as a medical device, which will be described in detail later, is placed in the body cavity by being swallowed into the body cavity from the mouth of the patient 2 as the subject, and then advances in the digestive tract by peristaltic movement. It is formed in a shape and has an imaging function for imaging the inside of the body cavity and generating the captured image information, and a transmission function for transmitting the captured image information to the outside of the body. The antenna unit 4 includes a plurality of receiving antennas 11 that are installed on the body surface of the patient 2 and receive captured image information transmitted from the capsule endoscope 3, as will be described in detail later. The external device 5 has an outer shape formed in a box shape, and will be described in detail later. Various processing of captured image information received by the antenna unit 4, recording of captured image information, and captured image display based on captured image information. Etc. A liquid crystal monitor 12 that displays the captured image on the surface of the exterior of the external device 5 and an operation unit 13 that provides operation instructions for various functions are provided.

  Further, the external device 5 is provided with an LED for displaying a warning regarding the remaining amount of the battery for the driving power source and an operation unit 13 including a switch such as a power switch on the surface of the exterior. In addition, a calculation execution unit using a CPU and a memory is provided inside the capsule endoscope 3. For example, image processing described later is performed on captured image information received and recorded in the calculation execution unit. May be configured to be executed.

  The external device 5 is detachably attached to the body of the patient 2 and is attached to the cradle 6 as shown in FIG. The terminal device is detachably connected. For example, a personal computer is used as the terminal device 7, a terminal main body 9 having various data processing functions and storage functions, a keyboard 8 a and a mouse 8 b for inputting various operation processes, and a display for displaying various process results. 8c. The terminal device 7 has, as a basic function, for example, captured image information recorded in the external device 5 via the cradle 6 and a rewritable memory built in the terminal body 9 or the terminal body 9. It has a function of performing image processing for writing and recording in a portable memory such as a rewritable semiconductor memory that is detachable and displaying the recorded image information on the display 8c. The captured image information stored in the external device 5 may be taken into the terminal device 7 by a USB cable or the like instead of the cradle 6.

  The image processing performed by the terminal device 7 is included in the terminal body 9 as processing for selecting an image to be displayed according to elapsed time from captured image information captured and recorded from the external device 5 and image processing described later, for example. This is performed in the control unit 9a. The control unit 9a includes a CPU (Central Processing Unit) and the like, and can hold the processing result temporarily in a register (not shown) or the like, for example, when performing the processing as described above.

  Next, the external shape and internal structure of the capsule endoscope 3 will be described with reference to FIG. The capsule endoscope 3 has a substantially hemispherical cover member 14a formed by an exterior member 14 having a U-shaped cross section and a transparent member that is watertightly attached to the open end of the exterior member 14 with an adhesive. And have. Therefore, the exterior of the capsule endoscope 3 is formed to have a watertight structure and a capsule shape in a state where the exterior member 14 and the cover member 14a are connected.

  A capsule-shaped internal hollow portion having the exterior member 14 and the cover member 14a, and an observation site image incident through the cover member 14a is taken into a portion corresponding to the approximate center of the hemispherical arc of the cover member 14a. The objective lens 15 is housed and disposed in the lens frame 16. A charge coupled device (hereinafter referred to as a CCD) 17 that is an image pickup device is disposed at the imaging position of the objective lens 15. Further, around the lens frame 16 that houses the objective lens 15, four white LEDs 18 that emit and emit illumination light are arranged on the same plane (only two LEDs are shown in the figure). ing). In the hollow portion of the exterior member 14 on the rear end side of the CCD 17, the CCD 17 is driven to generate a photoelectrically converted imaging signal, and the imaging signal is subjected to predetermined signal processing to generate a captured image signal. A processing circuit 19 that performs an imaging process to be performed and an LED driving process to control the operation of lighting / non-lighting of the LED 18, and a captured image signal generated by the imaging process of the processing circuit 19 is converted into a wireless signal and transmitted. A communication processing circuit 20 for transmitting, a transmission antenna 23 for transmitting a radio signal from the communication processing circuit 20 to the outside, a plurality of button-type batteries 21 for supplying power for driving the processing circuit 19 and the communication processing circuit 20, and Is arranged.

  The CCD 17, the LED 18, the processing circuit 19, the communication processing circuit 20, and the transmission antenna 23 are arranged on a substrate (not shown), and the substrates are connected by a flexible substrate (not shown). The processing circuit 19 includes an arithmetic circuit (not shown) for performing image processing to be described later. That is, as shown in FIG. 3, the capsule endoscope 3 includes an imaging device 43 having the CCD 17, the LED 18, and the processing circuit 19, a transmitter 37 having the communication processing circuit 20, and a transmission antenna 23. Have

  Next, a detailed configuration of the imaging device 43 of the capsule endoscope 3 will be described with reference to FIG. The imaging device 43 is transferred from the CCD 17, an LED driver 18 A that controls the operation of lighting / non-lighting of the LED 18, a CCD driver 17 A for controlling the driving of the CCD 17 to transfer the photoelectrically converted charges, and the CCD 17. A processing circuit 19A that generates an image pickup signal using electric charges and generates a picked-up image signal by performing predetermined signal processing on the image pickup signal, the LED driver 18A, the CCD driver 17A, the processing circuit 19A, and a transmitter 37 includes a switch unit for supplying driving power from the battery 21 and a timing generator 19B for supplying a timing signal to the switch unit and the CCD driver 17A. The switch unit includes a switch 19C for turning on / off the power supply from the battery 21 to the LED driver 18A, and a switch 19D for turning on / off the power supply to the CCD 17, the CCD driver 17A, and the processing circuit 19A. , And a switch 19E for turning on / off the power supply to the transmitter 37. The timing generator 19B is always supplied with driving power from the battery 21.

  In the imaging device 43 of the capsule endoscope 3 having such a configuration, when the switch 19C, the switch 19D, and the switch 19E are in an off state, each part other than the timing generator 19B is in a non-operating state. When a timing signal is output from the timing generator 19B, the switch 19D is turned on, whereby the CCD 17, the CCD driver 17A, and the processing circuit 19A supplied with power from the battery 21 are in an operating state.

  At the initial stage of the driving of the CCD 17, the electronic shutter of the CCD 17 is operated to remove unnecessary dark current, and then the timing generator 19B turns on the switch 19C to drive the LED driver 18A to light the LED 18 and expose the CCD 17. To do. The LED 18 is turned on for a predetermined time required for exposure of the CCD 17 and then turned off when the switch 19C is turned off to reduce power consumption.

  Charges stored within the predetermined time when the CCD 17 is exposed are transferred to the processing circuit 19A under the control of the CCD driver 17A. The processing circuit 19A generates an imaging signal based on the charges transferred from the CCD 17, and performs predetermined signal processing on the imaging signal to generate an endoscope image signal. For example, when the signal transmitted from the transmitter 37 is an analog wireless system, the processing circuit 19A generates an analog imaging signal in which the composite synchronization signal is superimposed on the CDS output signal, and then stores the analog imaging signal. The image is output to the transmitter 37 as an endoscope image signal. Further, for example, when the signal transmitted from the transmitter 37 is a digital wireless system, the processing circuit 19A is a digital signal obtained by further performing a coding process such as scrambling on the serial digital signal generated by the analog / digital converter. A captured image signal is generated, and the digital captured image signal is output to the transmitter 37 as an endoscope image signal.

  The transmitter 37 modulates an analog captured image signal or a digital captured image signal, which is an endoscope image signal supplied from the processing circuit 19A, and wirelessly transmits the signal from the transmission antenna 23 to the outside. At this time, the switch 19E is turned on / off by the timing generator 19B so that the driving power is supplied to the transmitter 37 only at the timing when the captured image signal is output from the processing circuit 19A.

  Note that the switch 19E may be controlled so that driving power is supplied to the transmitter 37 after a predetermined time has elapsed since the captured image signal was output from the processing circuit 19A. Further, the switch 19E is provided in the capsule endoscope 3 to detect a predetermined pH value by a pH sensor (not shown), to detect a humidity higher than a predetermined value by a humidity sensor (not shown), a pressure sensor (not shown), or not shown. When the signal is output from the timing generator 19B based on a detection result such as detection of pressure or acceleration exceeding a predetermined value by the acceleration sensor, the transmitter 37 is supplied with power when inserted into the body cavity of the patient 2 as the subject. It is also possible to have a configuration that is controlled so as to supply.

  The imaging device 43 of the capsule endoscope 3 normally captures two images per second (2 frames per second = 2 fps). For example, in the case of an examination of the esophagus, 15 to 30 images per second. Image (15 fps to 30 fps). Specifically, the capsule endoscope 3 is provided with a timer circuit (not shown). By this timer circuit, for example, when the timer count is within a predetermined time, high-speed imaging with a large number of images per second is performed, and the predetermined time has elapsed. After that, the drive of the imaging device 43 is controlled so as to achieve low-speed imaging with a small number of images taken per second. Alternatively, when the capsule endoscope 3 is turned on, a timer circuit is activated and, for example, the time until the patient 2 passes through the esophagus immediately after swallowing is imaged at high speed. It is also possible to control the drive of the device 43. Further, a capsule endoscope for low-speed imaging and a capsule endoscope for high-speed imaging may be provided separately and used separately according to the observation target region.

  Next, the antenna unit 4 installed on the body surface of the patient 2 will be described. As shown in FIG. 4, when performing an endoscopic examination by swallowing the capsule endoscope 3, the patient 2 wears a jacket 10 in which an antenna unit 4 including a plurality of receiving antennas 11 is installed. As shown in FIG. 7, this antenna unit 4 has a plurality of receiving antennas 11 having unidirectional directivities, such as patch antennas used for GPS, in the direction of the body of the patient 2. Place it facing. That is, since the capsule body 3D of the capsule endoscope 3 is placed in the body, the plurality of antennas 11 are arranged so as to surround the capsule body 3D in the body. By using this highly directional antenna 11, the antenna 11 is less susceptible to interference from radio waves from other than the capsule body 3D in the body.

  As shown in FIG. 8, the jacket 10 covers the antenna unit 4 installed on the body surface of the patient 2 and an electromagnetic shielding fiber so as to cover the main body 5D of the external device 5 installed on the waist of the patient 2 with a belt. And a shield jacket 72 formed of As the electromagnetic shield fiber forming the shield jacket 72, metal fiber, metal chemical fiber, copper sulfide-containing fiber, or the like is used. The shield jacket 72 is not limited to a jacket shape, and may be, for example, a vest or a one-piece shape.

  As an example of attaching the external device 5 to the shield jacket 72, as shown in FIG. 9, a key hole 74 is provided in the external main body 5D of the external device 5, and a key 75 provided in the shield jacket 72 is attached to the key hole. By being inserted into 74, the belt 73 can be detachably mounted. Alternatively, a pocket (not shown) is simply provided in the shield jacket 72, and the external main body 5D is stored in the pocket, or a magic tape (registered trademark) is installed on the external main body 5D and the shield jacket 72 of the external device 5, and the magic You may attach and fix with a tape (trademark).

  That is, by attaching the shield jacket 72 to the body on which the antenna unit 4 is disposed, the external radio wave with respect to the antenna unit 4 is shielded and shielded from being affected by interference from external radio waves.

  Next, the configuration of the antenna unit 4 and the external device 5 will be described with reference to FIG. The antenna unit 4 includes a plurality of reception antennas 11a to 11d that receive radio signals transmitted from the transmission antenna 23 of the capsule endoscope 3, and an antenna changeover switch 45 that switches the antennas 11a to 11d. The external device 5 converts a radio signal from the antenna changeover switch 45 into a captured image signal, performs reception processing such as amplification and amplification, and performs predetermined signal processing on the captured image signal supplied from the reception circuit 33. And a signal processing circuit 35 for generating a captured image display signal and captured image data; a liquid crystal monitor 12 for displaying a captured image based on the captured image display signal generated by the signal processing circuit 35; It comprises a memory 47 for storing captured image data generated by the signal processing circuit 35 and an antenna selection circuit 46 for controlling the antenna selector switch 45 according to the magnitude of the radio signal received by the receiving circuit 33.

  A plurality of receiving antennas 11 shown as receiving antennas 11 a to 11 d in the figure of the antenna unit 4 receive radio signals transmitted from the transmitting antenna 23 of the capsule endoscope 3 with a certain radio wave intensity. The plurality of reception antennas 11a to 11d are sequentially switched to the reception antennas that receive the radio signals, with the antenna switch 45 being controlled by an antenna selection signal from the antenna selection circuit 46 of the external device 5. That is, the radio signal received for each of the receiving antennas 11 a to 11 d sequentially switched by the antenna selector switch 45 is output to the receiver 33. The receiver 33 detects the reception intensity of the radio signal for each of the receiving antennas 11a to 11d, calculates the positional relationship between the receiving antennas 11a to 11d and the capsule endoscope 3, and demodulates the radio signal. The captured image signal is output to the signal processing circuit 35 by processing. The antenna selection circuit 46 is controlled by the output from the receiver 33.

  The operation of the antenna selector switch 45 by the antenna selection circuit 46 will be described. As shown in FIG. 5, the radio signal transmitted from the capsule endoscope 3 has an intensity reception that is a transmission period of a reception intensity signal indicating the reception intensity of the radio signal in the transmission period of one frame of the captured image signal. It is assumed that the period and the video signal period that is the transmission period of the captured image signal are sequentially repeated and transmitted.

  The antenna selection circuit 46 is supplied with the reception strength of the reception strength signals received by the reception antennas 11 a to 11 d via the reception circuit 33. The antenna selection circuit 46 compares the received signal strengths of the antennas 11a to 11d supplied from the receiver 33, and receives the picked-up image signal in the video signal period. The antenna 11i (i = a to d) having the highest strength signal is determined, and a control signal for switching the antenna switching circuit 45 to the antenna 11i is generated and output. Thereby, when the reception intensity of the reception intensity signal of the other antenna is higher than that of the antenna currently receiving the image signal, the reception antenna in the video signal period is switched from the next frame.

  In this way, each time a radio signal is received from the capsule endoscope 3, the received intensity of the captured image signal or the received intensity signal is compared, and the reception intensity is maximized by the antenna selection circuit 46 that receives the comparison result. The antenna 11i is designated as an image signal receiving antenna. Thereby, even if the capsule endoscope 3 moves in the body of the patient 2, it is possible to receive an image signal acquired by the antenna 11 that can detect a signal having the highest reception intensity at the moving position. In addition, since the moving speed of the capsule endoscope 3 in the body is divided into a very slow part and a fast part, the antenna switching operation is not always performed once per imaging operation, and there are a plurality of speeds in the high-speed imaging mode. The antenna switching operation may be performed once for each imaging operation.

  Since the capsule endoscope 3 is moving inside the patient 2, the capsule endoscope 3 sends a detection result signal that is a result of detecting the radio wave intensity from the external device 5 at an appropriate time interval, and the capsule endoscope 3 is based on the signal. You may make it update the output when the type | mold endoscope 3 transmits. In this way, even when the capsule endoscope 3 moves inside the patient 2, it is possible to set an appropriate transmission output, prevent wasteful consumption of the energy of the battery 21, and transmission / reception of signals. The state can be maintained in an appropriate state.

  Next, a method for acquiring information indicating the positional relationship between the plurality of receiving antennas 11 and the capsule endoscope 3 will be described with reference to FIG. In FIG. 6, the case where the capsule endoscope 3 is set to the origin of the three-dimensional coordinates X, Y, and Z will be described as an example. In order to simplify the description among the plurality of receiving antennas 11a to 11d, three receiving antennas 11a, 11b, and 11c are used, and the distance between the receiving antenna 11a and the receiving antenna 11b is Dab. The distance between 11b and the receiving antenna 11c is Dbc, and the distance Dac between the receiving antenna 11a and the receiving antenna 11c. Further, the receiving antennas 11a to 11c and the capsule endoscope 3 have a predetermined distance relationship.

  When the radio signal having a constant transmission intensity transmitted from the capsule endoscope 3 is received by each receiving antenna 11j (j = a, b, c), the reception intensity is as follows. This is a function of the distance Li (i = a, b, c) from the transmission antenna 23) of the endoscope 3. Specifically, it depends on the distance Li accompanied by the radio wave attenuation. Therefore, the distance Li between the capsule endoscope 3 and each receiving antenna 11j is calculated from the reception intensity received by the receiving antenna 11j of the radio signal transmitted from the capsule endoscope 3. In calculating the distance Li, relational data such as radio wave attenuation due to the distance between the capsule endoscope 3 and the receiving antenna 11j is set in the antenna selection circuit 46 in advance. The calculated distance data indicating the positional relationship between the capsule endoscope 3 and each receiving antenna 11j is stored in the memory 47 as position information of the capsule endoscope 3. Based on the captured image information stored in the memory 47 and the position information of the capsule endoscope 3, it is useful for setting the position of the endoscopic observation findings in the image information processing method described later by the terminal device 7.

  Next, an image processing operation in the image processing apparatus of this embodiment will be described.

  In the present embodiment, the image of the image in the body cavity imaged by the capsule endoscope 3 is the number of dots in the x-axis direction ISX × the number of dots in the y-axis direction ISY (1 ≦ ISX, 1 ≦ ISY). It is a value that satisfies, for example, consists of three planes of ISX = 300, ISY = 300), R (red), G (green), and B (blue). That is, assume a value from 0 to 255. In the embodiment of the present invention, the i-th image in N images (1 ≦ N) continuously captured in time series is denoted as Ii (1 ≦ i ≦ N). Furthermore, in the present embodiment, the vth pixel (1 ≦ v ≦ ISX × ISY) in each plane of the image Ii is denoted as riv, giv, and biv, respectively.

  In addition, the image processing operation in the image processing apparatus of the present embodiment is performed as a process in the control unit 9a included in the terminal body 9 of the terminal device 7 described above.

  First, the control unit 9a uses the capsule endoscope 3 to capture N sheets of images that are continuously captured in time series based on the sampling values k (k = 1, 2,...) Preset in the terminal device 7. In the image, sampling is performed for every k-th image Ik, I2k,..., Ink (where nk is an integer satisfying 1 ≦ nk ≦ N) (step S1 in FIG. 11). For example, when the capsule endoscope 3 captures two images per second and the sampling value k is set to 20, the control unit 9a uses the N images captured by the capsule endoscope 3. In the image, the image is sampled one by one every 10 seconds from the imaging start time.

  Then, the control unit 9a uses the sampled image Ink of (N / k) (N / k is an integer obtained by rounding down) as a sample image group, and then each image Ink of the sample image group. In contrast, the following processing is performed. The sample image group may include (n + 1) images including the first image I1 in addition to (N / k) images Ink.

  The control unit 9a performs, for example, noise removal and inverse γ correction by median filtering on the Rnk, Gnk, and Bnk planes that make up the input image Ink. In order to exclude them from the processing target, detection is performed by processing based on a threshold value (step S2 in FIG. 11). The processing based on the threshold value is, for example, a dark pixel when the density values of rnkv, gnkv, and bnkv are all 10 or less, and all the density values of rnkv, gnkv, and bnkv are 230 or more. If there is a halation pixel, the process is performed.

  The control unit 9a divides the input image Ink into, for example, a plurality of 8 × 8 rectangular areas (step S3 in FIG. 11). In the following description, one of the rectangular areas divided by the control unit 9a is indicated as Hp (p is an integer of 1 or more).

  Thereafter, the control unit 9a determines the average value of gnkv / rnkv (hereinafter referred to as μgp) and the average value of bnkv / gnkv (hereinafter referred to as μgp), which are values based on the ratio of the RGB values of each pixel in each region Hp of the image Ink. Hereinafter, two feature amounts indicating the chromaticity of the image, which are composed of μbp), are calculated (step S4 in FIG. 11).

  Furthermore, the control unit 9a discretizes the feature quantities μgp and μbp obtained in each region Hp, and creates a histogram in the feature space based on the frequency of occurrence of the discretized feature quantities μgp and μbp (FIG. 11). Step S5). Specifically, the control unit 9a treats the values of the feature values μgp and μbp as, for example, all values of 1 or more as 1 while taking values from 0 to 1, respectively, and further from 0 to 1 Discretization and creation of a histogram are performed by rounding values obtained by multiplying the feature values μgp and μbp by 80 times to integer values.

  The control unit 9a smoothes the histogram by applying, for example, an average value filter of a predetermined size to each of the discretized feature quantities μgp and μbp (step S6 in FIG. 11). Note that the histogram created when the control unit 9a performs the above processing is as shown in FIG. 12, for example.

  Next, in the histogram created by performing the above-described processing on all the (N / k) images as the sample image group, the control unit 9a has an element with the maximum occurrence frequency (μgp, μbp). That is, the peak point is detected (step S7 in FIG. 11). Specifically, the control unit 9a extracts nine elements including one element and elements in the vicinity of the one element from the generated histogram, and then generates an occurrence among the extracted nine elements. The element with the highest frequency is detected as a peak point.

  In the created histogram, the control unit 9a uses the gradient vector to indicate which of the detected peak points each element other than (μgp, μbp) = (0, 0) is directed to. For example, a Valley-Seeking method is used as an analysis method based on the analysis method (step S8 in FIG. 11). Then, the control unit 9a performs unsupervised clustering processing for each element in the histogram, which is processing for setting an element having a gradient vector toward the same peak point as an element belonging to the same cluster (step S9 in FIG. 11). Each cluster created when the control unit 9a performs the unsupervised clustering process is as shown in FIG. 13, for example.

  When the controller 9a detects that two or more cluster boundaries are in contact with each other in the unsupervised clustering process described above, the controller 9a performs integration or separation determination of the two or more clusters as described below. Further processing may be performed.

  In that case, the control unit 9a extracts, among the created clusters, nine elements including eight neighboring elements, by extracting elements including two or more clusters, that is, belongs to the boundary of two or more clusters. It is determined that the two or more clusters are in contact with each other (step S21 in FIG. 15). Further, the control unit 9a extracts one element having the lowest occurrence frequency from the elements belonging to the boundary between the two or more clusters to be processed, and sets the occurrence frequency in the one element to μmin. (Step S22 in FIG. 15). Further, the control unit 9a extracts one peak point having the highest occurrence frequency from each peak point in the two or more clusters to be processed, and sets the occurrence frequency at the one peak point to μmax ( Step S23 in FIG. 15).

  After extracting μmin and μmax, the control unit 9a compares the value of μmin / μmax with the threshold value μthr. When the control unit 9a detects that the value of μmin / μmax is larger than the threshold value μthr (step S24 in FIG. 15), the control unit 9a determines that the two or more clusters to be processed are separate clusters, and the 2 The above clusters remain separated (step S25 in FIG. 15). When the control unit 9a detects that the value of μmin / μmax is equal to or less than the threshold value μthr (step S24 in FIG. 15), the control unit 9a determines that the two or more clusters to be processed belong to the same cluster, The two or more clusters are integrated, and an integrated cluster is created in which the peak point of the occurrence frequency μmax is a new peak point (step S26 in FIG. 15). Note that the threshold value μthr described above is, for example, 0.1 in the present embodiment.

  After performing the above-described unsupervised clustering process (the process shown in step S9 in FIG. 11), the control unit 9a acquires the cluster information of each generated cluster (step S10 in FIG. 11). The cluster information acquired by the control unit 9a includes, for example, a cluster number, an element that is a peak point of each cluster, an area and a volume in each feature space of each cluster, and feature quantities μgp and μbp in each cluster. It is information such as an average value vector.

  Thereafter, based on the acquired cluster information, the control unit 9a deletes clusters whose area or volume in the feature space is less than a predetermined threshold, for example, as shown in FIG. 14 (step S11 in FIG. 11).

  Further, the control unit 9a creates a discriminator such as a function based on a linear discriminant function or a Bayes' theorem, for example, created from the mean value vector of the feature values μgp and μbp in each cluster remaining in the feature space and the teacher data set. By using it, it is determined to which class each cluster remaining in the feature space belongs (step S12 in FIG. 11). In the present embodiment, the classes are assumed to be four classes consisting of gastric mucosa, villi, stool and foam. In the present embodiment, it is assumed that the teacher data set is a plurality of images constituting the four classes of teacher data.

  Then, the control unit 9a classifies each cluster remaining in the feature space into four classes of gastric mucosa, villi, stool, and foam, and classifies clusters that cannot be classified into the four classes as unknown classes (FIG. 11 step S13).

  Here, the specific example of the process shown to step S12 of FIG. 11 and step S13 is explained in full detail below. It is assumed that the control unit 9a performs the processing described below for all the clusters remaining in the feature space.

In the identification and classification of the four classes described above, the prior probability that one class ωa (a = 1, 2,..., C, C indicates the number of classes) occurs is P (ωa), and remains in the feature space. The feature vector determined from the feature quantities μgp and μbp in each cluster is x , the probability density function based on the occurrence probability of the feature vector x from all classes is p ( x ), and the feature vector x from one class ωa When the state-dependent probability density (multivariate normal probability density) function based on the occurrence probability is p ( x | ωa), the posterior probability P (ωa | x ) where the generated feature vector x belongs to one class ωa is calculated. Is calculated as the following formula (1).

・・・（１）

なお、状態依存確率密度関数ｐ（ｘ｜ωａ）及び確率密度関数ｐ（ｘ）は、下記数式（２）及び数式（３）として示される。

... (1)

Note that the state-dependent probability density function p ( x | ωa) and the probability density function p ( x ) are expressed as the following formula (2) and formula (3).

・・・（２）

・・・（３）

なお、上記数式（２）及び数式（３）において、ｄはｘの特徴量の個数と同数である次元数を示し、μａ及びΣａはクラスωａにおける特徴ベクトルｘの平均ベクトル及び一のクラスωａにおける分散共分散行列を示すものとする。また、（ｘ−μａ）^ｔは（ｘ−μａ）の転置行列を示し、｜Σａ｜はΣａの行列式を示し、Σａ^−１はΣａの逆行列を示すものとする。さらに、説明の簡単のため、事前確率Ｐ（ωａ）は、全クラスにおいて等しい値をとると仮定し、また、確率密度関数ｐ（ｘ）は、上記数式（３）により全クラス共通の関数として表されるものとする。

... (2)

... (3)

In Equations (2) and (3), d indicates the number of dimensions that is the same as the number of feature quantities of x , μa and Σa are the average vector of feature vectors x in class ωa and in one class ωa. Let us denote the variance-covariance matrix. Also, ( x − μa ) ^t represents a transposed matrix of ( x − μa ), | Σa | represents a determinant of Σa, and Σa− ¹ represents an inverse matrix of Σa. Further, for simplicity of explanation, it is assumed that the prior probability P (ωa) takes an equal value in all classes, and the probability density function p ( x ) is a function common to all classes according to the above equation (3). Shall be represented.

Along with the statistical discriminator based on the Bayes' theorem as described above, the average vector μa and the variance-covariance matrix Σa used as the classification criteria are elements constituting a parameter in one class ωa, and are the first image I1. Is input to the terminal device 7 based on a plurality of images constituting four classes of teacher data consisting of gastric mucosa, villi, feces, and bubbles, each time determined in each region of the image After being calculated in advance for each class from the feature vector x , it is recorded in the terminal device 7 as an initial value. At this time, the control unit 9a may estimate the parameter by adding the feature vector of each class in the image Ii to the feature vector in the teacher data of each class.

The average vector μa is comprised average of two features, each having the feature vector x, and a vector having the same number of dimensions as the feature vector x. That is, when the feature vector x is expressed as x = (μgp, μbp), the average vector μa is an average value of each of the two feature amounts of the feature vector x , μ (μgp) and μ (μbp). And is expressed as μa = (μ (μgp), μ (μbp)). The variance-covariance matrix Σa is a matrix indicating the distribution and spread of the distribution of the feature vector x belonging to one class ωa. For the dimension number d that is the same as the number of feature quantities of the feature vector x , d It shall be expressed as a xd matrix.

Control unit 9a, the posterior probability P that belong to the generated feature vector x Class .omega.1 | a (.omega.1 x), the posterior probability P that belong to the generated feature vector x Class .omega.2 | a (.omega.2 x), generated feature vector x posterior probability P that belong to but class [omega] 3 | a ([omega] 3 x), the posterior probability P generated feature vector x belongs to a class .omega.4 | formula and (.omega.4 x), from the equation (1) based on Bayes' Theorem ( 3) to calculate each. Then, the control unit 9a identifies that the feature vector x belongs to the class ωa that gives the maximum posterior probability P1 (ωa | x ) among these four posterior probabilities, and based on the identification result, the feature vector x Is classified into class ωa, and the value of probability density function p1 ( x | ωa) giving the maximum posterior probability P1 (ωa | x ) is calculated.

Then, in order to determine whether or not the classification result of one cluster classified into the class ωa is accurate in the above processing, the control unit 9a performs processing based on the distance from the average value, that is, Further processing based on a threshold is performed on the value of the probability density function p1 ( x | ωa) that gives the maximum posterior probability P1 (ωa | x ).

Specifically, the control unit 9a first determines, for example, the standard deviation σ of the feature value μgp with respect to the average value μ (μgp) of the feature value μgp among the average values of the two feature values of the average vector μa. A threshold vector xb1 including a value obtained by adding a product of (μgp) and a multiplication coefficient α as a predetermined constant is determined. Such a threshold vector xb1 is represented, for example, by the following mathematical formula (4), and in the present embodiment, the value of the multiplication coefficient α is 1.5.

xb1 = (μ (μgp) + α × σ (μgp), μbp) (4)

When the threshold vector xb1 by the equation (4) is determined, the control unit 9a, the threshold vector xb1 above equation (1), is substituted as x in Equation (2) and Equation (3), classified one cluster The value of the probability density function p ( xb1 | ωa) is calculated as the threshold value of the determined class ωa.

When the control unit 9a detects that the value of p1 ( x | ωa) is larger than the value of p ( xb1 | ωa), the classification result obtained by classifying one cluster into the class ωa in the above-described process is accurate. Judge that there is.

Further, when the control unit 9a detects that the value of p1 ( x | ωa) is equal to or less than the value of p ( xb1 | ωa), the classification result obtained by classifying one cluster into the class ωa is not valid in the above-described processing. Judged to be accurate and classify one cluster into an unknown class.

  Further, after performing the above-described class classification, the control unit 9a generates a subset for each predetermined interval t based on the captured timing from (N / k) images Ink as a sample image group (FIG. 11 step S14). For example, when images every 10 seconds from the imaging start time are sampled one by one (k = 20 is set) and the interval t is set to 180, the control unit 9a sets the sample image group as In (N / k) images Ink, images included every 1800 seconds from the imaging start time are set as a subset.

  Thereafter, in each image Ink included in each subset, the control unit 9a includes class distribution information indicating the distribution of class types generated in the feature space, and feature amounts μgp and μbp included in the clusters generated in the feature space. The feature amount distribution information indicating the distribution state for each class is acquired (step S15 in FIG. 11). Then, the control unit 9a calculates an average value vector and a variance covariance matrix as classification criteria for classifying images based on the feature amount distribution information, and then uses the average value vector and the variance covariance matrix. The subset classifier is configured (step S16 in FIG. 11). Further, the control unit 9a performs classification for each of the N images by using the classifier configured for each subset by the above-described processing (step S17 in FIG. 11). By performing such processing, the control unit 9a optimally sets the parameter of the multivariate normal probability density function for defining the distribution of each class for each subset in the statistical classifier described above. Can do.

  The control unit 9a performs the above-described processing from step S1 in FIG. 11 to step S17 in FIG. 11 to classify images captured by the capsule endoscope 3 with high accuracy and high speed. be able to.

  Note that the control unit 9a may not calculate the average value vector and the variance-covariance matrix for classes that have not occurred in one subset based on the acquired feature amount distribution information.

  Further, the processing performed by the control unit 9a as described above is not limited to that performed after N images have been captured. For example, immediately after a subset image based on the interval t is captured. It may be performed at any time.

  Further, the control unit 9a obtains, for example, a predetermined size in which the feature amount distribution information acquired in the process shown in step S15 in FIG. A smoothing process, which is a process using the smoothing filter, may be performed. Then, the control unit 9a performs such a smoothing process, so that the portion of the feature amount distribution information in which the variation of the feature amount μgp or μbp is significant, or the variation of the feature amount μgp or μbp is maximized. From this portion, it is possible to specify the imaging site and the lesion site.

  An example of a method for the control unit 9a to specify the imaging site and the lesion site based on the above-described smoothing processing result will be described below.

  For example, based on the feature amount distribution information acquired by the control unit 9a in the process shown in step S15 of FIG. 11, the change in the time direction for each subset of the feature amount μgp included in the region classified into the gastric mucosa class and the villus class is expressed. In this case, the change is shown as a graph as shown in FIG.

  Originally, the color tone of the biological mucous membrane or the like does not frequently change in the time direction when the same imaging region is continuously imaged. Therefore, it is considered that the value of the feature value μgp (or feature value μbp) obtained from temporally close images takes a similar value.

  Then, as a filter of a predetermined size set in advance based on the above-described viewpoint, for example, a time direction with respect to the feature amount μgp shown as a graph as shown in FIG. 16 using a one-dimensional equal weighting filter of size 20 When the control unit 9a performs the smoothing process, the change in the time direction for each subset of the feature amount μgp is shown as a graph as shown in FIG. The graphs shown in FIG. 16 and FIG. 17 represent the change in the time direction for each subset of the feature value μgp, but the feature value μbp also shows the change in the time direction for each subset. It is possible to draw a graph.

  In addition, when the control unit 9a detects that there is a time zone in which the value of the feature value μgp or the feature value μbp varies significantly even after the smoothing process is performed, in the time zone, for example, imaging It is determined that a change in the site has been confirmed. Alternatively, when the control unit 9a detects that there is a time zone in which the value of the feature value μgp or the feature value μbp varies significantly even after performing the smoothing process, It is determined that the presence of bile bubbles or lesions has been confirmed.

  For example, when the capsule endoscope 3 moves from the stomach to the small intestine (duodenum), digestive fluid such as bile is secreted, so that a yellowish subject image is captured. Therefore, when the control unit 9a detects the presence of a time zone in which the value of the characteristic amount μbp in the gastric mucosa class and the villi class shows a significant decrease, the imaging site of the capsule endoscope 3 is in the stomach in the time zone. It is possible to identify that it has changed from small to large intestine.

  In addition, for example, when the capsule endoscope 3 passes through a bleeding site as one of the lesion sites, a reddish subject image is captured due to the presence of blood in the bleeding site. Therefore, when the control unit 9a detects the presence of a time zone in which the value of the feature value μgp shows a significant decrease in the same class, it indicates that the capsule endoscope 3 has imaged the lesion site in the time zone. Can be identified.

  And the control part 9a may perform the process described further in addition to the series of processes shown from step S1 of FIG. 11 to step S17 of FIG. 11 mentioned above. In the processing described below, it is assumed that the class distribution information acquired by the control unit 9a includes five classes of information including gastric mucosa, villi, feces, bubbles, and unknown. The class distribution information described above is not limited to the information including the five classes of information described above. For example, the gastric mucosa and villus classes are treated as the same class as the biological mucosa class, and stool, foam, and unknown. The class may contain information that can be treated as the same class as the non-biological mucosa class, or it may contain information in which the white-toned biological mucosa and the yellow-toned biological mucosa are treated as the same class It may be a thing.

  After performing the process of step S17 of FIG. 11 described above, the control unit 9a divides the input i-th image Ii into, for example, a plurality of 8 × 8 rectangular areas (step S31 of FIG. 18). .

  Here, the first image and the last image of the s-th subset (assuming that s is an integer) generated in the processing up to step S17 in FIG. 11 are Is1 and Is2 (s1 and s2 are respectively). 1 ≦ s1 <s2 ≦ N), the control unit 9a is a predetermined subset satisfying s1 ≦ i <s2 or i = s2 among the subsets generated in the above-described processing. Class distribution information and feature amount distribution information are acquired (step S32 in FIG. 18).

  After that, the control unit 9a determines that each rectangular region included in the image Ii is a gastric mucosa, villi, feces, bubbles based on the class distribution information and the feature amount distribution information in a predetermined subset that satisfies s1 ≦ i <s2 or i = s2. Then, the class belonging to the five classes of unknown is identified, and classification is performed based on the identification result (step S33 in FIG. 18). Note that the control unit 9a may identify and classify a class that has not occurred in the processing up to step S17 in FIG. 11 using the feature amount distribution information estimated in advance from the teacher data. good.

  Further, based on the classification result in step S33 of FIG. 18, the control unit 9a determines whether there is a region classified as gastric mucosa or villi, that is, a region where the biological mucous membrane is imaged among the rectangular regions of the image Ii. Determine whether or not. Then, when there is an area where the biological mucous membrane is imaged among the rectangular areas of the image Ii (step S34 in FIG. 18), the control unit 9a determines that the biological mucous membrane is imaged in the image Ii. A process for detecting a lesion site by comparing a value based on all the feature values in the region with the values of the feature values μgp and μbp of one region in which it is determined that the biological mucous membrane is imaged in the image Ii, that is, As the lesion detection processing, for example, processing for detecting bleeding or redness is performed (step S35 in FIG. 18).

  Specifically, for example, the control unit 9a captures the biological mucous membrane in the image Ii and the values of the feature amounts μgp and μbp of one region in which the biological mucous membrane is determined to be captured in the image Ii. The average value of giv / riv (hereinafter referred to as μgi) and the average value of biv / giv (hereinafter referred to as μbi) as values based on the ratio of the RGB values of each pixel included in all the regions determined as Based on the above, a comparison between μgp and μgi and a comparison between μbp and μbi are performed to detect a region where bleeding or redness is imaged among regions where the biological mucosa is imaged in the image Ii. By performing such processing, the control unit 9a changes the bleeding or redness of the average chromaticity of all the regions in which it is determined that the biological mucous membrane is imaged in the region segmentation result with class classification. One region having the chromaticity to be exhibited can be accurately detected as a bleeding region or a redness region. Note that the control unit 9a further performs processing based on the comparison result between the feature amount of one region classified into the unknown class in the image Ii and μgi and μbi, so that the region where the biological mucous membrane is imaged in the image Ii. In addition, it may be possible to detect a region where bleeding or redness is imaged even in a region classified as an unknown class in the image Ii.

  If the above-described processing has not been completed for all the input N images (step S36 in FIG. 18), the controller 9a adds 1 to the image number i (step S37 in FIG. 18), and The processing shown in steps S31 to S36 in FIG. 18 is continued for the image.

  In addition to the area classified as the gastric mucosa or villus, the control unit 9a may determine that the villi that is yellowish due to bile or the like is also an area where the biological mucosa is imaged.

  In addition, the image processing method described above is not limited to an image that is applied only to an image captured by a capsule endoscope. For example, an insertion unit that can be inserted into a living body and has an imaging function. The present invention may be applied to an image captured by an endoscope that is configured to include:

  As described above, according to the image processing method of the present embodiment, it is possible to classify images with high accuracy and high speed for each imaging target, and further specify the imaged organ based on the classification result. As a result, the efficiency of observation by the user can be improved.

  In addition, according to the present embodiment, the lesion site detection accuracy can be increased by using the lesion detection process described above for each region classified as the region where the biological mucous membrane is imaged.

  Further, in the first and second embodiments of the present invention, the control unit 9a is described as performing a series of processes by dividing one image Ii into a plurality of rectangular regions having a size of 8 × 8. However, the present invention is not limited to this. For example, the processing may be performed by dividing the pixel by 1 × 1, that is, for each pixel, or by dividing into rectangular regions of other sizes. Processing may be performed.

  Furthermore, in the first embodiment and the second embodiment of the present invention, the control unit 9a is described as performing a series of processing by dividing one image Ii into a plurality of rectangular regions having a size of 8 × 8. However, the present invention is not limited to this. For example, the processing may be performed by dividing into regions based on the classification result in one image Ii according to edge information or the like. The processing may be performed by dividing into regions having the following shape.

  In addition, this invention is not limited to embodiment mentioned above, Of course, a various change and application are possible in the range which does not deviate from the meaning of invention.

1 is an external front view showing the external appearance of an image processing apparatus and peripheral devices that perform an image processing operation according to a first embodiment of the present invention. The principal part expanded sectional view which cut and showed a part of capsule type | mold endoscope which produces | generates the predetermined image information processed in the image processing apparatus of this embodiment. 1 is a block diagram showing a schematic internal configuration of a capsule endoscope apparatus that supplies predetermined image information to an image processing apparatus according to an embodiment. The figure which showed the usage example of the capsule type endoscope apparatus which supplies predetermined | prescribed image information to the image processing apparatus of this embodiment. The timing chart which showed an example of the signal output from the capsule type endoscope shown in FIG. Explanatory drawing explaining the position detection of the capsule type endoscope shown in FIG. The principal part expanded sectional view which showed the antenna unit at the time of using the capsule type endoscope apparatus shown in FIG. Explanatory drawing explaining the shield jacket at the time of using the capsule type endoscope apparatus shown in FIG. Explanatory drawing explaining the mounting state to the subject of the external apparatus of the capsule type endoscope apparatus shown in FIG. FIG. 3 is a block diagram showing an electrical configuration of the capsule endoscope shown in FIG. 2. 5 is a flowchart showing an image processing operation according to the present embodiment. The figure which shows an example of the histogram in the feature space produced by the process which a control part performs. The figure which shows an example of the cluster in the feature space produced by the process which a control part performs. The figure which shows a mode that the cluster whose area or volume in feature space is less than a predetermined threshold among the clusters shown in FIG. 13 is deleted. The flowchart which shows an example of the process of integration or isolation | separation determination of two or more clusters which the boundary has touched which is the process which a control part performs. The graph which shows the change in the time direction for every subset of feature-value microgp which the area | region classified into the gastric mucosa class and the villus class has. The graph at the time of performing the smoothing process to a time direction in the feature-value microgp shown by the graph of FIG. The flowchart which shows an example of the lesion detection process which is a process which a control part performs.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Capsule type endoscope apparatus, 2 ... Patient, 3 ... Capsule type endoscope, 3D ... Capsule main body, 4 ... Antenna unit, 5 ... External device, 5D ..Main unit, 6 ... cradle, 7 ... terminal device, 8a ... keyboard, 8b ... mouse, 8c ... display, 9 ... terminal main unit, 9a ... control unit, DESCRIPTION OF SYMBOLS 10 ... Jacket, 11, 11a, 11b, 11c, 11d ... Reception antenna, 12 ... Liquid crystal monitor, 13 ... Operation part, 14 ... Exterior member, 14a ... Cover member, 15 ... objective lens, 16 ... lens frame, 17 ... charge coupled device, 17A ... CCD driver, 18 ... LED, 18A ... LED driver, 19, 19A ... processing circuit, 19B: Timing generator, 19C 19D, 19E ... switch, 20 ... communication processing circuit, 21 ... battery, 23 ... transmitting antenna, 33 ... receiving circuit, 35 ... signal processing circuit, 37 ... transmitter , 43 ... Imaging device, 45 ... Antenna selector switch, 46 ... Antenna selection circuit, 47 ... Memory, 72 ... Shield jacket, 73 ... Belt, 74 ... Keyhole, 75 ···key

Claims (26)

Image signal input means for inputting image signals based on a plurality of images captured by a medical device having an imaging function;
Image dividing means for dividing each of the plurality of images into a plurality of regions based on the image signal input by the image signal input means;
Feature amount calculating means for calculating a feature amount in each of the plurality of regions divided by the image dividing means;
Cluster classification means for generating a plurality of clusters in a feature space based on the feature amount and the occurrence frequency of the feature amount, and classifying the plurality of clusters into any of a plurality of classes,
Subset generating means for generating a plurality of subsets based on the timing at which each of the plurality of images is captured using the plurality of images;
Based on the distribution state of the feature quantity in the feature space of each of the images included in the one subset generated by the subset generation means, the images included in the one subset are each in one of the plurality of classes. A classification standard calculation means for calculating a classification standard for classification,
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the classification criterion calculation unit does not calculate the classification criterion for a class in which the feature amount does not occur in one subset of the plurality of classes. .   The image processing apparatus according to claim 1, wherein the plurality of classes include at least a class relating to a biological mucosa and a class relating to a non-biological mucosa. Furthermore, based on the classification criteria, classification means for classifying each region in one image included in the one subset,
If the one image has a region classified as a class related to the biological mucous membrane based on the classification result of the one image, lesion detection is performed for each region to detect a lesion location. Means,
The image processing apparatus according to claim 3, further comprising: An image dividing step of dividing the image into a plurality of regions based on an image signal input based on an image captured by a medical device having an imaging function;
A feature amount calculating step for calculating a feature amount in each of the plurality of regions divided by the image dividing step;
A cluster classification step of generating a plurality of clusters in a feature space based on the feature amount and the occurrence frequency of the feature amount, and classifying the plurality of clusters into any of a plurality of classes;
A subset generation step of generating a plurality of subsets based on the timing at which each of the plurality of images is captured using the plurality of images;
Based on the distribution state of the feature quantity in the feature space of each image included in the one subset generated by the subset generation step, a classification criterion for classifying the images included in the one subset is calculated. A classification criteria calculation step;
An image processing method in an image processing apparatus comprising:   The image processing apparatus according to claim 5, wherein the classification criterion calculation step does not calculate the classification criterion for a class in which the feature amount does not occur in one subset of the plurality of classes. Image processing method.   7. The image processing method in the image processing apparatus according to claim 5, wherein the plurality of classes include at least a class related to a biological mucosa and a class related to a non-biological mucosa. A classification step for classifying each region in one image included in the one subset based on the classification criterion;
If the one image has a region classified as a class related to the biological mucous membrane based on the classification result of the one image, lesion detection is performed for each region to detect a lesion location. Steps,
The image processing method in the image processing apparatus according to claim 7, further comprising: Image signal input means for inputting image signals based on a plurality of images captured in time series by a medical device having an imaging function;
Image dividing means for dividing each of the plurality of images into a plurality of regions based on the image signal input by the image signal input means;
Feature amount calculating means for calculating a feature amount in each of the plurality of regions divided by the image dividing means;
Image region classification means for classifying each of the plurality of regions into any of a plurality of classes based on the feature amount;
In each of the plurality of images, representative value calculation means for calculating a representative value of at least one class of feature values;
A fluctuation detecting means for detecting the fluctuation of the representative value in time series ;
An image processing apparatus comprising: Furthermore, it has smoothing means for applying smoothing in the time series direction to each of the representative values,
10. The variation detection unit detects, based on the representative value smoothed by the smoothing unit, either an image or a time at which the variation of the representative value is maximized. The image processing apparatus described.   The image processing apparatus according to claim 9, wherein the variation in the representative value is generated based on a change in an observation site in a living body.   The image processing apparatus according to claim 9 or 10, wherein the change in the representative value is generated based on the presence of feces in a living body.   The image processing apparatus according to claim 9, wherein the variation of the representative value is generated based on the presence of bile in a living body.   The image processing apparatus according to claim 9, wherein the variation in the representative value is generated based on the presence of a lesion site in a living body.   The image processing apparatus according to claim 9, wherein the representative value is an average value of feature amounts in each of the plurality of classes. An image dividing step of dividing the image into a plurality of regions based on an image signal input based on an image captured by a medical device having an imaging function;
A feature amount calculating step for calculating a feature amount in each of the plurality of regions divided by the image dividing step;
An image region classification step for classifying each of the plurality of regions into one of a plurality of classes based on the feature amount;
In each of the plurality of images, a representative value calculating step of calculating a representative value of the feature quantity of at least one class;
A fluctuation detecting step for detecting the fluctuation of the representative value in time series ;
An image processing method in an image processing apparatus comprising: And a smoothing step of applying smoothing in a time series direction to each of the representative values,
The process of detecting either an image or a time at which the variation of the representative value is maximized based on the representative value after being smoothed by the smoothing step is performed in the variation detecting step. An image processing method in the image processing apparatus according to 16.   The image processing method in the image processing apparatus according to claim 16 or 17, wherein the variation of the representative value is generated based on a change in an observation site in a living body.   The image processing method in the image processing apparatus according to claim 16 or 17, wherein the variation of the representative value is generated based on the presence of feces in a living body.   18. The image processing method in the image processing apparatus according to claim 16, wherein the variation of the representative value is generated based on the presence of bile in a living body.   The image processing method in the image processing apparatus according to claim 16 or 17, wherein the variation of the representative value is generated based on the presence of a lesion site in a living body.   The image processing method in the image processing apparatus according to any one of claims 16 to 21, wherein the representative value is an average value of feature amounts in each of the plurality of classes.   The image processing apparatus according to claim 4, wherein the lesion detection unit is provided in the medical device.   The image processing method in the image processing apparatus according to claim 8, wherein the lesion detection step is performed in the medical device.   The image processing apparatus according to claim 9, wherein the variation detection unit performs a process of detecting either an image or a time at which the variation of the representative value detected in time series is a maximum.   The image in the image processing apparatus according to claim 16, wherein the variation detection step performs a process of detecting either an image or a time at which the variation of the representative value detected in time series is a maximum. Processing method.

Images