JP7516072B2 - IMAGE PROCESSING APPARATUS, MEDICAL IMAGE DIAGNOSIS APPARATUS, IMAGE PROCESSING METHOD, PROGRAM, AND LEARNING APPARATUS - Google Patents

Description

The present invention relates to an image processing device, a medical image diagnostic device, an image processing method, a program, and a learning device.

In the medical field, diagnoses are made using cross-sectional images (also called tomograms or medical images) acquired by various medical imaging diagnostic devices (modalities). In this diagnosis, the type of cross-sectional image (classification according to which cross-section of the subject is imaged; hereafter referred to as cross-section type) is identified, and information indicating the cross-section type is associated with the cross-sectional image and displayed or saved. In addition, image processing according to the identified cross-section type is performed on the cross-sectional image, and analysis processing is performed using the identified information. In addition, the contour of a region of interest (such as an organ or a lesion) in the cross-sectional image may be identified, and the area, size, shape, etc. of the region of interest may be used for diagnosis. Conventionally, the task of identifying the cross-section type and contour is generally often performed manually.

Patent Document 1 discloses a technology for automatically identifying the type of cross-section of a two-dimensional ultrasound image of the heart. As an example, Patent Document 1 describes an example in which the cross-section type of an apical approach image is identified from among left ventricular long axis cross-section, four-chamber cross-section, five-chamber cross-section, and two-chamber cross-section. In this technology, images that are representative of each cross-section type are stored as templates, and the cross-section type is identified by calculating which template the input image is closest to.

International Publication No. 2007/058195

The method of Patent Document 1 uses only a single case (cross-sectional image) as a template, and is therefore insufficient to accommodate the diverse variations in cross-sectional images. For example, because the shapes of organs, etc. vary depending on the subject (patient), the actual cross-sectional images differ depending on the subject, even for images of the same cross-sectional type. In addition, the model and performance of the modality and the imaging parameters (imaging conditions) also affect the cross-sectional images obtained. Therefore, in order to achieve highly accurate identification of cross-sectional types, it is important to adequately accommodate such diverse variations (variations) in cross-sectional images.

The inventors are therefore considering an approach that uses a large number of cross-sectional images as learning data and performs statistical analysis of those cross-sectional images to generate a statistical model that corresponds to the diverse variations in cross-sectional images.

The present invention aims to provide a technology for reducing the data size as much as possible without reducing the discriminatory power of a statistical model generated by statistically analyzing multiple medical images.

In addition to the above-mentioned objective, the achievement of effects that are derived from the configurations shown in the detailed description of the invention described below and that cannot be obtained by conventional techniques can also be considered as another objective of the disclosure of this specification.

A first aspect of the present invention provides an image processing apparatus comprising : a model acquisition unit that acquires a statistical model generated by statistically analyzing learning data including a plurality of subject images in cross sections including a subject region corresponding to a scan region obtained by a modality and other regions using information on a pixel group of the same target range in each of the plurality of subject images; an image acquisition unit that acquires an input image to be processed which is a cross-sectional image; and an estimation unit that performs an estimation process to estimate identification information to be used for diagnosis using the input image using the statistical model, wherein the model acquisition unit sets the target range to be a partial range of the subject image and to include at least a portion of the subject region in each of the plurality of subject images, and the estimation unit provides an image processing apparatus characterized in that, in the estimation process, the estimation unit refers to a pixel group within the target range in the input image in a limited manner by using a correspondence table that associates serial numbers and coordinate values of each pixel constituting a pixel group within the target range . Moreover, a first aspect of the present invention provides an image processing device comprising: a model acquisition unit that acquires a statistical model generated by statistically analyzing learning data including a plurality of object images in cross sections obtained by a modality and including an object region corresponding to a scan region and other regions, using information on a pixel group of the same target range in each of the plurality of object images; an image acquisition unit that acquires an input image to be processed, which is a cross-sectional image; and an estimation unit that performs an estimation process to estimate identification information used in diagnosis using the input image using the statistical model, by projecting features of a pixel group within the object range of the input image onto a subspace and then back-projecting them into an original image space to generate a reconstructed image, wherein the model acquisition unit sets the object range to be a portion of the object images and to include at least a portion of the object region in each of the plurality of object images. Moreover, a first aspect of the present invention provides an image processing apparatus including: a model acquisition unit that acquires a statistical model generated by statistically analyzing learning data including a plurality of subject images, the learning data being two-dimensional ultrasound images of the heart in a cross section including a subject region corresponding to a scan region obtained by a modality and other regions, using information on a pixel group of the same target range in each of the plurality of subject images ; an image acquisition unit that acquires an input image to be processed, the input image being a cross-sectional image; and an estimation unit that performs an estimation process using the statistical model to estimate at least two or more cross-section types from among an apical two-chamber view, an apical three-chamber view, an apical four-chamber view, a short axis view, a parasternal long axis view, and an air -left image acquired with a probe not in contact with the subject, wherein the model acquisition unit sets the target range to be a portion of the subject image and to include at least a portion of the subject region in each of the plurality of subject images, and the estimation unit provides an image processing apparatus characterized in that, in the estimation process, the estimation unit refers to a pixel group within the target range in the input image in a limited manner by using a correspondence table that associates serial numbers and coordinate values of each pixel constituting the pixel group within the target range . Moreover, a first aspect of the present invention provides an image processing apparatus comprising : a model acquisition unit that acquires a statistical model generated by statistically analyzing learning data including a plurality of subject images, the learning data being two-dimensional ultrasound images of the heart in a cross section including a subject region corresponding to a scan region obtained by a modality and other regions, using information on a pixel group of the same target range in each of the plurality of subject images ; an image acquisition unit that acquires an input image to be processed, the input image being a cross-sectional image; and an estimation unit that performs an estimation process to estimate contour information of a predetermined region including at least one of the left ventricle, the left atrium, the right ventricle, and the right atrium in the input image using the statistical model, wherein the model acquisition unit sets the target range to be a portion of the subject image and to include at least a portion of the subject region in each of the plurality of subject images, and the estimation unit provides an image processing apparatus characterized in that, in the estimation process, the estimation unit refers to a pixel group within the target range in the input image in a limited manner by using a correspondence table that associates serial numbers and coordinate values of each pixel constituting the pixel group within the target range .

A second aspect of the present invention provides a medical image diagnostic device comprising: an image acquisition unit that acquires as an input image a medical image which is a cross-sectional image to be processed; and an estimation unit that performs an estimation process to estimate at least two or more cross-sectional types from an apical two-chamber view, an apical three-chamber view, an apical four-chamber view, a short axis view, a parasternal long axis view, and an air-left image acquired with the probe not in contact with the subject, using a statistical model generated by statistically analyzing pixel information of corresponding image regions among a plurality of medical images which are cross-sectional images including a subject region corresponding to a scan region by a modality and other regions, the medical images being two-dimensional ultrasound images of the heart . In addition, a second aspect of the present invention provides a medical image diagnostic apparatus comprising: an image acquisition unit that acquires, as an input image, a medical image which is a cross-sectional image and is to be processed; and an estimation unit that performs an estimation process to estimate contour information of a predetermined region in the input image, the predetermined region including at least one of the left ventricle, the left atrium, the right ventricle, and the right atrium, using a statistical model generated by statistically analyzing pixel information of corresponding image regions among a plurality of medical images which are cross-sectional images including a subject region corresponding to a scan region by a modality and other regions, the medical images being two-dimensional ultrasound images of the heart.

A third aspect of the present invention provides an image processing method including the steps of: acquiring a statistical model generated by statistically analyzing learning data including a plurality of subject images in cross sections obtained by a modality and including a subject region corresponding to a scan region and other regions, using information on a pixel group of the same target range in each of the plurality of subject images; acquiring an input image to be processed, which is a cross-sectional image ; and performing an estimation process to estimate identification information used in diagnosis using the input image, using the statistical model; wherein the step of acquiring the statistical model sets the target range to be a partial range of the subject image and to include at least a portion of the subject region in each of the plurality of subject images ; and the step of performing the estimation process provides an image processing method characterized in that the estimation process refers to a pixel group within the target range in the input image in a limited manner by using a correspondence table that associates serial numbers and coordinate values of each pixel constituting a pixel group within the target range . Further, a third aspect of the present invention provides an image processing method including the steps of: acquiring a statistical model generated by statistically analyzing learning data including a plurality of subject images in cross sections obtained by a modality and including a subject region corresponding to a scan region and other regions, using information on a pixel group of the same target range in each of the plurality of subject images; acquiring an input image to be processed, which is a cross-sectional image; and performing an estimation process of estimating identification information used in diagnosis using the input image using the statistical model, by projecting features of a pixel group within the target range of the input image onto a subspace and then back-projecting it into an original image space to generate a reconstructed image, wherein the step of acquiring the statistical model includes setting the target range to be a partial range of the subject images and to include at least a portion of the subject region in each of the plurality of subject images.

A fourth aspect of the present invention provides a program for causing a computer to execute each step of the image processing method described above.

A fifth aspect of the present invention provides a learning device that uses medical images obtained by a modality and including a subject region corresponding to a scan region by the modality and other regions for learning, the learning device comprising: a learning data preparation unit that prepares learning data including a plurality of subject images; a target range acquisition unit that acquires a target range that is uniformly applied to the plurality of subject images; and a model generation unit that generates a statistical model by statistically analyzing information on a group of pixels within the target range in each of the plurality of subject images, wherein the target range is set to be a portion of the subject image and to include at least a portion of the subject region in each of the plurality of subject images .

The present invention makes it possible to reduce the data size as much as possible without reducing the discriminatory power of a statistical model generated by statistically analyzing multiple medical images.

FIG. 1 is a diagram showing an example of the arrangement of an image processing system according to an embodiment. FIG. 1 shows examples of an apical two-chamber view, an apical three-chamber view, an apical four-chamber view, and an air-standing view. 10 is a flowchart showing an example of a processing procedure of the image processing device. 1 is a flowchart showing an example of a learning process for generating a statistical model from learning data. 13 is a graph showing an example of the relationship between the number of eigenvectors and the cumulative contribution rate. 11 is a flowchart showing an example of a cross-section type identification process.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. However, the components described in this embodiment are merely examples, and the technical scope of the present invention is determined by the claims, and is not limited to the individual embodiments described below.

The image processing device according to the embodiment of the present invention provides a function of performing estimation processing on a medical image using a statistical model generated by statistically analyzing pixel information of corresponding image regions among a plurality of medical images including a subject region and other regions. The estimation processing is, for example, a process of estimating a cross-section type of a medical image, a process of estimating contour information of a predetermined region in a medical image, etc., but is not limited to these and may include other estimation processing. The cross-section type is a classification from the viewpoint of which cross-section of the subject is captured in the image. Here, an image processing device is used as an example for explanation, but the present invention may also be applied to a medical image diagnostic device.

A medical image is an image that shows the internal structure of a subject (such as the human body) and is acquired by an imaging device called a modality. It is also called a cross-sectional image, tomographic image, or reconstructed image. Typical examples of medical images include ultrasound images obtained by ultrasound diagnostic devices, X-ray CT images obtained by X-ray CT devices, and MRI images obtained by MRI devices. Such medical images are used, for example, in the medical field for diagnosis, examination, research, and so on.

A medical image includes a subject region and other regions. The subject region refers to a region that contains information necessary for the estimation process, and is typically a region of a medical image that corresponds to the inside of a subject (a region in which the internal structure and characteristics of the subject are imaged). The region that corresponds to the inside of a subject may also be referred to as a region that corresponds to a scan region by a modality. However, for example, if only information on a specific object (organ, tissue, structure, lesion, etc.) present in the subject is used in the estimation process, the subject region may be the region that corresponds to the specific object, rather than the entire region that corresponds to the inside of the subject.

The multiple medical images prepared as training data may include medical images with different shapes of the subject region. Therefore, during training, a procedure is adopted in which corresponding image regions (also called calculation ranges) are set between all medical images, and a statistical model is generated by statistically analyzing information on pixel groups within the image regions in each of the multiple medical images. Here, the image region is set to be a part of the range of the medical image and to include at least a part of the subject region in each of the multiple medical images. With this method, a pixel group that always includes the image features of the subject region is extracted from each of all medical images provided as training data, regardless of whether the shape of the subject region is the same. Therefore, a statistical model with high expressiveness and discriminative power can be generated. Furthermore, since pixel groups of the same size (same number of pixels) are extracted from each of all medical images, regardless of whether the shape of the subject region is the same, it is easy to apply to statistical analysis. In addition, the data size of the statistical model can be reduced compared to using the entire medical image. Therefore, it is possible to reduce the data size as much as possible without reducing the expressiveness and discriminative power of the statistical model.

The shape of the subject region may vary depending on the scan region of the modality, the imaging parameters (imaging conditions) of the modality, the type of probe, etc. In other words, if the scan region, imaging parameters (imaging conditions), type of probe, etc. are known, the shape of the subject region can be determined to some extent. Therefore, at least one of the scan region, imaging parameters (imaging conditions), and type of probe may be used as learning data.

One of the features of the image processing device according to the embodiment of the present invention is that it estimates or identifies the cross-section type of a medical image to be processed (a cross-section image of unknown cross-section type) based on the results of statistical analysis of a large number of cross-section images (learning data) of known cross-section types. Specifically, the image processing device uses multiple statistical models corresponding to multiple cross-section types, respectively, and identifies the cross-section type of the input image by evaluating which of the multiple statistical models best fits the input image. For example, the input image is reconstructed using each statistical model, and the cross-section type associated with the statistical model that has the smallest difference between the reconstructed image and the input image, in other words, that best reproduces the input image, is determined as the identification result.

In this case, the statistical model is a model generated by statistically analyzing multiple cross-sectional images of the same cross-sectional type, and represents the distribution (statistical tendency) of the characteristics of the images of that cross-sectional type. The statistical model may be generated (learned) by, for example, performing principal component analysis on a group of images of the same cross-sectional type using a learning device, and obtaining a subspace that represents the characteristics of the group of images of that cross-sectional type. In this case, the information of the subspace corresponds to the statistical model corresponding to the cross-sectional type. The statistical model is also called a learned model.

Conventional technology used templates generated from only a single case (image), which resulted in a problem of reduced recognition accuracy due to differences in the features between the input image and the template caused by variations in cross-sectional images due to the subject, imaging device, imaging conditions, etc. In contrast, an image processing device according to an embodiment of the present invention uses a statistical model generated from multiple cross-sectional images, making it possible to recognize cases based on the statistical trends of multiple cases, thereby improving recognition accuracy compared to conventional technology.

<Embodiment>
The image processing device according to the embodiment is a device for identifying the cross-section type of an input image. In this embodiment, a two-dimensional ultrasound image of the heart is used as the input image and learning data, and the input image is identified as one of four types, namely, an apical two-chamber image, an apical three-chamber image, an apical four-chamber image, and an air-left image. The apical two-chamber image is an image showing two chambers, the left atrium and the left ventricle, the apical three-chamber image is an image showing three chambers, the left atrium, the left ventricle, and the right ventricle, and the apical four-chamber image is an image showing a total of four chambers, the left and right atria and the left and right ventricles. The air-left image is an image in which the probe is not in contact with the subject and none of the cavities are shown. Note that these are examples of cross-section types, and other types of images can also be identified by similar processing.

The configuration and processing of the image processing device of this embodiment will be described below with reference to FIG. 1. FIG. 1 is a block diagram showing an example of the configuration of an image processing system (also called a medical image processing system) including the image processing device of this embodiment. The image processing system includes an image processing device 10 and a database 22. The image processing device 10 is connected to the database 22 in a communicable state via a network 21. The network 21 includes, for example, a LAN (Local Area Network) or a WAN (Wide Area Network).

The database 22 holds and manages a large number of images and information associated with each image. The images held in the database 22 include images with unknown cross-section types and images with known cross-section types. The former images are used for cross-section type identification processing by the image processing device 10. The latter images are associated with at least cross-section type information (correct answer data) and are used as learning data for generating a statistical model. Each image may be associated with information such as imaging parameters (imaging conditions) when the ultrasound diagnostic device captured the image. The image processing device 10 is able to acquire data held in the database 22 via the network 21.

The image processing device 10 includes a communication IF (Interface) 31 (communication unit), a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 33, a storage unit 34, an operation unit 35, a display unit 36, and a control unit 37.

The communication IF 31 (communication unit) is composed of a LAN card or the like, and realizes communication between an external device (such as the database 22) and the image processing device 10. The ROM 32 is composed of non-volatile memory or the like, and stores various programs and various data. The RAM 33 is composed of volatile memory or the like, and temporarily stores programs being executed and data being worked on. The storage unit 34 is composed of a HDD (Hard Disk Drive) or the like, and stores programs and data. The operation unit 35 is composed of a keyboard, mouse, touch panel, etc., and inputs instructions from a user (such as a doctor or technician) to various devices.

The display unit 36 is configured with a display and the like, and presents various information to the user. The control unit 37 is configured with a CPU (Central Processing Unit) and the like, and controls the processing in the image processing device 10. The control unit 37 has, as its functional configuration, an image acquisition unit 51, a statistical model acquisition unit 52, an estimation unit 53, a display processing unit 54, and a calculation target range acquisition unit 55. The control unit 37 may also have a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field-Programmable Gate Array), etc.

The image acquisition unit 51 acquires an input image to be processed from the database 22. The input image is a cross-sectional image of a subject captured by a modality. The image acquisition unit 51 may acquire the input image directly from the modality. In this case, the image processing device 10 may be a device connected to the modality, or may be a function implemented within the modality (e.g., a console).

The image acquisition unit 51 also acquires the learning data from the database 22. The learning data is an image group consisting of multiple cross-sectional images (medical images) different from the input image, and includes multiple cross-sectional images belonging to each of the cross-sectional types to be identified. Each image in the learning data holds information on the cross-sectional type (an identifier representing one of the apical two-chamber image, apical three-chamber image, apical four-chamber image, and air-left image) as a known value (ancillary information). In order to increase the robustness of the statistical model, it is desirable that the multiple cross-sectional images used as the learning data are images of different subjects (patients). However, images of the same subject (patient) taken under different conditions (time, period, modality model, imaging parameters, etc.) may be included in the learning data. In addition, each frame of a two-dimensional moving image taken over one cardiac cycle may be used as the learning data. The number of cross-sectional images used as the learning data should be at least tens to hundreds. Note that the method of storing the cross-section type information in the learning data may be a method other than storing the cross-section type information as supplementary information for each cross-section image. For example, any method may be used, such as using a list in which the identifiers (image ID, file name, etc.) of the cross-section image are associated with the cross-section type, storing cross-section images in separate folders for each cross-section type, or embedding the cross-section type information in the file name or properties of the cross-section image.

In this embodiment, an example will be described in which the input image and learning data are a single two-dimensional ultrasound image of the cardiac region, but other types of images (other organs, other cross-section types) may also be used. The present invention is also applicable to multiple two-dimensional images (e.g., images of multiple time phases) and two-dimensional video images. It is also applicable regardless of the type of modality.

The statistical model acquisition unit 52 acquires pixel value information and cross-section type information for each of the cross-sectional images included in the learning data acquired by the image acquisition unit 51, and performs statistical analysis for each group of images with the same cross-section type (i.e., for each cross-section type). That is, in this embodiment, the statistical model acquisition unit 52 performs statistical analysis for each of the image groups of the apical two-chamber image, the image group of the apical three-chamber image, the image group of the apical four-chamber image, and the image group of the air-left image included in the learning data. Then, from the result of the statistical analysis, the statistical model acquisition unit 52 acquires subspace information (information on the basis constituting the subspace) for each cross-section type as a statistical model for each cross-section type. Note that, if a statistical model generated in advance is stored in the database 22 or the storage unit 34, the statistical model acquisition unit 52 may acquire the statistical model from the database 22 or the storage unit 34. That is, the statistical model acquisition unit 52 may function as a learning device that generates a statistical model from learning data, may have a function of reading a statistical model generated in advance and stored in a storage device, or may have both functions.

The estimation unit 53 performs estimation processing on the input image using a statistical model. In this embodiment, the estimation unit 53 identifies the cross-section type of the input image based on the input image acquired by the image acquisition unit 51 and the subspace information, which is the statistical model acquired by the statistical model acquisition unit 52. The display processing unit 54 displays the input image and the identification result of the cross-section type of the input image in the image display area of the display unit 36. In addition to the above-mentioned display contents, the display unit 36 can display various information necessary for observing two-dimensional ultrasound images, such as scale information of the displayed input image and waveform information of the electrocardiogram.

The calculation target range acquisition unit 55 acquires a calculation target range that is uniformly applied to all cross-sectional images (medical images) included in the learning data acquired from the database 22. The calculation target range is information that defines the pixel group used in the statistical analysis, i.e., the pixel group used in the subspace calculation. The statistical model acquisition unit 52 and the estimation unit 53 perform the calculation of the statistical model and the estimation process using only the information of the pixel group within the calculation target range acquired by the calculation target range acquisition unit 55.

Each component of the image processing device 10 functions according to a computer program. For example, the control unit 37 (CPU) reads and executes a computer program stored in the ROM 32 or the memory unit 34, etc., using the RAM 33 as a working area, thereby realizing the function of each component. Note that some or all of the functions of the components of the image processing device 10 may be realized by using a dedicated circuit. Also, some of the functions of the components of the control unit 37 may be realized by using cloud computing. For example, a calculation device located at a different location from the image processing device 10 may be communicably connected to the image processing device 10 via the network 21, and the image processing device 10 and the calculation device may send and receive data to each other, thereby realizing the functions of the components of the image processing device 10 or the control unit 37.

Figures 2A to 2D show schematic cross-sectional images of four types of cross-sections exemplified as objects to be identified in this embodiment. Figures 2A to 2D are two-dimensional ultrasound images captured with a sector probe, with Figure 2A showing an apical two-chamber image, Figure 2B showing an apical three-chamber image, Figure 2C showing an apical four-chamber image, and Figure 2D showing an airborne image. The apical two-chamber image shows two regions, the left ventricle 203 and the left atrium 204, the apical three-chamber image shows three regions, the left ventricle 205, the left atrium 206, and the right ventricle 207, and the apical four-chamber image shows four regions, the left ventricle 210, the left atrium 211, the right ventricle 208, and the right atrium 209. The airborne image (Figure 2D) shows none of the regions because it is an image in which the probe is not in contact with the subject.

Next, an example of processing of the image processing device 10 in FIG. 1 will be described using the flowchart in FIG. 3.

(Step S100: Acquire calculation range)
In the calculation of the statistical model (step S101) and the estimation process (step S103) in the subsequent stages, pixel value information of the cross-sectional image and the input image is used for the calculation. However, as shown in Figures 2A to 2D, medical images captured by a modality contain many pixels other than the subject region, and using the entire image for calculation not only generates unnecessary calculation load, but also may lead to an increase in the data size of the statistical model and a decrease in discrimination power. Therefore, in this embodiment, only the necessary range of the medical image (for example, in the case of an ultrasound image obtained by a sector probe, a sector-shaped scan area, etc.) is used for the calculation of the statistical model and the estimation process.

First, in step S100, the calculation range acquisition unit 55 acquires pixel value information (image data) of each image included in the learning data from the database 22. Then, using the pixel value information of multiple images in the learning data, it determines the spatial range (calculation range) to be calculated in the subsequent steps.

A method for determining the calculation target range will be specifically described. In a two-dimensional ultrasound image, as shown in Figures 2A to 2D, among the image regions expressed as rectangles, only the sector-shaped scan region (hereinafter referred to as the sector region or subject region) contains a signal. Therefore, when performing the following steps S101 and S103, it is clear that only the sector region needs to be the calculation target. However, it is not possible to simply set the sector region in each image as the calculation target range. This is because, in the following steps S101 and S103, the dimension of the vector (the number of elements of the column vector) when the image is converted into a column vector needs to be the same between all the learning data and the input image, but the shape and size of the sector region of each image are not necessarily the same. If the case is different, or if the model and performance of the modality or the imaging parameters (imaging conditions) are different, the size of the central angle of the sector region and the effective image display size (depth of field of view) may be different. Therefore, it is necessary to define a calculation target range that can be applied uniformly to all the learning data and input images in some way.

In this embodiment, the calculation target range acquisition unit 55 calculates a region that covers all of the subject regions (after spatial normalization) of all learning data, and acquires this as the calculation target range. When the subject region is sector-shaped as in Figures 2A to 2D, the calculation target range will also be a sector of approximately similar shape. Note that in the case of a convex probe, the subject region will be shaped like a Baumkuchen, so the shape of the calculation target range should also be shaped like a Baumkuchen. The Baumkuchen shape is an annular sector with the tip of the sector missing.

In addition, in the case of a linear probe, the subject region is a parallelogram, so the shape of the calculation range can also be a parallelogram. In the case of CT or MRI axial cross-sectional images, the subject region is approximately circular, so the shape of the calculation range can also be approximately circular. By setting the calculation range to match the shape of the subject region in this way, pixels outside the subject region can be excluded from the calculation as much as possible. In this case, the calculation range will be non-rectangular, and will have an area smaller than the circumscribing rectangle of the subject region.

Here, it is not essential to cover all the subject regions of all images, and a region that can cover a predetermined percentage (e.g., 98%) of the subject regions of the images may be set as the calculation target range. In other words, it is sufficient that the calculation target range is set so that at least a part of the sector region (subject region) is included in each image. Note that the method of calculating the calculation target range is not limited to this, and for example, the ventricles and atria may be extracted from each of the learning data by image processing, and a sector region that covers the entire region may be obtained as the calculation target range. In this case, the ventricle and atrium regions may be information provided manually rather than information calculated by image processing. The calculation target range acquisition unit 55 stores the determined calculation target range in the RAM 33 and the storage unit 34 for use in subsequent steps.

The calculation target range may be a range manually defined by the operator. For example, the operator may visually set a region that includes the atrial and ventricular regions of all learning data, and store this in advance in the database 22 or the storage unit 34 as the calculation target range. In this case, the calculation target range acquisition unit 55 acquires the calculation target range by reading out the calculation target range stored in the database 22 or the storage unit 34.

The calculation range does not have to include all of the areas where signals originating from the subject are present. For example, the calculation range may be set to include areas where pixel values are equal to or greater than a predetermined threshold, thereby excluding areas where pixel values are low (dark) on the periphery of the subject area. The calculation range is not limited to a sector shape, and any shape, including a circle or triangle, may be used.

ここで、例えばｍ［１］は計算対象範囲における１番目のピクセルを表す記号で、ｘ１・ｙ１はそれぞれ該ピクセルのｘ座標値・ｙ座標値である。そして、Ｋは計算対象範囲を構成するピクセルの数を表す。画像のｘ方向とｙ方向のピクセル数をそれぞれＮｘ、Ｎｙとしたときに、Ｋ＜Ｎｘ×Ｎｙである。 Specifically, the defined calculation range is expressed as a correspondence table (lookup table) that associates the serial numbers (pixel numbers) of each pixel that makes up the pixel group within the calculation range with their coordinate values, as shown in the formula below.

Here, for example, m[1] is a symbol representing the first pixel in the calculation range, and x1 and y1 are the x-coordinate and y-coordinate values of the pixel, respectively. K represents the number of pixels that make up the calculation range. When the number of pixels in the x- and y-directions of the image are Nx and Ny, respectively, K<Nx×Ny.

The method of storing the definition of the calculation range is not limited to the above example. For example, a mask image of Nx x Ny pixels in which the pixels in the calculation range are set to 1 and the other pixels are set to 0 may be used. However, the method of storing the calculation range as a correspondence table between serial numbers (1 to K) and coordinate values as in this embodiment has the following advantages over the general method of using a mask image described above. In the method using a mask image, when performing some processing limited to the calculation range, it is necessary to check whether each of the Nx x Ny pixels is included in the calculation range. In other words, it is necessary to scan Nx x Ny pixels. On the other hand, the storage method of this embodiment allows direct and limited reference to a group of K pixels (fewer than Nx x Ny) within the calculation range of the image. This makes it possible to reduce processing time.

(Step S101: Obtaining a statistical model)
In step S101, the statistical model acquisition unit 52 acquires a plurality of statistical models respectively corresponding to a plurality of cross-section types to be identified.

Figure 4 shows an example of a process (learning process) executed by the statistical model acquisition unit 52 to generate a statistical model from training data.

In step S401, the statistical model acquisition unit 52 acquires learning data from the database 22 via the image acquisition unit 51. The statistical model acquisition unit 52 then groups the multiple cross-sectional images acquired as learning data by cross-sectional type. In the case of this embodiment, four image groups are generated: an apical two-chamber image, an apical three-chamber image, an apical four-chamber image, and an air-standing image. The subsequent processing of steps S402 to S406 is performed for each grouped image group.

In step S402, the statistical model acquisition unit 52 performs spatial normalization on each image included in the image group. Spatial normalization is a process for aligning the scales of images. The learning data includes a mixture of images captured with different models and images captured with different imaging parameters (imaging conditions). Therefore, the position and size of the subject region (the region in the image where a signal exists (the sector-shaped region in Figures 2A to 2D)) in the image may differ depending on the image. Therefore, if the image group collected as learning data is subjected to statistical analysis as it is, the variation in the position and size of the subject region may lead to a decrease in the expressive power of the statistical model. Therefore, in this embodiment, spatial normalization is performed prior to statistical analysis to align the position and size of the subject region in each image.

Specifically, the statistical model acquisition unit 52 first acquires the probe position coordinates and the effective image display size for each image of the learning data as information to be used for spatial normalization. The probe position coordinates are the image coordinates of the vertices of the sector-shaped subject region (reference numeral 201 in FIG. 2A), and the effective image display size is the number of pixels of the radius of the sector-shaped subject region (reference numeral 202 in FIG. 2A). The effective image display size corresponds to the field of view depth of the ultrasound image. The information of the probe position coordinates and the effective image display size can be acquired, for example, by reading known information held in the learning data. This known information may be information created by a person who prepares the learning data, or may be information obtained from a modality (ultrasound diagnostic device) that captures the image. Alternatively, the statistical model acquisition unit 52 may acquire the probe position coordinates and the effective image display size by extracting the subject region (contour of the sector) from each image by image processing. Next, the statistical model acquisition unit 52 enlarges or reduces the entire image so that the effective image display size of each image becomes a predetermined number of pixels. The predetermined number of pixels, i.e., the effective image display size after normalization processing, is predefined (for example, 512 pixels). After that, the statistical model acquisition unit 52 translates the images so that the probe position coordinates of all the images are the same.

Note that spatial normalization is not essential. For example, if it is known in advance that the variation in the position and size of the subject region is sufficiently small, spatial normalization may be omitted. In addition, although the present embodiment has been described with reference to an example in which the effective image display size is used to scale the image, other information may be used. For example, when the image size (the number of pixels in the x and y directions of the entire image) differs for each image, normalization may be performed to make the image sizes consistent. Furthermore, normalization may be performed by considering the physical size per pixel, rather than the number of pixels, so that the physical size per pixel is consistent. This allows normalization to be performed even when information on the effective image display size cannot be obtained.

In step S403, the statistical model acquisition unit 52 performs a cropping process to make the number of pixels in each spatially normalized image consistent. Specifically, the statistical model acquisition unit 52 defines a rectangular area of a predetermined size that encompasses the sector-shaped subject area, and cuts out the rectangular area from each image. At this time, the number of pixels in the x and y directions of the rectangular area are set to Nx and Ny, respectively. Note that, although the cropping process is performed after the spatial normalization process in this embodiment, the same results can be obtained by first cutting out the subject area from the image, and then enlarging or reducing the size of the cut-out image to Nx x Ny.

なお、処理対象の画像が１枚の２次元画像ではなく複数枚の２次元画像の場合には、それらの画素値情報を直列に並べたものを列ベクトルａとすればよい。 In step S404, the statistical model acquisition unit 52 converts pixel value information of the pixel group within the calculation range of each image into a column vector. The cropped image is composed of Nx×Ny pixels, but the number of pixels used in the column vector is K. The column vector a of pixel value information of a certain image I is defined as follows, with the upper left corner of the image being the origin (0,0) and the pixel value at pixel position (x, y) being represented by I(x, y).

When the image to be processed is not one two-dimensional image but a plurality of two-dimensional images, the pixel value information of these images may be arranged in series to form column vector a.

In step S405, the statistical model acquisition unit 52 performs statistical analysis on the group of column vectors corresponding to all images included in the image group. For the statistical analysis, a known method such as Principal Component Analysis (PCA) may be used, and methods such as Kernel PCA and Weighted PCA may also be used. In addition, a statistical analysis method other than principal component analysis, such as Independent Component Analysis (ICA), may also be used. Below, an example will be described in which PCA is used.

ここで、ａバーが平均ベクトル、ｅ_ｉがｉ番目の基底における固有ベクトル、ｇ_ｉが固有ベクトルｅ_ｉに対応する係数である。また、Ｌは計算に用いる固有ベクトルの数を表している。 By using PCA, the average vector and eigenvectors of the column vector group, and the eigenvalues corresponding to each eigenvector are calculated. By using these, a subspace that represents the characteristics of a group of images of the same cross-section type can be expressed as follows, where d is a point that exists in the subspace.

Here, a bar is the mean vector, e _i is an eigenvector in the i-th basis, g _i is a coefficient corresponding to the eigenvector e _i , and L represents the number of eigenvectors used in the calculation.

The value of L will now be described. It is desirable that the value of L is significantly smaller than the number that can adequately express the learning data (for example, the number at which the cumulative contribution rate exceeds 95%). Specifically, it is advisable to set the value of L so that the cumulative contribution rate of the learning data is 50% or more and 70% or less. In the inventors' experiments, a decrease in the identification rate of the cross-section type was observed when the cumulative contribution rate was smaller than 50%. This is thought to be because when the cumulative contribution rate is too small, the expressive power of the statistical model is too low and it is not possible to adequately handle the diversity of images within the same cross-section type. In addition, a decrease in the identification rate of the cross-section type was observed when the cumulative contribution rate was larger than 70%. This is thought to be because when the cumulative contribution rate is too large, the expressive power of the statistical model is too high, and the statistical model fits well to all cross-section types, resulting in a decrease in identification power.

Even if the range of L values in which the cumulative contribution rate falls within the range of 50% to 70% differs for each cross-section type, it is desirable to make the L value uniform rather than changing it for each cross-section type. An example will be described with reference to FIG. 5. FIG. 5 is a graph showing the relationship between the number of eigenvectors (L value) and the cumulative contribution rate. According to FIG. 5, the image left in the air shows a higher cumulative contribution rate with a smaller basis number than the other types (two-chamber image, three-chamber image, and four-chamber image). In this example, it is desirable to set L = 7 to 21 for all cross-section types, rather than setting the L value for the image left in the air to be smaller than the other three types. This is because, according to experiments by the inventors, when calculating the reconstruction error in the subsequent step S604, a type with a smaller L value than the other types may be calculated to have a larger reconstruction error than the difference observed by the eye, and the identification performance may be reduced. For this reason, in this embodiment, a single value of L = 10 to 20 is adopted based on the cumulative contribution rate for the two-chamber image, three-chamber image, and four-chamber image.

In step S406, the statistical model acquisition unit 52 generates subspace information (the mean vector (a bar) and L eigenvectors (e ₁ to e _L ) constituting the subspace) which is a statistical model for each cross-section type from the results of the statistical analysis, and stores the information in the RAM 33. This completes the learning process of the statistical model.

The functions of steps S401 to S406 performed by the statistical model acquisition unit 52 are an example of the "learning device" of the present invention. The function of step S401 and the functions of steps S404 to S406 are examples of the "learning data preparation unit" and "model generation unit" of the present invention, respectively.

The statistical model generation process (learning process) in steps S401 to S406 is data processing independent of the input image cross-section type identification process described below. Therefore, the statistical model generation process may be performed in advance, and the generated statistical model may be stored in the database 22 or the storage unit 34 in advance. In this case, in step S101 in FIG. 3, the statistical model acquisition unit 52 reads the statistical model from the database 22 or the storage unit 34 and stores it in the RAM.
33. By calculating the statistical model in advance, it is possible to reduce the processing time required for identifying the cross-section type in the input image. The statistical model generation process may be performed by the statistical model acquisition unit 52 of the image processing device 10, or may be performed by another learning device different from the image processing device 10. The process of generating a statistical model by another learning device is also similar to that shown in FIG.

Note that between the spatial normalization process (step S402) and the crop process (step S403), the images in the learning data may be amplified. That is, each image in the learning data may be translated by ±10 pixels in each axis direction, and the displaced image may also be used as learning data. The augmentation may be performed by rotation as well as translation. In addition, both may be combined. In addition, when performing statistical analysis in step S405, the original image and the augmented image may be statistically analyzed together, or images with the same amount of displacement (translation and rotation) may be grouped and statistically analyzed separately. When performing statistical analysis by grouping images with the same amount of displacement, a statistical model of the number of cross-section types x the number of combinations of the amount of displacement is generated. In this way, the robustness of the statistical model against variations in the position and angle of the input image can be improved. In addition, in this embodiment, the pixel value itself is used as pixel value information, but other image features (for example, features related to the texture of the image) may be used as pixel value information.

(Step S102: Acquiring input image)
In step S102, when the user issues an instruction to acquire an image via the operation unit 35, the image acquisition unit 51 acquires the input image designated by the user from the database 22 and stores it in the RAM 33. At this time, the display processing unit 54 may display the input image in the image display area of the display unit 36.

(Step S103: Identification of cross section type)
In step S103, the estimation unit 53 identifies the cross-section type of the input image by evaluating the fit of each of the multiple statistical models acquired in step S101 to the input image. In this embodiment, partial space information for each cross-section type is used as the statistical model. That is, when the number of cross-section types to be identified is four, four pieces of partial space information are acquired. In this case, the estimation unit 53 may calculate a score representing the possibility that the features of the input image belong to the partial space for each cross-section type, and identify the cross-section type of the input image based on the score for each cross-section type. Although the method of calculating the score does not matter, for example, a reconstructed image obtained by projecting the features of the input image onto the partial space and then back-projecting it into the original image space may be generated for each cross-section type, and a score for each cross-section type may be calculated based on the similarity between each reconstructed image and the input image. The higher the similarity between the reconstructed image and the input image, that is, the smaller the difference between the reconstructed image and the input image, the more likely it is that the input image belongs to that partial space. Therefore, by comparing the scores for each cross-section type, the partial space to which the input image belongs, that is, the cross-section type of the input image, may be estimated. Note that the score may be positively or negatively correlated with the similarity between the reconstructed image and the input image, for example, the difference between the reconstructed image and the input image (called the reconstruction error) is an example of a score that is negatively correlated with the similarity.

Figure 6 shows a specific example of the cross-section type identification process executed by the estimation unit 53.

In step S601, the estimation unit 53 acquires pixel value information of the input image from the image acquisition unit 51. The estimation unit 53 also acquires probe position information and effective image display size information as information used for spatial normalization of the input image. For example, these pieces of information can be acquired by storing imaging parameters used when the ultrasound diagnostic device captures the input image as auxiliary information in the input image and reading them together with the input image as known information. Alternatively, if the input image does not store the information, the information may be acquired by extracting the subject region from the input image by image processing.

In step S602, the estimation unit 53 performs spatial normalization and cropping on the input image similar to steps S402 and S403 to align the number of pixels in the x and y directions of the input image to Nx and Ny, the same as the learning data. Then, the estimation unit 53 converts pixel value information of a pixel group within the calculation target range of the input image into a column vector a _t according to formula (1).

It is preferable that the spatial normalization process for the input image is the same as the spatial normalization process performed for the learning data in step S402. This is because higher classification accuracy can be expected if the scales of the learning data and the input image are the same. However, the spatial normalization process is not essential and may be omitted. Also, normalization based on the image size (the number of pixels in the x and y directions of the entire image) or the physical size per pixel may be performed instead of normalization based on the effective image display size. It is also possible to configure the system so that spatial normalization is performed on the learning data when calculating the partial space information, which is a statistical model, but spatial normalization is not performed on the input image. This method can be used in cases where the effective image display size can be obtained as known information by offline manual work during learning, but the effective image display size is unknown during classification. Alternatively, spatial normalization of the input image may be omitted by performing spatial normalization of the learning data in accordance with the effective image display size of the input image. This method is particularly effective when the effective image display size of the input image is known and fixed.

In step S603, the estimation unit 53 reconstructs the input image in each subspace. Now, for each subspace, a matrix E in which all eigenvectors e ₁ to e _L are arranged is defined by the following equation.

ここでＥ^ＴはＥの転置行列を表す。次に、以下の式を用いて、固有空間への投影点ｐ_ｔを今度は逆投影する。 The estimation unit 53 calculates a projection point p _t of the input image a _t onto the eigenspace using the following equation.

Here, E ^T represents the transpose matrix of E. Next, the projection point p _t onto the eigenspace is back-projected using the following equation.

左辺（ａハット_ｔ）は、行列Ｅで表現される部分空間の表現力の範囲内で入力画像ａ_ｔを表現した、再構築画像である。推定部５３は、この再構築画像を、断面種別ごとのＥを用いてそれぞれ算出する。

The left side (a hat _t ) is a reconstructed image that expresses the input image a _t within the range of the expressive power of the subspace expressed by the matrix E. The estimation unit 53 calculates this reconstructed image using E for each cross-section type.

In step S604, the estimation unit 53 calculates a score representing the difference (i.e., reconstruction error) between the input image and the reconstructed image calculated for each cross-section type, and identifies the cross-section type by comparing the scores for each cross-section type. In this embodiment, a known image similarity evaluation index, SSD (Square Sum Difference), is used as a score representing the difference between a column vector a _t representing the characteristics of the input image and a column vector a _t representing each reconstructed image. Then, the cross-section type associated with the subspace that brings about the reconstructed image with the smallest score is determined as the identification result. Note that, in addition to the above-mentioned SSD, any image similarity evaluation measure may be used to calculate the reconstruction error.

It should be noted that any method other than the above may be used as long as it calculates the score of the input image for each subspace. For example, only the projection process of formula (4) may be performed, and the projection point p t closest to the mean vector, i.e., the projection point p _t with the smallest norm, may be used as the classification result. Alternatively, various known subspace methods such as the kernel nonlinear subspace method and the mutual subspace method may be used.

The estimation unit 53 may determine that "the input image does not correspond to any of the cross-section types" when the scores calculated for each cross-section type satisfy a predetermined condition. The predetermined condition may be, for example, that there is no score smaller than a predetermined threshold (i.e., the reconstruction error is large for all cross-section types), that the difference in scores between cross-section types is smaller than a predetermined value (i.e., it is not possible to clearly identify which cross-section type it is), etc. This reduces the possibility that an incorrect identification result will be displayed and cause confusion to the user when, for example, a two-dimensional ultrasound image of an area other than the cardiac region (or of a cross-section type not to be identified) is input.

In step S605, the estimation unit 53 stores the classification result of step S604 in the RAM 33. The estimation unit 53 may also output the classification result to the database 22 and store it as additional information of the input image held by the database 22. Note that, instead of identifying one cross-section type, the score calculated for each subspace may be stored in the RAM 33 as the classification result. Note that the score may be the reconstruction error value itself, or may be a value normalized to fall within the range of 0.0 to 1.0, for example.

(Step S104: Display of the identification result)
In step S104, the display processing unit 54 displays the input image and its type identification result in the image display area of the display unit 36 in a display form that allows them to be easily viewed. Note that, if the purpose is to record the cross-section type, the display processing of this step is not necessarily required, and the identified cross-section type information may be simply stored in the database 22 or the storage unit 34.

The display processing unit 54 may display not only the cross-section types but also the scores calculated in step S103. In this case, the user can obtain information to determine how likely the identification result is by observing the difference in scores between the cross-section types. Alternatively, the display processing unit 54 may be configured to display only the scores and allow the user to determine the cross-section type.

If the purpose is to analyze or measure the input image according to the type of cross section, after this step, the image processing device 10 executes processing according to the type of cross section. For example, if the type of cross section of the input image is determined to be an apical four-chamber view, it performs branching processing such as extracting the left ventricle, left atrium, right ventricle, and right atrium, and if it is determined to be an apical two-chamber view, it performs branching processing such as extracting the left ventricle and left atrium. In this case, it is not necessary to save or display the type of cross section.

In this embodiment, four types of cross-section types (apical two-chamber view, apical three-chamber view, apical four-chamber view, and air-left view) are identified, but the same processing can be performed even if the number of cross-section types to be identified is different. For example, two types of views, apical four-chamber view and apical two-chamber view, may be identified, or five or more types may be identified by adding short-axis view and parasternal long-axis view. In other words, the cross-section types may include at least two or more of the apical two-chamber view, apical three-chamber view, apical four-chamber view, short-axis view, parasternal long-axis view, and air-left view.

When the input images acquired by the image acquisition unit 51 are frames of a moving image, the processing of steps S102, S103, and S104 may be performed sequentially for each frame of the moving image.

In addition, in this embodiment, the input image and the image used in the statistical analysis are both two-dimensional ultrasound images, i.e., the modalities are the same, but the modalities of the two may be different. For example, the image used in the statistical analysis may be a two-dimensional cross-sectional image cut out from a three-dimensional CT image, and the input image may be a two-dimensional ultrasound image. In this case, the image expressed by the statistical model and the input image may have different pixel value distributions and magnitude relationships. Therefore, in step S103, the normalized correlation coefficient (NCC) and mutual information (MI) are used to calculate the correlation coefficients.
It is desirable to calculate the reconstruction error using a similarity evaluation measure that is unbiased to differences in pixel value distribution, such as the Fourier transform (FFT) or the Fourier transform (FFT).

According to this embodiment, it is possible to more accurately identify the cross-section type of an image based on the statistical trends extracted from the learning data. It is also possible to reduce the data size of the statistical model (subspace information) required for cross-section type identification, thereby shortening the time required for estimation processing.

<Modification>
In the above embodiment, the identification of the cross-section type has been described as an example of the estimation process. However, as long as the partial space information of the image is used, the method of limiting the calculation target range for performing statistical processing can be applied to the identification/estimation process of any additional information of the input image, not limited to the identification of the cross-section type. For example, the method can be applied to a contour extraction process for estimating the contours of each region, such as the left ventricle, left atrium, right ventricle, and right atrium, from a two-dimensional ultrasound image of the cardiac region.

When applying the above-mentioned method for limiting the calculation range to a contour extraction process, a known method such as the back projection for lost pixels (BPLP) method disclosed in Non-Patent Document 1 may be used. The BPLP method is a method for estimating information representing the posture of an object appearing in an image from information on the pixel values of an unknown image, using subspace information related to data that combines information on the pixel values of multiple images and information representing the posture of an object appearing in the images. In other words, the BPLP method is a method for interpolating data in which a certain missing part occurs, based on the results of statistical analysis of training data without the missing part.

[Non-patent Document 1] Toshiyuki Amano, et. al. “An appearance based fast linear pose estimation” MVA 2009 IAPR Conference on Machine Vision Applications. 2009 May 20-22.

In this case, during learning (calculation of subspace information), statistical analysis (principal component analysis) is performed using column vectors that link pixel value information with the auxiliary information to be estimated (contour point cloud coordinates in this case). Then, during estimation, unknown parts (contour point cloud coordinates) are omitted and projection and backprojection are performed using the eigenspace method. In this way, the missing parts are restored by projection and backprojection based on the statistical information contained in the subspace information.

When using the BPLP method, the learning data input to the statistical analysis is a column vector in which pixel value information is arranged in a row, a column vector in which the coordinate values of the contour point group constituting the contour of the target area are arranged in a row, and a column vector in which these two are connected. Of the above, the column vector part in which pixel value information is arranged in a row may be the same as the pixel value information (i.e., column vector a) used to identify the cross-section type in the above embodiment. Therefore, the size reduction by the calculation target range shown in steps S100 and S101 of the above embodiment can be used as it is. Similarly, in the estimation process (projection and backprojection process of the input image), when converting the input image to the column vector a _t , it is possible to reduce the size by using only the pixel value information within the calculation target range.

In this way, the method of limiting the calculation range shown in the above embodiment can be applied not only to identifying cross-section types, but also to any estimation process, such as estimating contour information. In particular, it can be preferably applied to a process of estimating and identifying additional information from an input image using subspace information, which is a statistical model calculated by statistically analyzing learning data consisting of medical images and their additional information.

＜Other＞
Although the embodiments of the present invention have been described above, the present invention is not limited to these, and can be modified and changed within the scope of the claims.

The disclosed technology can be embodied, for example, as a system, device, method, program, or recording medium (storage medium). Specifically, it may be applied to a system consisting of multiple devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or it may be applied to an apparatus consisting of a single device.

Needless to say, the object of the present invention can be achieved by the following: A recording medium (or storage medium) on which is recorded software program code (computer program) that realizes the functions of the above-mentioned embodiments is supplied to a system or device. The storage medium is, of course, a computer-readable storage medium. Then, the computer (or CPU or MPU) of the system or device reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-mentioned embodiments, and the recording medium on which the program code is recorded constitutes the present invention.

10 Image processing device 37 Control unit 51 Image acquisition unit 52 Statistical model acquisition unit 53 Estimation unit

Claims (26)

a model acquisition unit that acquires a statistical model generated by statistically analyzing learning data including a plurality of subject images in cross sections including a subject region corresponding to a scan region and other regions obtained by a modality , using information on a pixel group of the same target range in each of the plurality of subject images ;
an image acquisition unit that acquires an input image to be processed, the input image being a cross-sectional image;
an estimation unit that performs an estimation process to estimate identification information used for diagnosis using the input image by using the statistical model;
An image processing device comprising:
the model acquisition unit sets the target range to be a partial range of the object image and to include at least a part of the object region in each of the plurality of object images ;
In the estimation process, the estimation unit refers to the pixel group within the target range in the input image in a limited manner by using a correspondence table in which serial numbers of each pixel constituting the pixel group within the target range correspond to coordinate values.
13. An image processing device comprising: a model acquisition unit that acquires a statistical model generated by statistically analyzing learning data including a plurality of subject images in cross sections including a subject region corresponding to a scan region and other regions obtained by a modality , using information on a pixel group of the same target range in each of the plurality of subject images ;
an image acquisition unit that acquires an input image to be processed, the input image being a cross-sectional image;
an estimation unit that performs an estimation process to estimate identification information used for diagnosis using the input image by using the statistical model, the process including a process of projecting features of a pixel group within the target range of the input image onto a subspace and then back-projecting the features onto the original image space to generate a reconstructed image;
An image processing device comprising:
The model acquisition unit sets the target range to be a partial range of the object image and to include at least a part of the object region in each of the plurality of object images.
13. An image processing device comprising: a model acquisition unit that acquires a statistical model generated by statistically analyzing learning data including a plurality of subject images, the learning data being two-dimensional ultrasound images of the heart in a cross section including a subject region corresponding to a scan region and other regions obtained by a modality , using information on a pixel group of the same target range in each of the plurality of subject images;
an image acquisition unit that acquires an input image to be processed, the input image being a cross-sectional image;
an estimation unit that performs an estimation process to estimate at least two or more cross-section types from among an apical two-chamber view, an apical three-chamber view, an apical four-chamber view, a short-axis view, a parasternal long-axis view, and an air-left image acquired in a state where the probe is not in contact with the subject, in the input image, using the statistical model;
An image processing device comprising:
the model acquisition unit sets the target range to be a partial range of the object image and to include at least a part of the object region in each of the plurality of object images ;
In the estimation process, the estimation unit refers to the pixel group within the target range in the input image in a limited manner by using a correspondence table in which serial numbers of each pixel constituting the pixel group within the target range correspond to coordinate values.
13. An image processing device comprising: a model acquisition unit that acquires a statistical model generated by statistically analyzing learning data including a plurality of subject images, the learning data being two-dimensional ultrasound images of the heart in a cross section including a subject region corresponding to a scan region and other regions obtained by a modality , using information on a pixel group of the same target range in each of the plurality of subject images;
an image acquisition unit that acquires an input image to be processed, the input image being a cross-sectional image;
an estimation unit that performs an estimation process to estimate contour information of a predetermined region including at least one of a left ventricle, a left atrium, a right ventricle, and a right atrium in the input image by using the statistical model,
the model acquisition unit sets the target range to be a partial range of the object image and to include at least a part of the object region in each of the plurality of object images ;
In the estimation process, the estimation unit refers to the pixel group within the target range in the input image in a limited manner by using a correspondence table in which serial numbers of each pixel constituting the pixel group within the target range correspond to coordinate values.
13. An image processing device comprising: 5. The image processing apparatus according to claim 1, wherein the plurality of subject images include subject images in which the shape of the subject region is different. 5. The image processing device according to claim 1, wherein the target area has a shape that is not rectangular. The image processing apparatus according to claim 6 , wherein the target range has an area smaller than a circumscribing rectangle of the subject region. 8. The image processing apparatus according to claim 6 , wherein the target area has a shape substantially similar to that of the subject region. 9. The image processing apparatus according to claim 1, wherein the object region is a region corresponding to an inside of the object. 10. The image processing apparatus according to claim 1, wherein the subject region is a region corresponding to a predetermined object within the subject. 9. The image processing apparatus according to claim 1, wherein the subject image is an image obtained by a modality, and the subject region is a region corresponding to a scan region by the modality. The image processing device according to claim 11 , wherein the statistical model is generated by using at least one of a scan area of the modality, an imaging parameter of the modality, and a type of a probe of the modality as learning data . The statistical analysis is a principal component analysis;
13. The image processing apparatus according to claim 1, wherein the statistical model is information of a subspace that represents characteristics of a pixel group within the target range of the plurality of object images. 5. The image processing device according to claim 1, wherein the estimation process includes a process of generating a reconstructed image by projecting features of a group of pixels within the target range of the input image onto a subspace and then back-projecting them onto the original image space. The image processing device according to claim 2, characterized in that, in the estimation process, the estimation unit refers to the pixel group within the target range in the input image in a limited manner by using a correspondence table that associates serial numbers of each pixel that constitutes the pixel group within the target range with coordinate values. the subject image and the input image are cross-sectional images,
16. The image processing device according to claim 1, wherein the estimation unit performs an estimation process for estimating a cross-section type of the input image. the subject image and the input image are two-dimensional ultrasound images of a heart,
17. The image processing device according to claim 16, wherein the cross-section types include at least two of an apical two-chamber view, an apical three-chamber view, an apical four-chamber view, a short axis view, a parasternal long axis view, and an air - left image acquired with the probe not in contact with the subject. the subject image and the input image are cross-sectional images,
4. The image processing device according to claim 1 , wherein the estimation unit performs an estimation process for estimating contour information of a predetermined area in the input image. the subject image and the input image are two-dimensional ultrasound images of a heart,
19. The image processing apparatus according to claim 18 , wherein the predetermined region includes at least one of the left ventricle, the left atrium, the right ventricle, and the right atrium. the plurality of object images are images that have been spatially normalized to align the scales;
20. The image processing device according to claim 1, wherein the estimation unit performs the spatial normalization on the input image before the estimation process. the subject image and the input image are two-dimensional ultrasound images;
21. The image processing apparatus according to claim 20 , wherein the spatial normalization includes a process of enlarging or reducing an image so that an effective image display size corresponding to a field of view depth of a two-dimensional ultrasound image has a predetermined number of pixels. 22. The image processing apparatus according to claim 1, wherein the target range is an image area corresponding to the subject area in the subject image. 23. The image processing apparatus according to claim 22 , wherein the number of pixels included in the target range is the same among the plurality of subject images. acquiring a statistical model generated by statistically analyzing learning data including a plurality of subject images of cross sections including a subject region corresponding to a scan region obtained by a modality and other regions, using information on a pixel group of the same target range in each of the plurality of subject images ;
obtaining an input image to be processed, the input image being a cross-sectional image ;
performing an estimation process for estimating discrimination information to be used for diagnosis using the input image by using the statistical model;
An image processing method comprising:
The step of acquiring the statistical model includes setting the target range to be a range of a part of the object image and to include at least a part of the object region in each of the plurality of object images;
The step of performing the estimation process includes, in the estimation process, referring to the pixel group within the target range in the input image in a limited manner by using a correspondence table in which serial numbers of each pixel constituting the pixel group within the target range correspond to coordinate values.
13. An image processing method comprising: acquiring a statistical model generated by statistically analyzing learning data including a plurality of subject images of cross sections including a subject region corresponding to a scan region obtained by a modality and other regions, using information on a pixel group of the same target range in each of the plurality of subject images ;
obtaining an input image to be processed, the input image being a cross-sectional image ;
and performing an estimation process for estimating discrimination information used for diagnosis using the input image by using the statistical model, the process including a process of generating a reconstructed image by projecting features of a pixel group within the target range of the input image onto a subspace and then back projecting the features onto an original image space, the image processing method including:
The step of acquiring the statistical model includes setting the target range to be a range of a part of the object image and to include at least a part of the object region in each of the plurality of object images.
13. An image processing method comprising: 26. A program for causing a computer to execute each step of the image processing method according to claim 24 or 25.

Images