JP7483361B2 - Medical image processing device, medical image diagnostic device, and medical image processing program - Google Patents

Description

The embodiments disclosed in this specification and the drawings relate to a medical image processing device, a medical image diagnostic device, and a medical image processing program.

There is a technology that extracts anatomical landmarks and other features from medical images. The extracted landmarks can be used to automatically set the scan range or adjust scan conditions such as the radiation dose during imaging.
However, since this technique extracts characteristic parts from a photographed image, characteristic parts that are not depicted in the photographed image cannot be extracted. For example, characteristic parts that actually exist in the human body and are depicted in MR (Magnetic Resonance) images but are not depicted in CT (Computed Tomography) images cannot be extracted even if this technique is applied to CT images.

JP 2016-509508 A JP 2007-526016 A JP 2008-12027 A

One of the problems that the embodiments disclosed in this specification and the drawings attempt to solve is to obtain more information from medical images. However, the problems that the embodiments disclosed in this specification and the drawings attempt to solve are not limited to the above problem. Problems that correspond to the effects of each configuration shown in the embodiments described below can also be positioned as other problems.

The medical image processing device according to the embodiment includes an acquisition unit, an image processing unit, and an extraction unit. The acquisition unit acquires an original image. The image processing unit performs image transformation on the original image to generate a transformed image that includes features that are difficult to recognize from the original image but are present in the subject. The extraction unit extracts feature areas from the original image and the transformed image.

FIG. 1 is a block diagram showing a medical image processing apparatus according to the first embodiment. FIG. 2 is a flowchart showing the process of extracting a characteristic portion of the medical image processing apparatus according to the first embodiment. FIG. 3 is a diagram showing a specific operation example of the medical image processing apparatus according to the first embodiment. FIG. 4 is a diagram showing a specific operation example of the medical image processing apparatus according to the modified example of the first embodiment. FIG. 5 is a flowchart showing the image registration process of the medical image processing apparatus according to the second embodiment. FIG. 6 is a diagram showing a specific operation example of the medical image processing apparatus according to the second embodiment. FIG. 7 is a flowchart showing an example of the operation of the medical image processing apparatus according to the third embodiment. FIG. 8 is a diagram showing a specific example of a guide image generated by the medical image processing apparatus according to the third embodiment. FIG. 9 is a block diagram showing an X-ray CT apparatus, which is a medical image diagnostic apparatus according to the fourth embodiment.

The medical image processing device, medical image diagnostic device, and medical image processing program according to this embodiment will be described below with reference to the drawings. In the following embodiment, parts with the same reference numerals perform similar operations, and duplicated descriptions will be omitted as appropriate. One embodiment will be described below with reference to the drawings.

The medical image processing device according to the embodiment described below may be included in a workstation or in a server such as a cloud server.

First Embodiment
A medical image processing apparatus according to a first embodiment will be described with reference to the block diagram of FIG.
The medical image processing apparatus 1 according to the first embodiment includes a memory 11, a processing circuitry 13, an input interface 15, and a communication interface 17. The processing circuitry 13 includes an acquisition function 131, an image processing function 133, an extraction function 135, and a display control function 137.

The memory 11 is a storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or integrated circuit storage device that stores various information. In addition to an HDD or SSD, the memory 11 may be a drive device that reads and writes various information between a portable storage medium such as a CD (Compact Disc), DVD (Digital Versatile Disc), or flash memory, or a semiconductor memory element such as a RAM (Random Access Memory). The memory storage area may be within the medical image processing device 1 or may be within an external storage device connected via a network. The memory 11 is intended to store trained models, various medical data (raw data, projection data, intermediate data such as sinograms, etc.), and various medical images (reconstructed images, computed tomography (CT) images, magnetic resonance (MR) images, ultrasound images, positron emission tomography (PET) images, single photon emission CT (SPECT) images, etc.), but these trained models, medical data, medical images, etc. may be stored externally.

The processing circuit 13 includes, for example, a processor such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a GPU (Graphics Processing Unit), and a memory such as a ROM (Read Only Memory) or a RAM (Random Access Memory) as hardware resources. The processing circuit 13 may also be realized by an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), other Complex Programmable Logic Devices (CPLDs), or Simple Programmable Logic Devices (SPLDs). The processing circuit 13 executes the acquisition function 131, the image processing function 133, the extraction function 135, and the display control function 137 by a processor that executes a program deployed in the memory. Note that each function (acquisition function 131, image processing function 133, extraction function 135, and display control function 137) is not limited to being realized by a single processing circuit. The processing circuit 13 may be configured by combining multiple independent processors, with each processor executing a program to achieve each function.

The processing circuit 13 acquires an input image (hereinafter, referred to as an original image) using the acquisition function 131.
Using the image processing function 133, the processing circuitry 13 performs image transformation on the original image to generate a transformed image that includes features that are difficult to recognize from the original image but are present in the subject.
The processing circuitry 13 extracts features from the original image and the converted image using an extraction function 135. The features may be, for example, anatomical features or lesions such as tumors.
The processing circuitry 13 displays information indicating the characteristic parts on at least one of the original image and the converted image by the display control function 137. For example, the processing circuitry 13 may control the output of the image so that an image in which the characteristic parts extracted from the original image and the characteristic parts extracted from the converted image are superimposed on the original image is displayed on a screen or the like via a display or projector.

The input interface 15 accepts various input operations from the user and outputs signals based on the accepted input operations to the memory 11, the processing circuit 13, the communication interface 17, etc. For example, a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touchpad, a touch panel display, etc. can be used as appropriate. Note that in this embodiment, the input interface 15 is not limited to one equipped with physical operating parts such as a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touchpad, and a touch panel display. For example, a processing circuit that receives a signal corresponding to an input operation from an external input device provided separately from the device and outputs this signal to the processing circuit 13 is also included as an example of the input interface 15.

The communication interface 17 is a wireless or wired interface for communicating with the outside world, and a general interface can be used, so a description of it will be omitted here.

Next, the extraction process of a characteristic portion by the medical image processing apparatus 1 according to the first embodiment will be described with reference to the flowchart of FIG.
In step S21, the processing circuitry 13 acquires an original image by the acquisition function 131. For example, the original image may be a positioning image (also called a scanogram image) captured by a positioning scan in the case of a CT device. Note that the original image is not limited to the positioning image, and may be a CT image captured by a main scan.
In addition, the original image does not have to be a CT image as long as the characteristic portion can be extracted. For example, the original image may be an MR image, a CT image, an X-ray image, or another medical image diagnostic device. Alternatively, the original image is not limited to an image taken by a medical image diagnostic device, but may be an image obtained by a photographing device such as a thermograph, an image obtained by an optical camera, or even a photograph published in a medical book.

In step S22, the processing circuitry 13 performs image conversion on the original image to generate a converted image using the image processing function 133. The converted image may be an image that includes features that are difficult to recognize from the original image but exist in the subject, as described above, in other words, an image from which a feature portion different from a feature portion that may be extracted from the original image may be extracted.
As a specific example, which will be described later with reference to Figure 3, the original image is an image obtained by photographing a subject using a first medical image diagnostic device, and the converted image is an estimated image assuming that the subject is photographed using a second medical image diagnostic device different from the first medical image diagnostic device.
For example, if the original image is an image captured by a CT device, a trained model that converts the image into an image captured by a medical image diagnostic device (such as an MRI device) with a different imaging principle is generated by deep learning or the like. Examples of the network model include image generation networks such as DCNN (Deep Convolutional Neural Network) and pix2pix. By applying the trained model to the original image, which is a CT image, an MR image can be obtained as a converted image.

In addition, the converted image may be generated by applying image processing to the original image, not limited to applying the trained model to the original image. For example, the original image may be subjected to edge enhancement processing, and the edge-enhanced image may be used as the converted image. Even with such image processing, it is possible to obtain an image with a luminance value and texture different from that of the original image, and therefore it is possible to calculate a feature portion different from that of the original image.
In addition, the number of converted images is not limited to one, and multiple different types of converted images may be generated. In addition, the image size of the converted image does not matter, and the converted image may be the same size as the original image or may be a different size.

The conversion process from an original image to a transformed image may create an image of any dimension from an original image of any dimension. For example, a three-dimensional transformed image may be generated from a three-dimensional original image, a cross-sectional image may be generated from a cross-sectional image, or a two-dimensional cross-sectional image may be generated from a three-dimensional original image. Also, a three-dimensional image may be generated from a two-dimensional cross-sectional image.

Furthermore, an image at a position or cross section different from that of the original image may be generated as the converted image.
For example, an existing cross-section estimation process may be used to estimate an image of a cross-section that is different from the position of the original image, or an image of a cross-section that is shifted in the body axis direction, so that an image that is orthogonal to the position of the original image is estimated as the transformed image.

In step S23, the processing circuit 13 extracts characteristic parts from the original image and the converted image using the extraction function 135. The extraction process of the characteristic parts can use existing extraction methods, so a detailed description will be omitted. The characteristic parts may be calculated as points, or as regions having a volume or area. For example, if the characteristic parts are extracted as points, the region may be calculated by performing general segmentation processing based on the extracted points.

The extracted information is not limited to information about characteristic parts, and statistical values such as volume or area may be calculated. Furthermore, information that cannot be obtained from either the original image or the converted image alone, but can be calculated by combining the two, may be extracted and generated. For example, information that can be determined by superimposing or subtracting CT images and MR images may be used. Alternatively, identifiable medical information may be calculated based on multiple image features extracted by pattern recognition from a Ragiogenomics perspective that combines radiation information, genetic information, and multivariate analysis.

In step S24, the processing circuit 13 uses the display control function 137 to display information about the extracted characteristic parts on at least one of the original image and the converted image. For example, when the characteristic parts are superimposed on the original image, the characteristic parts extracted from the converted image are displayed on the original image in addition to the characteristic parts extracted from the original image itself.

The information about the characteristic parts may be displayed using at least one of symbols, characters, colors, and boundaries. The characteristic parts may be displayed in a distinguishable manner, such as by changing the color depending on whether the characteristic part was extracted from the original image or the converted image.

The extracted characteristic features and the image in which the characteristic features are superimposed may be stored in at least one of the memory 11 and an external database (not shown).

Next, a specific example of the operation of the medical image processing apparatus 1 according to the first embodiment will be described with reference to FIG.
In the example of Fig. 3, a CT image of the subject's head taken by an X-ray CT device is used as the original image, and an MR image assuming that the same cross-sectional position of the subject's head is taken by an MRI device is used as the converted image. Note that the MR image assumed as the converted image is not an actual MR image of the subject taken by an MR device, but an image converted from a CT image, and is therefore referred to as an estimated MR image here. Also, a case is assumed in which anatomical characteristic parts are extracted as characteristic parts.

As shown in FIG. 3, the processing circuit 13 uses the image processing function 133 to convert the original image, CT image 31, by applying a trained model for converting a CT image to an MR image, for example, to generate an estimated MR image 32.

Then, the processing circuit 13 performs feature extraction processing using the extraction function 135 to extract anatomical features from the CT image 31. Here, it is assumed that the brain stem is extracted as the anatomical feature 33 from the CT image 31. In the example of FIG. 3, the anatomical feature 33 is displayed as a circular area surrounding the shaded area.

Furthermore, the processing circuit 13 performs feature extraction processing using the extraction function 135 to extract anatomical features from the estimated MR image 32. Here, it is assumed that the choroid plexus, a type of capillary network, is extracted as the anatomical feature 34 from the estimated MR image 32. Since MR images depict blood vessels and soft tissues more clearly than CT images, it is possible to extract anatomical features from the estimated MR image 32 that are difficult to extract from the CT image 31. In the example of FIG. 3, the anatomical feature 34 is displayed as a circular area surrounding a white area.

The display control function 137 causes the processing circuit 13 to superimpose the anatomical feature 33 and the anatomical feature 34 onto the CT image 31. As a result, a display image 35 is generated as the image to be displayed. As shown in the display image 35, the display format of the anatomical feature is displayed differently depending on the source from which the anatomical feature was extracted, so that it is possible to understand at a glance what image the anatomical feature was obtained based on.

The processing of the part surrounded by the thick line in Fig. 3 may be configured by a trained model. That is, a trained model may be constructed in which a network model such as a multi-layer convolutional neural network (DCNN) is trained based on training data in which the original image (CT image 31) is input data and a display image 35 in which the anatomical feature 33 and the anatomical feature 34 are superimposed on the CT image 31 is used as correct answer data, and a trained model is constructed in which the display image 35 is output from the CT image 31. This makes it possible to calculate the anatomical feature extracted from the CT image and the estimated MR image directly from the CT image.
In addition, when extracting a lesion as a characteristic site, for example, when extracting osteosarcoma as a lesion, the formation state of osteosarcoma inside the bone cannot be fully grasped with an X-ray image taken with an X-ray imaging device or a CT image taken with an X-ray CT device. On the other hand, an MR image taken with an MRI device can depict the state inside the bone more clearly than an X-ray image or a CT image, so that by generating an MR image as a converted image, it is possible to achieve the same effect as when extracting an anatomical characteristic site.

According to the first embodiment described above, by extracting characteristic parts from a converted image obtained by image-converting an original image, it is possible to extract characteristic parts that are difficult to extract from the original image. This makes it possible to obtain more information from the medical image of the original image.

As a result, for example, the extraction result of the characteristic parts according to this embodiment can be used as an index for adjusting the imaging conditions in the medical image diagnostic device, specifically the scan conditions such as the scan position and dose in the case of an X-ray CT device, based on the information on the characteristic parts. Furthermore, if the processing circuit can extract anatomical characteristic parts at a threshold value or more by the display control function, the extraction result can be used as an index for automatic selection processing, such as displaying the image as an appropriate image to be displayed, or storing the image in a database as an appropriate image for diagnosis.
Furthermore, the information on the characteristic parts extracted from the original image and the converted image can be used when the image is captured by a medical image diagnostic device different from the medical image diagnostic device that captured the original image. For example, the position and area of human tissue are extracted from a CT image captured by an IVR (Interventional Radiology)-CT and a converted image of the CT image. When X-ray photography is then performed by an X-ray diagnostic device, a corrected X-ray image can be generated based on the information on the position and area of the human tissue. Specifically, for example, a tissue area that hinders diagnosis or treatment can be extracted as a characteristic part, and the pixel value of the tissue area on the X-ray image can be corrected to improve visibility.

(Modification of the first embodiment)
In a modified example of the first embodiment, the original image is an image captured under a first shooting condition, and the converted image is an estimated image assuming that the image is captured under a second shooting condition different from the first shooting condition.

A specific operation example of the medical image processing apparatus 1 according to the modified example of the first embodiment will be described with reference to FIG.
4 is a diagram showing an example of generating a contrast image as a converted image by taking a non-contrast image of a cross section of the abdomen including the liver of a subject using an X-ray CT scanner with a contrast agent from the non-contrast image of the same cross section using an X-ray CT scanner. Note that the contrast image assumed as the converted image is an image obtained by image conversion, not an image actually taken by injecting a contrast agent into the subject from whom the non-contrast image was taken, and is therefore called an estimated contrast image here. The characteristic parts are assumed to be anatomical characteristic parts.

4, the image processing function 133 causes the processing circuitry 13 to perform image conversion on a non-contrast image 41, which is an original image, by applying a trained model that converts a non-contrast image into a contrast image, for example, to generate an estimated contrast image 42. The trained model may be constructed by training a network model such as a multi-layer convolutional neural network (DCNN) based on learning data in which a non-contrast image is input data and a contrast image is used as correct answer data, and outputting a converted image from the original image.
The learning data can be obtained from images of a normal contrast examination in which time-phase images are taken. Since an image of a time phase before the contrast agent has sufficiently flowed can be treated in the same way as a non-contrast image, the learning data can be prepared by using an image taken at a time phase in which the contrast agent has not sufficiently flowed as input data and a contrast image taken during the period in which the contrast agent is flowing in the tissue as correct answer data. The contrast image used as the correct answer data may be one contrast image taken at a specific time phase, or may be an image calculated by arithmetic processing such as averaging or maximum value processing from contrast images taken at multiple time phases.

Next, the processing circuitry 13 performs a feature extraction process using the extraction function 135 to extract anatomical features from the non-contrast image 41. Here, it is assumed that blood vessels are not extracted from the non-contrast image 41.
Furthermore, the processing circuitry 13 performs a feature extraction process using the extraction function 135 to extract anatomical features from the estimated contrast image 42. Here, it is assumed that blood vessels including the portal vein are extracted as anatomical features 44 from the estimated contrast image 42. Since blood vessels are depicted more clearly in a contrast image compared to a non-contrast image, it is possible to extract anatomical features from the estimated contrast image 42 that are difficult to extract from the non-contrast image 41. In the example of FIG. 4 , the anatomical features 44 are displayed as a circular region surrounding a white region.

The display control function 137 causes the processing circuitry 13 to superimpose the anatomical feature 44 onto the non-contrast image 41. As a result, a display image 45 is generated as the image to be displayed.

As in the case of FIG. 3, the processing of the part surrounded by the thick line in FIG. 4 may be configured with a trained model. That is, a trained model may be constructed in which a network model such as a multi-layer convolutional neural network (DCNN) is trained based on training data in which the original image (non-contrast image 41) is used as input data, and display image 45, in which anatomical features extracted from non-contrast image 41 and estimated contrast image 42 are superimposed on non-contrast image 41, is used as correct answer data, and a trained model is constructed in which a display image is output from the original image. This makes it possible to calculate anatomical features extracted from the estimated contrast image directly from the non-contrast image.

Although the non-contrast image is used as the original image, the contrast image may be used as the original image. In this case, the processing circuitry 13 performs image processing using the image processing function 133 to generate, as a converted image, a contrast image of a different time phase from the time phase of the contrast image used as the original image.

In the case of real prep, which is an imaging technique in which an image is taken after a certain amount of contrast agent has flowed into a blood vessel, it is necessary to set the blood vessel position as a ROI (Region of Interest) in advance. For this reason, a reference image scan is generally performed before the actual imaging. In the reference image scan, a reference image is generated by injecting a contrast agent into the subject and imaging the subject in a state in which the area of interest is imaged. The ROI is set for the blood vessel of interest depicted in the reference image.
According to the modification of the first embodiment, by using an estimated contrast image obtained by processing according to this modification, an ROI can be easily set by a user or automatically without using a contrast agent.

Second Embodiment
The medical image processing apparatus according to the second embodiment differs from the first embodiment in that the extraction results of anatomical features are used for image registration.
The configuration of the medical image processing apparatus according to the second embodiment is similar to the configuration of the medical image processing apparatus shown in FIG. 1, and therefore a description thereof will be omitted.

An example of the operation of the medical image processing apparatus 1 according to the second embodiment will be described with reference to the flowchart of FIG.
In step S51, the processing circuit 13 acquires the original image using the acquisition function 131.
In step S52, the processing circuitry 13 uses the acquisition function 131 to acquire an alignment target image that is to be subjected to alignment processing with respect to the original image.

In step S53, the processing circuit 13 performs image conversion on at least one of the original image and the image to be aligned by the image processing function 133 to generate a converted image. Since the converted image according to the second embodiment is intended to be used for image alignment, the converted image may be converted into an image that may be capable of calculating a feature portion that can be used for image alignment with the original image. For example, if the original image is an image taken by a CT device, it is considered that blood vessels and soft tissues are not clearly depicted, so it is sufficient to convert the image into an MR image or a contrast image that can clearly depict blood vessels and soft tissues compared to a CT image. In addition, when extracting a lesion such as a tumor as a feature portion, it is unclear in an X-ray image, but it is considered that the lesion can be more clearly confirmed with other medical image diagnostic devices such as an X-ray CT device or an MRI device, so it is sufficient to generate a converted image according to the purpose of diagnosis.
In addition, to generate the transformed image, a trained model may be used as in the first embodiment, or other image transformations may be used.

In step S54, similar to step S23 in FIG. 2, the processing circuitry 13 extracts anatomical features from each of the original image, the registration target image, and the converted image using the extraction function 135.
In step S55, the processing circuitry 13 performs image registration between the original image and the registration target image based on the extracted anatomical features by the display control function 137. Since a general image registration method may be used as the image registration method, a description thereof will be omitted here.

Next, a specific example of the operation of the medical image processing apparatus 1 according to the second embodiment will be described with reference to FIG.
6, a case is assumed in which image registration is performed between a CT image of the abdomen of a subject and an ultrasound image. A CT image 61 is acquired as an original image, and an ultrasound image 62 is acquired as an image to be registered. In addition, an anatomical feature is assumed as a feature.

The processing circuitry 13 performs image conversion using the image processing function 133 on the CT image 61, which is the original image, by the same processing as that shown in FIG.
Next, the processing circuitry 13 performs feature extraction processing using the extraction function 135 to extract anatomical features from each of the CT image 61, the ultrasound image 62, and the estimated MR image 63. Here, it is assumed that an anatomical feature 64 is extracted from the CT image 61, an anatomical feature 65 related to blood vessels from the ultrasound image 62, and an anatomical feature 66 related to blood vessels from the estimated MR image 63.

The display control function 137 causes the processing circuit 13 to perform image registration processing between the CT image 61 and the ultrasound image 62 based on the anatomical features 64, 65, and 66, and generate a display image 67. As shown in FIG. 6, in the image registration, an anatomical feature corresponding to the anatomical feature 65 of the blood vessel extracted in the ultrasound image 62 is not extracted from the CT image 61. However, the corresponding position of the anatomical feature 65 of the ultrasound image 62 can be estimated using the anatomical feature 66 of the blood vessel extracted in the estimated MR image 63 as a clue, so that image registration between the CT image 61 and the ultrasound image 62 can be performed at a position corresponding to the estimated MR image 63.

According to the second embodiment described above, even if image registration is difficult using only the characteristic parts of the original image, it is possible to perform image registration by generating a converted image and extracting the characteristic parts from the converted image. For example, image registration between a CT image and an ultrasound image can be performed based on characteristic parts that cannot be confirmed in a CT image but can be confirmed in an MR image, thereby realizing fusion processing.

Third Embodiment
The third embodiment differs from the above-mentioned embodiments in that a reference image to be acquired in the next examination and a guide image indicating imaging conditions including a position where the reference image can be acquired on the original image are displayed based on the extracted anatomical characteristic site. For example, when a lesion such as a tumor is confirmed from a CT image, an ultrasound image of a cross section to be acquired by an ultrasound examination can be displayed as a reference image and a guide image from the characteristic site related to the lesion.
The configuration of the medical image processing apparatus according to the third embodiment is similar to the configuration of the medical image processing apparatus shown in FIG. 1, and therefore a description thereof will be omitted.

An example of the operation of the medical image processing apparatus according to the third embodiment will be described with reference to the flowchart of Fig. 7. The processes from step S21 to step S23 are similar to those in Fig. 2.
In step S71, the display control function 137 causes the processing circuitry 13 to display a guide image based on the extracted anatomical feature, and displays the converted image as a reference image.

Next, a specific example of a guide image generated by the medical image processing apparatus 1 according to the third embodiment will be described with reference to FIG.
In Fig. 8, a CT image 81, which is an original image, and a reference image 82, which is an ultrasound image serving as a reference for the cross section to be acquired, are displayed in parallel. A guide image 83, which indicates the imaging conditions under which the reference image 82 should be captured, is superimposed on the CT image 81. Note that, although the guide image 83 in Fig. 8 shows an example in which the position and imaging angle are displayed, other imaging conditions such as the focus position may also be displayed. A message 84 encouraging an ultrasound examination is also displayed. Along with the message 84, a button 85 may be designed so that clicking or touching the button will take the user to an examination order screen and an examination order can be issued immediately.

In the example of FIG. 8, the CT image 81 is used as the original image, and a tumor is extracted as a characteristic part. When the extracted characteristic part is to be examined by ultrasound examination, the processing circuit 13 converts the CT image 81 into an ultrasound image and displays the converted image as the reference image 82 by the image processing function 133 and the display control function 137 in order to display how the characteristic part would be depicted in an ultrasound image. In addition, by using the extracted characteristic part, the correspondence relationship between the positions of the tumor in the CT image 81 and the reference image 82 can be known, and therefore the position at which the reference image 82 should be photographed can be displayed as a guide image 83 on the CT image 81. Note that in the example of FIG. 8, the guide image 83 is displayed only as an outer frame, but the reference image 82 may be displayed superimposed on the CT image 81.

In addition, an ideal ultrasound image for use in diagnosis may be displayed together with or instead of the converted image as the reference image 82. For example, if a tumor is malignant, it is considered that the relationship between the tumor and the nutrient blood vessels is easily clarified by a color Doppler image or a power Doppler image, an elastography image, or the like obtained by an ultrasound examination. Therefore, by displaying an example of an ideal ultrasound image as the reference image 82, the user can easily obtain an ideal ultrasound image.
It should be noted that the reference image 82 and the guide image 83 may be displayed in a format in which either one of them is displayed.

According to the third embodiment described above, by displaying at least one of the reference image and the guide image together with the original image, it is possible to encourage the acquisition of images necessary for diagnosis and to assist in the acquisition of ideal images and diagnosis.

Fourth Embodiment
A medical image diagnostic apparatus according to the fourth embodiment will be described with reference to the block diagram of Fig. 9. In the fourth embodiment, an X-ray CT apparatus will be described as an example of the medical image diagnostic apparatus, but any type of medical image diagnostic apparatus may be used. For example, in addition to an X-ray CT apparatus, an MRI apparatus, a PET apparatus, a SPECT apparatus, or a combination of these may be used.

The X-ray CT device 2 shown in FIG. 9 has a gantry device 20, a bed device 300, and a console device 40 that realizes the processing of the X-ray CT device. For the sake of explanation, multiple gantry devices 20 are drawn in FIG. 9.

In this embodiment, the rotation axis of the rotating frame 23 in the non-tilted state or the longitudinal direction of the table top 330 of the bed device 300 is defined as the Z-axis direction, the axial direction perpendicular to the Z-axis direction and horizontal to the floor surface is defined as the X-axis direction, and the axial direction perpendicular to the Z-axis direction and perpendicular to the floor surface is defined as the Y-axis direction.
For example, the gantry device 20 and the bed device 300 are installed in a CT examination room, and the console device 40 is installed in a control room adjacent to the CT examination room. Note that the console device 40 does not necessarily have to be installed in the control room. For example, the console device 40 may be installed in the same room as the gantry device 20 and the bed device 300. In any case, the gantry device 20, the bed device 300, and the console device 40 are connected to each other by wire or wirelessly so as to be able to communicate with each other.

The gantry device 20 is a scanning device configured to perform X-ray CT imaging of the subject P. The gantry device 20 includes an X-ray tube 21, an X-ray detector 22, a rotating frame 23, an X-ray high voltage device 24, a control device 25, a wedge 26, a collimator 27, and a data acquisition device 28 (hereinafter also referred to as DAS (Data Acquisition System) 28). The X-ray tube 21, the X-ray detector 22, and the DAS 28 are collectively referred to as an imaging unit that acquires medical images (here, CT images).

The X-ray tube 21 is a vacuum tube that generates X-rays by irradiating thermoelectrons from a cathode (filament) to an anode (target) through the application of high voltage from the X-ray high voltage device 24 and the supply of filament current. Specifically, X-rays are generated when the thermoelectrons collide with the target. For example, the X-ray tube 21 is a rotating anode type X-ray tube that generates X-rays by irradiating a rotating anode with thermoelectrons. The X-rays generated by the X-ray tube 21 are shaped into a cone beam, for example, via a collimator 27, and irradiated to the subject P.

In this embodiment, the X-ray detector 22 will be described in terms of both a so-called photon counting type detector and a general X-ray detector, a so-called integral type detector.

When the X-ray detector 22 is a photon-counting detector, it detects the X-rays emitted from the X-ray tube 21 and passing through the subject P in photon units. The X-ray detector 22 has, for example, multiple X-ray detection element rows in which multiple X-ray detection elements are arranged in the channel direction along one arc centered on the focal point of the X-ray tube 21. The X-ray detector 22 has, for example, a row structure in which multiple X-ray detection element rows in which multiple X-ray detection elements are arranged in the channel direction are arranged in the slice direction (row direction).

The X-ray detector 22 is specifically an indirect conversion type detector having, for example, a scintillator array, a grid, and a photosensor array. The X-ray detector 22 is an example of a detection unit.

The scintillator array has multiple scintillators. The scintillators convert incident X-rays into a number of photons that correspond to the intensity of the incident X-rays.

The grid is placed on the X-ray incident surface of the scintillator array and has an X-ray shielding plate that has the function of absorbing scattered X-rays. Note that the grid is sometimes called a collimator (one-dimensional collimator or two-dimensional collimator).

The optical sensor array has a function of amplifying the light received from the scintillator, converting it into an electrical signal, and generating an output signal (energy signal) having a peak value according to the energy of the incident X-rays, and has an optical sensor such as a photomultiplier tube (PMT). The generated energy signal is output to the DAS 28.

On the other hand, when the X-ray detector 22 is an integral type detector, it detects the X-rays irradiated from the X-ray tube 21 and passing through the subject P, and outputs an electrical signal corresponding to the amount of X-rays to the DAS 28. The scintillator array has multiple scintillators. The scintillator has scintillator crystals that receive incident X-rays and output light with a photon amount corresponding to the incident X-rays. The photosensor array has a function of amplifying the light received from the scintillator and converting it into an electrical signal, and has a photosensor such as a photomultiplier tube (PMT).

The X-ray detector 22 described above is assumed to be an indirect conversion type detector, but instead of the scintillator array and the photosensor array, it may be a direct conversion type detector having a semiconductor element that converts incident X-rays into an electrical signal.

The rotating frame 23 supports the X-ray generating unit (X-ray tube 21, wedge 26, and collimator 27) and the X-ray detector 22 so that they can rotate around the rotation axis. Specifically, the rotating frame 23 is an annular frame that supports the X-ray tube 21 and the X-ray detector 22 facing each other and rotates the X-ray tube 21 and the X-ray detector 22 using a control device 25 described below. The rotating frame 23 is rotatably supported by a fixed frame (not shown) made of a metal such as aluminum. More specifically, the rotating frame 23 is connected to the edge of the fixed frame via a bearing. The rotating frame 23 receives power from the drive mechanism of the control device 25 and rotates at a constant angular velocity around the rotation axis Z.

The rotating frame 23 further includes and supports the X-ray high voltage device 24 and the DAS 28 in addition to the X-ray tube 21 and the X-ray detector 22. Such a rotating frame 23 is housed in a substantially cylindrical housing in which an opening (bore) 19 forming an imaging space is formed. The opening substantially coincides with the FOV. The central axis of the opening coincides with the rotation axis Z of the rotating frame 23. The imaging data generated by the DAS 28 is transmitted, for example, from a transmitter having a light-emitting diode (LED) to a receiver (not shown) having a photodiode provided in a non-rotating part of the gantry (for example, a fixed frame. Illustration in FIG. 9 is omitted) by optical communication, and is then transferred to the console device 40. The method of transmitting imaging data from the rotating frame to the non-rotating part of the gantry is not limited to the optical communication described above, and any method of non-contact data transmission may be used.

The X-ray high voltage device 24 has electrical circuits such as a transformer and a rectifier, and includes a high voltage generator that has the function of generating the high voltage to be applied to the X-ray tube 21 and the filament current to be supplied to the X-ray tube 21, and an X-ray control device that controls the output voltage according to the X-rays emitted by the X-ray tube 21. The high voltage generator may be of a transformer type or an inverter type. The X-ray high voltage device 24 may be provided on the rotating frame 23 described later, or on the fixed frame (not shown) side of the gantry device 20.

The control device 25 has a processing circuit having a CPU (Central Processing Unit) and the like, and a driving mechanism such as a motor and an actuator. The processing circuit has a processor such as a CPU or an MPU (Micro Processing Unit) and a memory such as a ROM (Read Only Memory) or a RAM (Random Access Memory) as hardware resources. The control device 25 may also be realized by an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), other Complex Programmable Logic Devices (CPLDs), or Simple Programmable Logic Devices (SPLDs). The control device 25 controls the X-ray high voltage device 24 and the DAS 28, etc., according to commands from the console device 40. The processor realizes the above control by reading and executing programs stored in the memory.

The control device 25 also has a function of controlling the operation of the gantry 20 and the bed 300 upon receiving an input signal from the input interface 15 (described later) attached to the console device 40 or the gantry 20. For example, the control device 25 receives an input signal and controls the rotation of the rotating frame 23, the tilt of the gantry 20, and the operation of the bed 300 and the tabletop 330. The control of tilting the gantry 20 is realized by the control device 25 rotating the rotating frame 23 around an axis parallel to the X-axis direction based on the inclination angle (tilt angle) information input by the input interface 15 attached to the gantry 20. The control device 25 may be provided in the gantry 20 or in the console device 40. The control device 25 may be configured to directly incorporate the program into the circuit of the processor instead of storing the program in the memory. In this case, the processor realizes the above control by reading and executing the program incorporated in the circuit.

The wedge 26 is a filter for adjusting the amount of X-rays irradiated from the X-ray tube 21. Specifically, the wedge 26 is a filter that transmits and attenuates the X-rays irradiated from the X-ray tube 21 so that the X-rays irradiated from the X-ray tube 21 to the subject P have a predetermined distribution. For example, the wedge 26 (wedge filter, bow-tie filter) is a filter made of processed aluminum to have a predetermined target angle and a predetermined thickness.

The collimator 27 is a lead plate or the like that narrows the irradiation range of the X-rays that have passed through the wedge 26, and a slit is formed by combining multiple lead plates or the like. The collimator 27 is sometimes called an X-ray aperture.

The DAS 28 is realized, for example, by an ASIC (Application Specific Integrated Circuit) equipped with circuit elements capable of generating imaging data. The DAS 28 and the X-ray detector 22 constitute a detector unit.

When the X-ray detector 22 is a photon-counting detector, the DAS 28 generates digital data (hereinafter also referred to as spectrum data) indicating the count of X-rays detected by the X-ray detector 22 for each of a number of energy bands (hereinafter also referred to as energy bins or simply bins). The spectrum data is a set of count value data identified by the channel number of the detector element from which it was generated, the row number, the view number indicating the collected view (also referred to as the projection angle), and the energy bin number. The spectrum data is transferred to the console device 40.

On the other hand, if the X-ray detector 22 is an integral detector, the DAS 28 reads out an electrical signal from the X-ray detector 22 and generates projection data, which is digital data relating to the X-ray dose detected by the X-ray detector 22, based on the read out electrical signal. The projection data is a set of data indicating the channel number, row number, and view number indicating the projection angle of the X-ray detection element from which the projection data was generated, and the integral value of the detected X-ray dose.

For example, when the X-ray detector 22 is an integral detector, the DAS 28 includes a preamplifier, a variable amplifier, an integration circuit, and an A/D converter for each detection element. The preamplifier amplifies the electrical signal from the connected detection element with a predetermined gain. The variable amplifier amplifies the electrical signal from the preamplifier with a variable gain. The integration circuit integrates the electrical signal from the preamplifier over one view period to generate an integrated signal. The peak value of the integrated signal corresponds to the X-ray dose value detected by the connected detection element over one view period. The A/D converter performs analog-to-digital conversion of the integrated signal from the integration circuit to generate projection data. Hereinafter, when there is no distinction between spectrum data and projection data, they are referred to as imaging data.

The bed device 300 is a device on which a subject P to be scanned is placed and moved, and includes a base 310 , a bed driving device 320 , a top board 330 , and a support frame 340 .
The base 310 is a housing that supports the support frame 340 so that the support frame 340 can move in the vertical direction.
The bed driving device 320 is a motor or actuator that moves the tabletop 330 on which the subject P is placed in the longitudinal direction of the tabletop 330. The bed driving device 320 moves the tabletop 330 under the control of the console device 40 or the control of the control device 25. For example, the bed driving device 320 moves the tabletop 330 in a direction perpendicular to the subject P so that the body axis of the subject P placed on the tabletop 330 coincides with the central axis of the opening of the rotating frame 23. The bed driving device 320 may also move the tabletop 330 along the body axis direction of the subject P in accordance with the X-ray CT imaging performed using the gantry device 20. The bed driving device 320 generates power by driving at a rotation speed according to the duty ratio of a drive signal from the control device 25. The bed driving device 320 is realized by a motor such as a direct drive motor or a servo motor.

The top plate 330 provided on the upper surface of the support frame 340 is a plate on which the subject P is placed. The bed driving device 320 may move the support frame 340 in the longitudinal direction of the top plate 330 in addition to the top plate 330.

The console device 40 has a memory 11, a display 401, an input interface 15, and a processing circuit 13. Data communication between the memory 11, the display 401, the input interface 15, and the processing circuit 13 is performed via a bus. Note that although the console device 40 is described as being separate from the gantry device 20, the gantry device 20 may include the console device 40 or some of the components of the console device 40.

The memory 11 is the same as in the first embodiment and is a storage device such as an HDD, SSD, or integrated circuit storage device that stores various information. The memory 11 stores, for example, imaging data and reconstructed image data.

The display 401 displays various information. For example, the display 401 outputs medical images (CT images) generated by the processing circuit 13, a GUI (Graphical User Interface) for receiving various operations from the operator, and the like. For example, the display 401 may be a liquid crystal display (LCD), a cathode ray tube (CRT) display, an organic electroluminescence display (OELD), a plasma display, or any other display, as appropriate. The display 401 may also be provided on the pedestal device 20. The display 401 may also be a desktop type, or may be configured as a tablet terminal capable of wireless communication with the console device 40 main body.

The input interface 15 is similar to that in the first embodiment, and accepts various input operations from the operator, converts the accepted input operations into electrical signals, and outputs the electrical signals to the processing circuit 13. For example, the input interface 15 accepts from the operator the acquisition conditions for acquiring imaging data, the reconstruction conditions for reconstructing CT images, the image processing conditions for generating post-processed images from CT images, and the like.

The processing circuitry 13 controls the operation of the entire X-ray CT device 2 in response to electrical signals of input operations output from the input interface 15. For example, the processing circuitry 13 has a processor such as a CPU, MPU, or GPU, and a memory such as a ROM or RAM, as hardware resources. The processing circuitry 13 executes an acquisition function 131, an image processing function 133, an extraction function 135, a display control function 137, and a system control function 139 by a processor that executes a program deployed in the memory. The acquisition function 131, the image processing function 133, the extraction function 135, and the display control function 137 perform the same processing as the medical image processing device 1 according to the first embodiment.

The system control function 139 controls each function of the processing circuitry 13 based on an input operation received from an operator via the input interface 15. Specifically, the system control function 139 reads out a control program stored in the memory 11, expands it on the memory in the processing circuitry 13, and controls each part of the X-ray CT apparatus 2 according to the expanded control program. For example, the processing circuitry 13 controls each function of the processing circuitry 13 based on an input operation received from the operator via the input interface 15, and executes a positioning scan (also called scanography) of the subject P for determining a scan range, imaging conditions, etc., and a main scan.
Although the console device 40 has been described as a single console that executes multiple functions, multiple functions may be executed by separate consoles.
According to the fourth embodiment described above, since the medical image diagnostic device includes the same configuration as the medical image processing device according to the first embodiment, it is possible to generate a converted image based on a medical image captured by the medical image diagnostic device and extract characteristic parts from the converted image, thereby obtaining more information from the medical image.

The functions according to the above-described embodiments can also be realized by installing a program that executes the relevant process in a computer such as a workstation and expanding the program in memory. In this case, the program that can cause the computer to execute the relevant method can also be stored and distributed on a storage medium such as a magnetic disk (such as a hard disk), an optical disk (such as a CD-ROM, DVD, Blu-ray (registered trademark) disk), or a semiconductor memory.

At least one of the above-described embodiments allows more information to be obtained from medical images.

Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are within the scope of the invention and its equivalents as set forth in the claims, as well as the scope and gist of the invention.

LIST OF SYMBOLS 1 Medical image processing device 11 Memory 13 Processing circuit 15 Input interface 17 Communication interface 19 Opening 20 Stand device 21 X-ray tube 22 X-ray detector 23 Rotating frame 24 X-ray high voltage device 25 Control device 26 Wedge 27 Collimator 28 Data acquisition device 31, 61, 81 CT image 32, 63 Estimated MR image 33, 34, 64, 65, 66 Anatomical feature 35 Display image 40 Console device 41 Non-contrast image 42 Estimated contrast image 45, 67 Display image 62 Ultrasound image 82 Reference image 83 Guide image 84 Message 85 Button 131 Acquisition function 133 Image processing function 135 Extraction function 137 Display control function 139 System control function 300 Bed device 310 Base 320 Bed drive device 330 Top plate 340 Support frame 401 Display

Claims (9)

An acquisition unit for acquiring an original image;
an image processing unit that performs image processing on the original image to generate a converted image including features that are difficult to recognize from the original image but are present in the subject;
an extraction unit that extracts feature portions from the original image and the converted image;
a display control unit that displays information about the characteristic portion in at least one of the original image and the converted image;
Equipped with
The image processing, excluding a conversion process to a medical image having a different imaging principle from that of the original image,
the display control unit displays the converted image as a reference image to be acquired in a next examination, and superimposes a guide image indicating an imaging condition including a position at which the reference image can be acquired on the original image based on the extracted characteristic portion.
Medical imaging equipment. the display control unit displays information about the characteristic portion in a superimposed manner on at least one of the original image and the converted image.
The medical image processing device according to claim 1 . The extraction unit extracts at least one of an anatomical feature and a lesion as the feature.
The medical image processing device according to claim 2 . the original image is an image obtained by imaging a subject using a first medical image diagnostic apparatus,
the converted image is an estimated image assuming a case where the subject is imaged using a second medical image diagnostic apparatus different from the first medical image diagnostic apparatus;
The medical image processing device according to claim 1 . the original image is an image captured under a first shooting condition,
The converted image is an estimated image assuming a case where the converted image is photographed under a second photographing condition different from the first photographing condition.
The medical image processing device according to claim 1 . The original image is a non-contrast image,
The transformed image is an estimated image assuming a contrast image.
The medical image processing device according to claim 5 . The acquisition unit further acquires a target image that is to be subjected to image registration with respect to the original image,
The display control unit performs image registration between the original image and the target image using the characteristic portion extracted from the converted image as a clue.
The medical image processing device according to claim 1 . A photographing unit that photographs an original image;
an image processing unit that performs image processing on the original image to generate a converted image including features that are difficult to recognize from the original image but are present in the subject;
an extraction unit that extracts feature portions from the original image and the converted image;
a display control unit that displays information about the characteristic portion in at least one of the original image and the converted image;
Equipped with
The image processing, excluding a conversion process to a medical image having a different imaging principle from that of the original image,
the display control unit displays the converted image as a reference image to be acquired in a next examination, and superimposes a guide image indicating an imaging condition including a position at which the reference image can be acquired on the original image based on the extracted characteristic portion.
Medical imaging diagnostic equipment. On the computer,
An acquisition function to acquire the original image;
an image processing function that performs image processing on the original image to generate a transformed image that includes features that are difficult to recognize from the original image but are present in the subject;
an extraction function for extracting feature portions from the original image and the converted image;
a display control function for displaying information about the characteristic portion in at least one of the original image and the converted image;
A medical image processing program for realizing the above,
The image processing, excluding a conversion process to a medical image having a different imaging principle from that of the original image,
the display control function displays the converted image as a reference image to be acquired in a next examination, and superimposes a guide image indicating an imaging condition including a position where the reference image can be acquired on the original image based on the extracted characteristic portion.
A medical image processing program.