JP7225008B2 - medical image processor - Google Patents

Description

An embodiment of the present invention relates to a medical image processing apparatus.

In recent years, regional cooperation in medical information is progressing. In such a case, for example, medical data including medical images are shared among medical institutions that are cooperating in regional cooperation. However, when different patient IDs are assigned to the same patient for each medical institution, it becomes important to identify the subject with respect to multiple medical images. If a subject is mixed up, there is a risk of misdiagnosis and unnecessary exposure of the subject.

To solve this problem, the similarity between the anatomical landmarks extracted from the current subject image and the landmarks extracted from the past subject images stored in the database is calculated to Techniques for identifying patients are known. Such techniques do not take into consideration changes in the subject over time. Therefore, there is a demand for a technique for identifying a subject with high accuracy without being affected by changes in the subject over time.

JP 2017-167965 A

Christopher M. Bishop, Pattern recognition and machine learning, (USA), 1st ed., Springer, 2006, pp. 225-290

The problem to be solved by the present invention is to improve the accuracy of identifying a subject with respect to a plurality of medical data.

A medical image processing apparatus according to an embodiment includes a prediction unit and a determination unit.
The prediction unit receives first medical data acquired at a first age regarding an imaging target region of a first subject, and determines the imaging target region from the first age to a second age different from the first age. The first medical data is input to a trained prediction model that outputs prediction data in which changes over time are reflected in the first medical data, and the learned prediction model is caused to output the prediction data.
The determination unit receives the second medical data acquired regarding the imaging target site at the second age and the predicted data, and determines whether the second subject and the first subject regarding the second medical data are the same. The second medical data and the prediction data are input to a trained determination model that outputs a determination result as to whether or not there is, and the learned determination model is caused to output the determination result.

FIG. 1 is a diagram illustrating the configuration of an X-ray diagnostic apparatus according to the first embodiment. FIG. 2 is a diagram illustrating an example of a combination of inputs and outputs in a learned prediction model by the X-ray diagnostic apparatus according to the first embodiment; FIG. 3 is a diagram illustrating an example of a combination of inputs and outputs in a learned determination model by the X-ray diagnosis apparatus according to the first embodiment; FIG. 4 is a flowchart exemplifying a processing procedure of subject identification processing by the X-ray diagnostic apparatus according to the first embodiment. FIG. 5 is a diagram illustrating the configuration of an X-ray diagnostic apparatus according to a first modification of the first embodiment; FIG. 6 is a diagram illustrating the flow of data in prediction processing and determination processing by the X-ray diagnostic apparatus according to the first modification of the first embodiment. FIG. 7 is a flowchart exemplifying the procedure of object identification processing by the X-ray diagnostic apparatus according to the first modification of the first embodiment. FIG. 8 is a diagram illustrating the configuration of an X-ray diagnostic apparatus according to a second modification of the first embodiment; FIG. 9 is a diagram exemplifying a learned prediction model and a learned determination model stored in a storage unit of an X-ray diagnostic apparatus according to a second modification of the first embodiment; FIG. 10 is a flowchart exemplifying the procedure of subject identification processing by the X-ray diagnostic apparatus according to the second modification of the first embodiment. FIG. 11 is a diagram illustrating an example of combinations of inputs and outputs in a learned prediction model by an X-ray diagnostic apparatus according to the third modification of the first embodiment; FIG. 12 is a diagram illustrating an example of a combination of inputs and outputs in a learned determination model by an X-ray diagnostic apparatus according to the third modification of the first embodiment; FIG. 13 is a diagram showing an example of learning data used to generate a learned prediction model according to the first embodiment. FIG. 14 is a diagram showing an example of learning data used to generate a learned determination model according to the first embodiment. FIG. 15 is a diagram showing an example of learning data used to generate a learned non-lesion prediction model according to the first modification of the first embodiment. FIG. 16 is a diagram showing an example of learning data used to generate a learned lesion prediction model according to the first modification of the first embodiment. FIG. 17 is a diagram showing an example of learning data used for generating a learned determination model according to the first modification of the first embodiment. FIG. 18 is a diagram illustrating the configuration of a model learning device; FIG. 19 is a flowchart illustrating a processing procedure of prediction model generation processing by the model learning device.

Hereinafter, an embodiment of a medical image diagnostic apparatus equipped with a medical image processing apparatus will be described in detail with reference to the drawings. In the following description, components having substantially the same functions and configurations are denoted by the same reference numerals, and redundant description will be given only when necessary. In the following embodiments, an example using an X-ray diagnostic apparatus as a medical image diagnostic apparatus will be described. The medical image diagnostic apparatus according to the following embodiments may be, for example, a single modality apparatus such as an X-ray diagnostic apparatus, a PET (Positron Emission Tomography)/CT apparatus, a SPECT (Single Photon Emission CT) A composite modality device such as a /CT device may also be used. A medical image diagnostic apparatus subjects a subject to medical imaging according to the imaging principle according to the modality device type, and collects medical data about the subject. The medical data is, for example, projection data, sinogram data or reconstructed images when the medical image diagnostic apparatus is an X-ray computed tomography apparatus, or coincidence data or sinogram data when the medical image diagnostic apparatus is a PET apparatus. .

(First embodiment)
FIG. 1 is a diagram showing a configuration example of an X-ray diagnostic apparatus 1 according to the first embodiment. As shown in FIG. 1, the X-ray diagnostic apparatus 1 includes an imaging device 10, a bed device 30, and a console device 40. As shown in FIG. The imaging device 10 includes a high voltage generator 11 , an X-ray generator 12 , an X-ray detector 13 , a C-arm 14 and a C-arm driving device 142 .

The high voltage generator 11 generates a high voltage to be applied between the anode and the cathode in order to accelerate thermal electrons generated from the cathode of the X-ray tube, and outputs the high voltage to the X-ray tube.

The X-ray generator 12 includes an X-ray tube that irradiates the subject P with X-rays, and an ROI (Region Of Interest) filter and X-ray diaphragm that have a function of attenuating or reducing the amount of irradiated X-rays.

An x-ray tube is a vacuum tube that produces x-rays. An X-ray tube accelerates thermal electrons emitted from a cathode (filament) with a high voltage. The X-ray tube generates X-rays by colliding these accelerated electrons with a tungsten anode.

The ROI filter is positioned between the X-ray tube and the X-ray diaphragm, and is composed of a metal plate such as copper or aluminum. The ROI filter has an open area in at least a portion, for example a central portion, and attenuates X-rays outside the open area. Therefore, the ROI filter fully transmits X-rays in the X-ray passing area of the opening area, and attenuates and transmits X-rays in other areas. The ROI filter is driven by a driving device (not shown) according to the region of interest input by the operator through the input interface 43 .

The X-ray diaphragm is positioned between the X-ray tube and the X-ray detector 13 and is composed of a lead plate as a metal plate. The X-ray diaphragm narrows down the X-rays generated by the X-ray tube so that only the region of interest of the subject P is irradiated by shielding the X-rays outside the aperture region. For example, the X-ray diaphragm has four diaphragm blades, and by sliding these diaphragm blades, the X-ray shielded area can be adjusted to any size. The diaphragm blades of the X-ray diaphragm are driven by a driving device (not shown) according to the region of interest input by the operator through the input interface 43 .

The X-ray detector 13 detects X-rays that have passed through the subject P. FIG. As such an X-ray detector 13, it is possible to use one that converts X-rays directly into charges, and one that converts X-rays into charges after converting them into light. It doesn't matter if it is. That is, the X-ray detector 13 includes, for example, a planar FPD (Flat Panel Detector) that converts X-rays that have passed through the subject P into charges and accumulates them, and a drive for reading out the charges accumulated in this FPD. and a gate driver for generating pulses. The size of the FPD is, for example, 8 to 12 inches. The FPD is configured by two-dimensionally arranging minute detection elements in the column direction and the line direction. Each detection element senses X-rays and includes a photoelectric film that generates charges according to the amount of incident X-rays, a charge storage capacitor that stores the charges generated in the photoelectric film, and a predetermined charge stored in the charge storage capacitor. It has a TFT (thin film transistor) that outputs at the timing of . Accumulated charges are sequentially read out by drive pulses supplied by the gate driver.

The C-arm 14 holds the X-ray generator 12 and the X-ray detector 13 so as to face each other with the subject P and the top plate 33 interposed therebetween, thereby performing X-ray imaging of the subject P on the top plate 33 . have a configuration that can be done. The C-arm 14 is slidably supported and rotatable around each of a plurality of rotation axes. The C-arm 14 is provided at appropriate locations with a plurality of power sources for achieving sliding and rotating motions. These power sources constitute the C-arm drive device 142 . The C-arm driving device 142 reads drive signals from the drive control function 442 and causes the C-arm 14 to slide, rotate, and linearly move.

The bed device 30 is a device for placing and moving the subject P, and includes a base 31 , a bed driving device 32 , a top plate 33 and a support frame 34 .

The base 31 is a housing that is installed on the floor and supports the support frame 34 so as to be movable in the vertical direction (Z direction).

The bed driving device 32 is a motor or actuator that is housed in the housing of the bed device 30 and moves the table 33 on which the subject P is placed in the longitudinal direction (Y direction) of the table 33 . The bed drive device 32 reads the drive signal from the drive control function 442 and moves the table top 33 horizontally or vertically with respect to the floor surface. By moving the C-arm 14 or the top board 33, the positional relationship of the imaging axis with respect to the subject P changes. Note that the bed driving device 32 may move the support frame 34 in the longitudinal direction of the top plate 33 in addition to the top plate 33 .

The top plate 33 is a plate provided on the upper surface of the support frame 34 and on which the subject P is placed.

The support frame 34 is provided above the base 31 and supports the top plate 33 so as to be slidable along its longitudinal direction.

In the bed apparatus 30, the top plate 33 may be movable with respect to the support frame 34, or the top plate 33 and the support frame 34 may be movable together with respect to the base 31. good.

The console device 40 has a memory 41 , a display 42 , an input interface 43 and a processing circuit 44 . Although the console device 40 is described as being separate from the imaging device 10 , the imaging device 10 may include the console device 40 or a part of each component of the console device 40 . The console device 40 corresponds to, for example, a medical image processing device.

In the following explanation, the console device 40 is assumed to execute a plurality of functions with a single console, but the plurality of functions may be executed by separate consoles. For example, the functions of the processing circuit 44 such as the image generation function 444, which will be described later, may be distributed and installed in different console devices.

The memory 41 is a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or an integrated circuit that stores various information. Moreover, the memory 41 may be a portable storage medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), a flash memory, or the like, in addition to an HDD, SSD, or the like. The memory 41 may be a driving device that reads and writes various information with semiconductor memory devices such as flash memory and RAM (Random Access Memory). Moreover, the storage area of the memory 41 may be in the X-ray diagnostic apparatus 1 or in an external storage device connected via a network.

The memory 41 stores, for example, an X-ray image, a program executed by the processing circuit 44, a trained model used for processing by the processing circuit 44, various data used for processing by the processing circuit 44, and the like. The memory 41 also stores the imaging conditions attached to the X-ray image data. The memory 41 is an example of a storage unit.

Imaging conditions include, for example, X-ray conditions (tube current, tube voltage, duration of X-ray exposure, etc.), detector spatial resolution, imaging target region, type of imaging target, date of imaging, and subject information. and at least one of The subject information corresponds to patient information including, for example, at least one of the subject's sex, subject's race, subject's date of birth, and subject's age information. The age information of the subject is, for example, the age of the subject at the time of imaging.

Here, the age is defined as a numerical value indicating the elapsed time from the subject's birth date to the time of imaging. The age may be, for example, a numerical value in years such as 0 years old, 1 year old, 10 years old, 80 years old, etc.; , monthly numbers, and yearly numbers.

The memory 41 stores a learned prediction model M1 and a learned judgment model M2. Each of the trained prediction model M1 and the trained determination model M2 is a trained machine learning model, and is a parameterized synthetic function obtained by synthesizing a plurality of functions. A parameterized composite function is defined by a combination of multiple adjustable functions and parameters.

Each of the trained prediction model M1 and the trained judgment model M2 is described in, for example, Christopher M. Bishop, Pattern recognition and machine learning, (USA), 1st Ed., Springer, 2006, pp. 225-290. Each of the trained prediction model M1 and the trained judgment model M2 may be any parameterized composite function that satisfies the above requirements.

In the following description, a trained machine learning model is a multi-layered network model in which each parameter has been learned so as to output corresponding output data based on input data. Note that the learned machine learning model may be configured such that the program is directly incorporated in the circuit of the processor instead of storing the program such as the learned model in the memory 41 . That is, the trained machine learning model may be preset in an ASIC (Application Specific Integrated Circuit) in the processing circuit 44 or a programmable logic device. In other words, the trained model may be built in an ASIC or programmable logic device.

Also, in the following description, the multi-layered network model is assumed to be a deep neural network (DNN) that imitates neural circuits in the brain of a living organism. Note that the multilayer network model is not limited to the DNN, and for example, a convolution neural network (CNN) may be used as the multilayer network model.

FIG. 2 is a diagram illustrating an example of inputs and outputs to the trained prediction model M1. As shown in FIG. 2, the trained prediction model M1 receives a past X-ray image, past age, and current age, and outputs a predicted X-ray image.

A past X-ray image is an X-ray image obtained by imaging a region of a subject to be imaged in the past. The past X-ray image corresponds to first medical data. A subject for past X-ray images may be referred to as a first subject.

The past age is the subject's age at the time when the past X-ray image was taken. A previous age may be referred to as a first age.

The current age is different from the past age. The current age is the age of the subject at present. The current age may be referred to as a second age.

A predicted X-ray image is an X-ray image obtained by reflecting changes over time in a region to be imaged from the past age to the present age on a past X-ray image. The subject for the predicted X-ray image is the same as the subject for the previous X-ray image, that is, the first subject. A predicted X-ray image corresponds to predicted data.

That is, the learned prediction model M1 receives the first medical data, the first age, and the second age, and outputs prediction data.

FIG. 3 is a diagram showing an example of inputs and outputs to the trained determination model M2. As shown in FIG. 3, the learned determination model M2 receives the current X-ray image and the predicted X-ray image, and determines whether the subject regarding the current X-ray image and the subject regarding the predicted X-ray image are the same. output the judgment result.

A current X-ray image is an X-ray image acquired by currently imaging the same imaging target region as that of the previous X-ray image. The current X-ray image is, for example, medical data taken at a medical institution different from the medical institution at which past X-ray images were taken, among a plurality of medical institutions sharing medical data. The current X-ray image corresponds to the second medical data. Hereinafter, the subject with respect to the current X-ray image will be referred to as the second subject.

That is, the learned determination model M2 receives the second medical data and the prediction data, and outputs a determination result as to whether or not the second subject and the first subject are the same. Also, the first medical data is medical data acquired before the second medical data.

Returning to FIG. 1, the display 42 displays various information. For example, the display 42 outputs a medical image (X-ray image) generated by the processing circuit 44, a GUI (Graphical User Interface) for receiving various operations from the operator, and the like. For example, the display 42 is a liquid crystal display or a CRT (Cathode Ray Tube) display. Also, the display 42 is an example of a display unit. Also, the display 42 may be provided in the imaging device 10 . The display 42 may be of a desktop type, or may be configured by a tablet terminal or the like capable of wireless communication with the main body of the console device 40 .

The input interface 43 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs the electrical signals to the processing circuit 44 . For example, the input interface 43 receives subject information, imaging conditions, input of various command signals, and the like from the operator. For example, the input interface 43 includes a trackball, a mouse, a keyboard, a trackball, a switch, a button, a joystick, a trackball, a switch, a button, a joystick, and an input operation by touching the operation surface for instructing movement of the C-arm 14 and setting a region of interest (ROI). and a touch panel display in which the display screen and the touch pad are integrated. Also, the input interface 43 is an example of an input unit. Also, the input interface 43 may be provided in the imaging device 10 . Also, the input interface 43 may be composed of a tablet terminal or the like capable of wireless communication with the main body of the console device 40 . It should be noted that the input interface 43 is not limited to having physical operation components such as a mouse and keyboard. For example, the input interface 43 includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs the electrical signal to the processing circuit 44. .

The processing circuit 44 controls the operation of the X-ray diagnostic apparatus 1 as a whole. The processing circuit 44 calls and executes programs in the memory 41 to perform system control function 441 , drive control function 442 , X-ray control function 443 , image generation function 444 , display control function 445 , prediction function 446 and determination function 447 . is a processor that executes

In FIG. 1, a single processing circuit 44 realizes a system control function 441, a drive control function 442, an X-ray control function 443, an image generation function 444, a display control function 445, a prediction function 446, and a determination function 447. However, a processing circuit may be configured by combining a plurality of independent processors, and each function may be implemented by each processor executing a program. The system control function 441, the drive control function 442, the X-ray control function 443, the image generation function 444, the display control function 445, the prediction function 446, and the determination function 447 are respectively a system control circuit, a drive control circuit, and an X-ray control circuit. , an image processing circuit, a display control circuit, a prediction circuit and a determination circuit, or may be implemented as separate hardware circuits.

The term "processor" used in the above description includes, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an ASIC, a programmable logic device (e.g., Simple Programmable Logic Device (SPLD) , Complex Programmable Logic Device (CPLD), Field Programmable Gate Array (FPGA), etc. Processors function by reading and executing programs stored in memory circuits. Instead of storing the program in the memory circuit, the program may be directly embedded in the circuit of the processor.In this case, the processor can read and execute the program embedded in the circuit. Note that each processor of this embodiment is not limited to being configured as a single circuit for each processor, but is configured as a single processor by combining a plurality of independent circuits, and its function is implemented by 1 may be integrated into a single processor to implement its functions.

The processing circuit 44 controls each of the plurality of components in the X-ray diagnostic apparatus 1 by the system control function 441 based on the input operation received from the operator via the input interface 43 . For example, processing circuitry 44 controls various components in imaging device 10 according to imaging conditions. The processing circuit 44 that implements the system control function 441 is an example of a system control section.

The processing circuit 44 uses the drive control function 442 to control the C-arm drive device 142 and the bed drive device 32 based on information on driving the C-arm 14 and the tabletop 33 input from the input interface 43, for example. The processing circuit 44 that implements the drive control function 442 is an example of a drive control section.

The processing circuit 44 uses the X-ray control function 443 to read, for example, information from the system control function 441 and controls X-ray irradiation conditions such as tube current, tube voltage, and irradiation time in the high voltage generator 11 . The processing circuit 44 that implements the X-ray control function 443 is an example of an X-ray control section.

The processing circuit 44 uses the image generation function 444 to generate an X-ray image based on the data output from the X-ray detector 13, for example. Note that the processing circuit 44 may perform various types of synthesizing processing, subtraction processing, and the like on the generated X-ray image. An X-ray image is an example of medical data. The processing circuit 44 that implements the image generation function 444 is an example of an image generator.

The processing circuit 44 uses the display control function 445 to read, for example, a signal from the system control function 441 , obtain a desired X-ray image from the memory 41 , and display it on the display 42 . The processing circuit 44 that implements the display control function 445 is an example of a display control unit.

The processing circuit 44 uses the prediction function 446 to input the past X-ray image, past age, and current age to the learned prediction model M1, and causes the learned prediction model M1 to output a predicted X-ray image. The processing circuitry 44 that implements the prediction function 446 is an example of a prediction section.

The processing circuit 44 uses the determination function 447 to input the current X-ray image and the predicted X-ray image to the learned determination model M2, and from the learned determination model M2, the first subject and the current X-ray image related to the predicted X-ray image. A determination result as to whether or not the image is the same as that of the second object is output. The processing circuit 44 that implements the determination function 447 is an example of a determination unit. Processing circuitry 44 associates the current x-ray image with the previous x-ray image if the first subject and the second subject are the same, via decision function 447 . Specifically, the processing circuit 44 stores the current X-ray image in the memory 41 as an X-ray image of the same subject as the previous X-ray image.

The processing circuitry 44 warns the operator using the determination function 447 when the first and second subjects are not the same. Specifically, the processing circuit 44 causes the display 42 to display a warning display indicating that the first subject and the second subject are different.

Next, the operation of object identification processing executed by the X-ray diagnostic apparatus 1 will be described. The object identification processing is to generate a predicted X-ray image based on the imaging conditions and the past image, generate a current X-ray image by performing X-ray imaging, and identify the first object and the current X-ray image related to the predicted X-ray image. This is a process of determining whether or not the line image is the same as that of the second subject, and if the line image is the same, stores the current image and the past image in association with each other.

Note that the processing procedure in the subject identification processing described below is merely an example, and each processing can be changed as appropriate as possible. Further, in the processing procedures described below, steps can be omitted, replaced, and added as appropriate according to the embodiment.

FIG. 4 is a flow chart showing an example of the procedure of subject identification processing according to this embodiment. The processing circuit 44 starts the object identification process, for example, when an operation for starting imaging of the object is input through the input interface 43 .

(Subject identification processing)
(Step S101)
Processing circuitry 44 performs prediction function 446 . The processing circuit 44 reads the past X-ray image and the past age from the subject information stored in the memory 41 by the prediction function 446 . Also, the processing circuit 44 reads out the current age based on the photographing information stored in the memory 41 . The processing circuit 44 inputs the past X-ray image, past age, and current age to the learned prediction model M1, and causes the learned prediction model M1 to generate a predicted X-ray image. Processing circuit 44 stores the generated predicted X-ray image in memory 41 .

(Step S102)
The processing circuit 44 executes a system control function 441, a drive control function 442, an X-ray control function 443, and an image generation function 444 to perform X-ray imaging and generate a current X-ray image. The processing circuit 44 stores the current X-ray image generated by the image generation function 444 in the memory 41 . At this time, the processing circuit 44 may cause the display 42 to display the current X-ray image through the display control function 445 .

(Step S103)
Processing circuitry 44 performs decision function 447 . The processing circuit 44 uses the determination function 447 to read from the memory 41 the predicted X-ray image generated in the process of step S101 and the current X-ray image generated in the process of step S102. Then, the processing circuit 44 inputs the predicted X-ray image and the current X-ray image to the learned judgment model M2. The processing circuit 44 outputs a determination result indicating whether or not the first subject and the second subject are the same from the learned determination model M2 to which the predicted X-ray image and the current X-ray image are input. . The processing circuit 44 stores the output determination result in the memory 41 .

As mentioned above, the first object is the same object as for the previous X-ray image. Therefore, the processing circuit 44 determines whether or not the first subject regarding the past X-ray image and the second subject regarding the current X-ray image are the same based on the determination result by the determination function 447 .

(Step S104)
When it is determined that the first subject and the second subject are the same in the process of step S103, that is, the first subject and the second subject are different in the determination result output from the learned determination model M2. If they are the same (S104-Yes), the process of step S105 is executed. Further, when it is determined that the first subject and the second subject are not the same, that is, when the determination result output from the trained determination model M2 differs between the first subject and the second subject (S104 -No), the process of step S106 is executed.

(Step S105)
Processing circuitry 44 associates the current x-ray image with the previous x-ray image through decision function 447 . For example, the processing circuitry 44 sets the second subject regarding the current X-ray image as the first subject regarding the past X-ray image in the patient information. The processing circuit 44 stores the current X-ray image in the memory 41 as an X-ray image of the same subject as the past X-ray image, and terminates the subject identification process.

(Step S106)
The processing circuitry 44 notifies the operator that the second subject regarding the current X-ray image is different from the first subject regarding the past X-ray image by the determination function 447 . For example, the processing circuit 44 causes the display 42 to display a warning display indicating that the second subject regarding the current X-ray image is different from the first subject regarding the past X-ray image, and terminates the subject identification process. .

Note that the processing related to the prediction function 446 may be executed before the determination function 447, and may be executed after the processing of step S103, for example.

The effect of the X-ray diagnostic apparatus 1 equipped with the medical image processing apparatus according to this embodiment will be described below.

The X-ray diagnostic apparatus 1 of the present embodiment receives first medical data obtained with respect to an imaging target region of a first subject at a first age, the first age, and a second age different from the first age. inputting the first medical data, the first age, and the second age to a trained prediction model that outputs prediction data in which changes over time in the imaging target region from the first age to the second age are reflected in the first medical data; Outputting prediction data to a learned prediction model, receiving second medical data and prediction data acquired regarding a region to be imaged at a second age, and determining that a second subject and a first subject regarding the second medical data are the same The second medical data and the predictive data can be input to a trained determination model that outputs a determination result as to whether or not, and the learned determination model can output the determination result. Further, in the X-ray diagnostic apparatus 1 of the present embodiment, the first medical data is medical data acquired before the second medical data.

That is, with the above configuration and operation, according to the X-ray diagnostic apparatus 1 of the present embodiment, an X-ray image in which the change over time of the imaging target region from the time of imaging the past X-ray image to the present is reflected in the past X-ray image. A line image can be used to identify a subject. By identifying a subject using an X-ray image that reflects changes over time of the imaging target site, the accuracy of identifying the subject can be improved without being affected by changes over time. By improving the accuracy of subject identification, for example, a current X-ray image acquired by X-ray imaging using the X-ray diagnostic apparatus 1 and a past X-ray image acquired from another medical institution can be used for the same patient. as an X-ray image taken, even if different patient IDs are assigned to the same patient for each medical institution, patient mix-up can be prevented with high accuracy.

Further, in this embodiment, the processing circuit 44 acquires the past age from the imaging information on the past X-ray image, and acquires the current age from the imaging information on the current X-ray image. Then, the processing circuit 44 can generate a predicted X-ray image corresponding to the acquired past age and current age by inputting the past age and current age into the learned prediction model M1.

For this reason, according to the X-ray diagnostic apparatus 1 of the present embodiment, regardless of the past age, that is, regardless of when the past X-ray image was taken, the imaging target from the past age to the present age A predicted X-ray image can be generated by reflecting changes over time of a part on a past X-ray image.

Further, in the present embodiment, the processing circuit 44 uses the learned determination model M2, which is a learned machine learning model, to perform the process of identifying the subject. Therefore, according to the X-ray diagnostic apparatus 1 of the present embodiment, it is possible to speed up the process of identifying the subject compared to the case of identifying the subject using landmarks or the like.

(First modification)
A first modification of this embodiment will be described with reference to FIGS. 5 to 7. FIG. This modification is obtained by modifying the configuration of the first embodiment as follows. FIG. 5 is a diagram showing a configuration example of an X-ray diagnostic apparatus 1 according to this modification. The difference between FIG. 5 and FIG. 1 is that the memory 41 stores the learned non-lesion prediction model M3, the learned lesion prediction model M4, and the learned judgment model M5 instead of the learned prediction model M1 and the learned judgment model M2. However, instead of the prediction function 446 and the determination function 447, the processing circuit 44 further has an extraction function 446A, a non-lesion prediction function 446B, a lesion prediction function 446C, a synthesis function 446D and a determination function 447A.

The memory 41 stores a learned non-lesion prediction model M3, a learned lesion prediction model M4, and a learned determination model M5. Each of the learned non-lesion prediction model M3, the learned lesion prediction model M4, and the learned judgment model M5 is a learned machine learning model, and a parameterized synthetic function obtained by synthesizing a plurality of functions.

FIG. 6 is a diagram illustrating an example of inputs and outputs in the learned non-lesion prediction model M3, the learned lesion prediction model M4, and the learned determination model M5.

The learned non-lesion prediction model M3 receives the non-lesion image, the past age, and the current age, and outputs the non-lesion prediction image in which the change over time of the non-lesion region from the past age to the current age is reflected in the non-lesion image. .

A non-lesion image is an X-ray image generated by extracting a non-lesion region (hereinafter referred to as a non-lesion region) from a past X-ray image. That is, the non-lesion image is an X-ray image composed of a non-lesion region among the X-ray images obtained by imaging the first subject in the past. A non-lesion image corresponds to non-lesion data.

The non-lesion predicted image is an X-ray image in which the temporal change of the non-lesion region from the past age to the present age is reflected in the non-lesion predicted image. The subject for non-lesion predictive images is the same as the subject for non-lesion images. A non-lesion predicted image corresponds to non-lesion predicted data.

The learned lesion prediction model M4 accepts lesion images, past age, and current age, and performs lesion prediction by reflecting temporal changes in the lesion area from the past age to the present age (hereinafter referred to as the lesion area) in the lesion image. Output the image.

A lesion image is an X-ray image generated by extracting a lesion area from a past X-ray image. That is, the lesion image is an X-ray image composed of a lesion area among the X-ray images acquired by imaging the subject in the past. A lesion image corresponds to lesion data.

A lesion prediction image is an X-ray image in which the change over time of the lesion area from the past age to the present age is reflected in the lesion prediction image. A lesion prediction image corresponds to lesion prediction data.

The subject for non-lesion images, the subject for non-lesion predictive images, the subject for lesion images, and the subject for lesion predictive images are the same as the subject for past X-ray images, that is, the first subject.

The learned determination model M5 receives the current X-ray image and the composite image, and outputs a determination result as to whether or not the subject regarding the current X-ray image and the subject regarding the composite image are the same. The composite image is an X-ray image in which both the temporal change of the non-lesion area from the past age to the present age and the temporal change of the lesion area from the past age to the present age are reflected in the past X-ray image. A composite image is generated by aligning and synthesizing the non-lesion predictive image and the lesion predictive image. A composite image corresponds to composite data. The subject regarding the composite image is the same as the subject regarding the past X-ray image, the subject regarding the non-lesion predictive image, and the subject regarding the lesion predictive image. That is, the subject regarding the composite image corresponds to the first subject.

Returning to FIG. 5, the processing circuit 44 calls and executes the programs in the memory 41 to perform system control function 441, drive control function 442, X-ray control function 443, image generation function 444, display control function 445, extraction function. 446A, a non-lesion prediction function 446B, a lesion prediction function 446C, a synthesis function 446D, and a decision function 447.

The processing circuit 44 extracts a lesion image indicating the lesion area from the past X-ray image by applying, for example, segmentation processing relating to the lesion area to the past X-ray image using the extraction function 446A. The processing circuit 44 also extracts a non-lesion image by subtracting the lesion image from the past X-ray image. Note that the processing circuitry 44 may extract a non-lesion image from the past X-ray image by applying segmentation processing for the non-lesion region to the past X-ray image. The processing circuit 44 that implements the extraction function 446A is an example of an extraction unit.

The processing circuit 44 uses the non-lesion prediction function 446B to input the non-lesion image, past age, and current age to the learned non-lesion prediction model M3, and causes the learned non-lesion prediction model M3 to output the non-lesion prediction image. The processing circuit 44 that implements the non-lesion prediction function 446B is an example of a non-lesion prediction section.

The processing circuit 44 uses the lesion prediction function 446C to input the lesion image, past age, and current age to the learned lesion prediction model M4, and causes the learned lesion prediction model M4 to output a lesion prediction image. The processing circuit 44 that implements the lesion prediction function 446C is an example of a lesion prediction section.

The processing circuitry 44 performs registration of the non-lesion predictive image and the lesion predictive image with a synthesizing function 446D. For example, processing circuitry 44 performs the registration using anatomical landmarks in the non-lesion prediction image and the lesion prediction image. Processing circuitry 44 then generates a composite image by combining the registered non-lesion predictive image and the lesion predictive image. Note that the processing circuit 44 may generate a composite image by superimposing the lesion prediction image on the non-lesion prediction image after registration. The processing circuitry 44 that implements the synthesizer function 446D is an example of a synthesizer.

The processing circuit 44 uses the determination function 447A to input the current X-ray image and the synthesized image to the learned determination model M5, and the learned determination model M5 receives the second subject regarding the current X-ray image and the first subject regarding the synthesized image. A determination result as to whether or not the sample is the same is output. The processing circuit 44 that implements the determination function 447A is an example of a determination unit. Processing circuitry 44 associates the current X-ray image with the previous X-ray image if the first subject and the second subject are the same, via decision function 447A. The processing circuit 44 uses the determination function 447A to warn the operator when the first subject and the second subject related to the composite image are not the same in the determination result output from the learned determination model M5.

Other configurations are similar to those of the first embodiment.

Next, the operation of object identification processing executed by the X-ray diagnostic apparatus 1 will be described. FIG. 7 is a flow chart showing an example of the procedure of subject identification processing according to this embodiment. Since the process of step S115 in FIG. 7 is the same as the process of step S102 in the first embodiment, description thereof is omitted.

(Subject identification processing)
(Step S111)
Processing circuitry 44 performs extraction function 446A. The processing circuit 44 reads the past X-ray image from the memory 41 by the extraction function 446A. A processing circuit 44 extracts a lesion image representing a lesion area in the past X-ray image from the past X-ray image. Then, the processing circuit 44 extracts a non-lesion image indicating a non-lesion area by subtracting the lesion image from the past X-ray image. The processing circuit 44 stores the extracted non-lesion image and lesion image in the memory 41 .

(Step S112)
Processing circuitry 44 performs non-lesion prediction function 446B. The processing circuit 44 reads the non-lesion image and the past age from the subject information stored in the memory 41 by the non-lesion prediction function 446B. Also, the processing circuit 44 reads out the current age based on the photographing information stored in the memory 41 . The processing circuit 44 uses the non-lesion prediction function 446B to input the non-lesion image, past age, and current age to the learned non-lesion prediction model M3, and causes the learned non-lesion prediction model M3 to generate a non-lesion prediction image. The processing circuit 44 stores the generated non-lesion predicted image in the memory 41 .

(Step S113)
Processing circuitry 44 performs lesion prediction function 446C. The processing circuit 44 reads the lesion image and past age from the subject information stored in the memory 41 by the lesion prediction function 446C. Also, the processing circuit 44 reads out the current age based on the photographing information stored in the memory 41 . The processing circuit 44 uses the lesion prediction function 446C to input the lesion image, past age, and current age to the learned lesion prediction model M4, and causes the learned lesion prediction model M4 to generate a lesion prediction image. The processing circuit 44 stores the generated lesion prediction image in the memory 41 .

(Step S114)
Processing circuitry 44 performs compositing function 446D. The processing circuit 44 reads the non-lesion predictive image and the lesion predictive image stored in the memory 41 by the synthesizing function 446D. The processing circuit 44 generates a synthesized image by synthesizing the non-lesion predicted image and the lesion predicted image. For example, the processing circuit 44 synthesizes a lesion image in the non-lesion predictive image at the same position as the extraction position of the lesion area in the past X-ray image. The processing circuit 44 stores the generated synthetic image in the memory 41 .

(Step S116)
Processing circuitry 44 performs decision function 447A. The processing circuit 44 reads from the memory 41 the composite image generated in the process of step S114 and the current X-ray image generated in the process of step S115 by the determination function 447A. Then, the processing circuit 44 inputs the composite image and the current X-ray image to the learned determination model M5. The processing circuit 44 uses the determination function 447A to determine whether the first subject related to the combined image and the second subject related to the current X-ray image are the same from the trained determination model M5 to which the combined image and the current X-ray image are input. output a determination result indicating whether or not there is The processing circuit 44 stores the output determination result in the memory 41 .

(Step S117)
If it is determined in the process of step S116 that the first subject regarding the composite image and the second subject regarding the current X-ray image are the same, that is, if the determination result output from the learned determination model M5 is the first subject If the specimen and the second specimen are the same (S117-Yes), the process of step S118 is executed. Further, when it is determined that the first subject regarding the synthesized image and the second subject regarding the current X-ray image are not the same, that is, in the determination result output from the learned determination model M5, the first subject and the second subject If the two subjects are different (S117-No), the process of step S119 is executed.

(Step S118)
Processing circuitry 44 associates the current X-ray image with the previous X-ray image through decision function 447A. Since other processes are the same as those in step S105, description thereof is omitted.

(Step S119)
The processing circuit 44 notifies the operator that the subject regarding the current X-ray image is different from the subject regarding the previous X-ray image by the determination function 447A. Since other processes are the same as those in step S106, description thereof is omitted.

The effect of the X-ray diagnostic apparatus 1 equipped with the medical image processing apparatus according to this modified example will be described below.

The X-ray diagnostic apparatus 1 of this modification extracts lesion data indicating a lesion area in the first medical data from the first medical data acquired at the first age with respect to the imaging target region of the first subject. By subtracting the lesion data from the data, non-lesion data indicating a non-lesion area is extracted, the non-lesion data is accepted, and the change over time of the non-lesion area from the first age to the second age different from the first age is non-lesion data. Non-lesion data is input to the learned non-lesion prediction model M3 that outputs non-lesion prediction data reflected in the lesion data, the learned non-lesion prediction model M3 is caused to output the non-lesion prediction data, the lesion data is received, Lesion data is input to a learned lesion prediction model M4 that outputs lesion prediction data in which changes in the lesion area over time from the 1st age to the 2nd age are reflected in the lesion data, and the lesion prediction data is input to the learned lesion prediction model M4. Synthesis data is generated by synthesizing the non-lesion prediction data and the lesion prediction data, receiving the second medical data and the synthesis data acquired regarding the imaging target site at the second age, and the second medical data The second medical data and the synthetic data are input to the learned determination model M5 that outputs the determination result as to whether the second subject and the first subject are the same, and the determination result is input to the learned determination model M5. can be output. Further, in the X-ray diagnostic apparatus 1 of this modified example as well, the first medical data is medical data acquired before the second medical data.

That is, with the above configuration and operation, according to the X-ray diagnostic apparatus 1 of this modified example, a non-lesion region from the past age to the present age using a trained model specialized in predicting changes in the lesion region over time. It is possible to generate an X-ray image that reflects both the temporal change of the lesion area and the temporal change of the lesion area from the past age to the present age. Then, the subject can be identified using an X-ray image that reflects both the temporal change of the non-lesion area and the temporal change of the lesion area. Thereby, even if the subject has a lesion, the accuracy of identification of the subject can be improved.

Note that the prediction function 446 of the first embodiment may be used instead of the non-lesion prediction function 446B. In this case, the memory 41 stores the learned prediction model M1 of the first embodiment instead of the learned non-lesion prediction model M3. At this time, instead of the synthesizing function 446D, the processing circuit 44 generates a synthesized image in which the predicted X-ray image output from the learned prediction model M1 is superimposed on the predicted lesion image output from the learned lesion prediction model M4. It has a superimposition function. The subject regarding the superimposed image is the same as the subject regarding the past X-ray image and the subject regarding the lesion prediction image. The processing circuitry 44 also uses a determination function 447 to determine whether or not the object associated with the superimposed image and the object associated with the current image are the same.

(Second modification)
A second modification of this embodiment will be described with reference to FIGS. 8 to 10. FIG. This modification is obtained by modifying the configuration of the first embodiment as follows. FIG. 8 is a diagram showing a configuration example of an X-ray diagnostic apparatus 1 according to this modification. The difference between FIG. 8 and FIG. 1 is that processing circuitry 44 further includes model specific functionality 448 .

FIG. 9 is a diagram for explaining a learned prediction model and a learned determination model stored in the memory 41 of this modification. As shown in FIG. 9, the memory 41 stores a plurality of learned prediction models (M11 to M13, etc.) corresponding to different imaging target parts. Similarly, the memory 41 stores a plurality of learned determination models (M21 to M23, etc.) corresponding to different imaging target regions.

The trained prediction model M11 for the head is a trained prediction model specialized for predicting changes over time of the imaging target region when the imaging target region is the head. The trained prediction model M12 for the chest is a trained prediction model that specializes in predicting temporal changes in the imaging target region when the imaging target region is the chest. The abdominal trained prediction model M13 is a trained prediction model that specializes in predicting changes over time of an imaging target region when the imaging target region is the abdomen.

The trained determination model M21 for the head is a trained determination model that specializes in predicting changes over time of the imaging target region when the imaging target region is the head. The trained determination model M22 for the chest is a trained determination model that specializes in predicting temporal changes in the imaging target region when the imaging target region is the chest. The abdominal trained determination model M23 is a trained determination model specialized for predicting changes over time of the imaging target region when the imaging target region is the abdomen.

Returning to FIG. 8, the processing circuit 44 calls and executes the programs in the memory 41 to perform a system control function 441, a drive control function 442, an X-ray control function 443, an image generation function 444, a display control function 445, and a prediction function. 446, decision function 447, and model identification function 448.

The processing circuit 44 uses the model specifying function 448 to select, from among a plurality of trained prediction models (M11 to M13, etc.) based on the imaging target region related to the past X-ray image, the imaging target region of the past X-ray image. Identify trained prediction models. Further, the processing circuit 44 uses the model specifying function 448 to select the imaging target of the past X-ray image from among a plurality of learned determination models (M21 to M23, etc.) based on the information about the imaging target region of the past X-ray image. Identify a trained judgment model that corresponds to the body part. The imaging target region related to the past X-ray image is included, for example, in additional information attached to the first medical data. Processing circuitry 44 that implements model identification function 448 is an example of a model identification unit.

Other configurations are similar to those of the first embodiment.

Next, the operation of object identification processing executed by the X-ray diagnostic apparatus 1 will be described. FIG. 10 is a flowchart showing an example of the procedure of object identification processing according to this embodiment. The processes in steps S123 and S125 to S127 in FIG. 10 are the same as the processes in steps S102 and S104 to S106 in the first embodiment, respectively, and thus description thereof is omitted.

(Subject identification processing)
(Step S121)
The processing circuit 44 acquires the imaging target part from the imaging conditions stored in the memory 41 by the model specifying function 448 . The processing circuit 44 uses the model specifying function 448 to specify a learned prediction model corresponding to the acquired imaging target region from among a plurality of learned prediction models (M11 to M13, etc.) stored in the memory 41. In addition, the processing circuit 44 uses the model specifying function 448 to specify a learned determination model corresponding to the acquired imaging target region from among a plurality of learned determination models (M21 to M23, etc.) stored in the memory 41. do.

(Step S122)
The processing circuit 44 uses the prediction function 446 to read out the learned prediction model identified by the process of step S121. The processing circuit 44 inputs the past X-ray image, the past age, and the current age to the learned prediction model specified by the process of step S121, and inputs the predicted X-ray image to the learned prediction model M1 specified by the process of S121. generate. Processing circuit 44 stores the generated predicted X-ray image in memory 41 .

(Step S124)
The processing circuit 44 uses the determination function 447 to input the predicted X-ray image and the current X-ray image to the learned determination model specified by the process of step S121, and apply the learned determination model specified by the process of step S121. A determination result indicating whether or not the first subject and the second subject are the same is output. The processing circuit 44 stores the output determination result in the memory 41 .

For example, when the imaging target region is the “chest”, in the process of step S121, the “learned prediction model for chest M12” is specified as the trained prediction model corresponding to the imaging target region, and the learning corresponding to the imaging target region is identified. A “learned determination model for chest M22” is specified as a completed determination model. Then, in the process of step S122, the trained prediction model M12 for the chest is used, and in the process of step S124, the learned determination model M22 for the chest is used.

The effect of the X-ray diagnostic apparatus 1 equipped with the medical image processing apparatus according to this modified example will be described below.

The X-ray diagnostic apparatus 1 of this modification identifies a learned prediction model corresponding to additional information about the first medical data from among a plurality of learned prediction models (M11 to M13, etc.), and selects a plurality of learned determination models. (M21 to M23, etc.) to identify a learned judgment model corresponding to the incidental information, the prediction unit inputs first medical data to the identified trained prediction model, and identifies the learned prediction The prediction data can be output to the model, the determination unit can input the second medical data to the identified learned determination model, and can cause the identified learned determination model to output the determination result. Further, in the X-ray diagnostic apparatus 1 of this modified example as well, the first medical data is medical data acquired before the second medical data.

With the above configuration and operation, according to the X-ray diagnostic apparatus 1 of this modified example, in addition to the effects of the first embodiment, a predicted X-ray image is generated using a trained model specialized for an imaging target site. can do.

For a plurality of items included in imaging conditions different from the imaging target site, a plurality of learned prediction models corresponding to the plurality of items may be prepared. At this time, the processing circuitry 44 uses the model identification function 448 to identify a learned prediction model corresponding to the items in the imaging conditions of the past X-ray images. For example, the memory 41 stores a plurality of trained prediction models corresponding to each of items such as the sex of the subject and the race of the subject. As for the learned judgment models, the memory 41 similarly stores a plurality of learned judgment models corresponding to each of these items. At this time, the processing circuit 44 uses the model specifying function 448 to specify a learned determination model corresponding to the items in the imaging conditions of the past X-ray images.

(Third modification)
A third modification of the present embodiment will be described with reference to FIGS. 11 and 12. FIG. This modification is obtained by modifying the configuration of the first embodiment as follows. In this modification, the memory 41 stores a learned prediction model M6 and a learned determination model M7.

FIG. 11 is a diagram illustrating an example of inputs and outputs to the trained prediction model M6. As shown in FIG. 11, the learned prediction model M6 receives the current X-ray image, the current age, and the past age, and makes a prediction X that reflects changes over time in the imaging target region from the current age to the past age in the current image. Output a line image.

A predicted X-ray image is an X-ray image in which the change over time of the imaging target region from the current age to the past age is reflected on the current X-ray image. A predicted X-ray image corresponds to predicted data.

FIG. 12 is a diagram showing an example of inputs and outputs to the trained determination model M7. As shown in FIG. 12, the learned determination model M7 receives the past X-ray image and the predicted X-ray image, and determines whether the subject regarding the past X-ray image and the subject regarding the predicted X-ray image are the same. output the judgment result.

In this modified example, the current X-ray image corresponds to the first medical data, and the past X-ray image corresponds to the second medical data. Also, the subject regarding the current X-ray image corresponds to the first subject, and the subject regarding the previous X-ray image corresponds to the second subject.

The processing circuit 44 uses the prediction function 446 to input the current X-ray image, current age, and past age to the learned prediction model M6, and causes the learned prediction model M6 to output a predicted X-ray image.

The processing circuit 44 inputs the past X-ray image and the predicted X-ray image to the learned judgment model M7 by the judgment function 447, and inputs the second subject and the predicted X-ray image related to the past X-ray image to the learned judgment model M7. is the same as the first subject with respect to is output.

The processing circuit 44 causes the determination function 447 to associate the current X-ray image with the past X-ray image when the first subject and the second subject are the same in the determination result output from the learned determination model M7. . The processing circuit 44 uses the determination function 447 to warn the operator when the determination result output from the learned determination model M2 does not match the first subject and the second subject.

Other configurations are similar to those of the first embodiment.

The effect of the X-ray diagnostic apparatus 1 equipped with the medical image processing apparatus according to this modified example will be described below.

In this modified example, the second medical data is medical data acquired before the first medical data.

According to the X-ray diagnostic apparatus 1 of this modified example, an X-ray image in which a change over time of an imaging target region from the time of imaging of the past X-ray image to the present time is reflected in the current X-ray image, and a past X-ray image can be used to identify the subject. Therefore, in this modified example, as in the first embodiment, the subject is identified using an X-ray image that reflects the time-dependent change of the imaging target site. The accuracy of identification of the subject can be improved without being

A desired reference age different from the past age and the current age is set, and the first predicted X-ray image in which the temporal change of the imaging target site from the past age to the reference age is reflected in the past X-ray image, and the current age Subject identification processing may be performed using a second predicted X-ray image in which the change over time of the imaging target site from age to reference age is reflected in the current X-ray image. In this case, for example, the past age corresponds to the first age, the reference age corresponds to the second age, the first predicted X-ray image corresponds to the predicted data, and the second predicted X-ray image corresponds to the second medical data. Equivalent to. Since other effects are the same as those of the first embodiment, description thereof is omitted.

(Fourth modification)
A fourth modification of this embodiment will be described. This modification is obtained by modifying the configuration of the first embodiment as follows.

In the prediction function 446, the processing circuit 44, when a plurality of X-ray images corresponding to the subject related to past X-ray images are stored in the memory 41, selects one of the plurality of X-ray images taken at the time closest to the present. The obtained X-ray image is input to the learned prediction model M1 as a past X-ray image. That is, the processing circuit 44 uses the prediction function 446 to input the most recent X-ray image to the learned prediction model M1 as a past X-ray image.

Other configurations are similar to those of the first embodiment.

In addition to the first embodiment, the X-ray diagnostic apparatus 1 of this modified example has a plurality of pieces of medical data acquired with respect to the imaging target region of the first subject at the time closest to the second medical data. Medical data can be input to the trained prediction model as first medical data.

The effect of the X-ray diagnostic apparatus 1 equipped with the medical image processing apparatus according to this modified example will be described below.

According to the X-ray diagnostic apparatus 1 of this modified example, among the plurality of X-ray images acquired in the past, the X-ray image with the smallest time-dependent change of the imaging target region with respect to the current X-ray image is designated as the past X-ray image. Used. Therefore, the magnitude of the predicted change over time can be reduced, and the accuracy of identification of the subject can be further improved.

Further, when the technical ideas of this embodiment and each modification of this embodiment are implemented in a medical image processing apparatus, the medical image processing apparatus has the components described in the console device 40, for example. In this case, the drive control function 442 and the X-ray control function 443 are not required. At this time, various data used in the prediction function 446 and the determination function 447 may be stored in the memory 41, and these various data may be taken into the medical image processing apparatus via the network. At this time, the medical image processing apparatus receives the first medical data acquired regarding the imaging target region of the first subject at the first age, the first age, and a second age different from the first age, and receives the first age. The first medical data, the first age, and the second age are input to a trained prediction model that outputs prediction data that reflects changes over time in the imaging target site from the first to the second age in the first medical data. causing the prediction model to output the prediction data, receiving the second medical data and the prediction data acquired regarding the imaging target site at the second age, and regarding the second medical data, the second subject and the first subject are the same The second medical data and the predictive data can be input to a trained determination model that outputs a determination result as to whether or not, and the learned determination model can output the determination result.

The technical idea of this embodiment and each modification of this embodiment is, for example, to install programs related to each of the display control function 445, the prediction function 446, and the determination function 447 in a computer such as a workstation, and store them on a memory. It can also be realized by expanding. At this time, the computer is installed in, for example, a medical image processing apparatus, an integrated server on the cloud, or the like. Various data used in the prediction function 446 and the determination function 447 may be stored in the memory 41 of the computer, and these various data may be taken into the computer via a network.

Specifically, the program for executing the prediction process causes a computer to accept past X-ray images, past age, and current age, and predicts X-rays in which changes over time of an imaging target site from past age to present age are reflected in past images. A predicted X-ray image is output by storing a learned prediction model M1 functioned to output a line image, and inputting a past X-ray image, past age, and current age to the learned prediction model M1. to make it happen.

Specifically, the program for executing the determination process causes the computer to receive the current X-ray image and the predicted X-ray image and determine whether the subject regarding the current X-ray image and the subject regarding the predicted X-ray image are the same. storing the learned judgment model M2 that is functioned to output the judgment result of, and outputting the judgment result by inputting the current X-ray image and the predicted X-ray image to the learned judgment model M2; Realize

(Regarding learning data)
Hereinafter, learning data used at the time of learning in each of the learned prediction model, the learned determination model, the learned non-lesion prediction model, and the learned lesion prediction model described so far will be described.

FIG. 13 shows an example of learning data (hereinafter referred to as prediction model learning data) used for learning the learned prediction model M1 according to the first embodiment. The predictive model learning data includes a plurality of learning samples. Each of the plurality of training samples includes one first X-ray image, one first age information, one second X-ray image, and one second age information. The first X-ray image, the first age information, and the second age information correspond to training data during learning of the learned prediction model M1. The second X-ray image corresponds to teacher data during learning of the learned prediction model M1.

The first X-ray image and the second X-ray image are X-ray images obtained by imaging the same subject at different times. The subject for the first X-ray image and the subject for the second X-ray image are the same.

The first age information is information indicating the age of the subject at the time when the first X-ray image was taken. The second age information is information indicating the age of the subject at the time when the second X-ray image was taken. The second age is different than the first age. The second X-ray image is an X-ray image in which changes over time of the imaging target region from the first age to the second age are reflected in the first X-ray image.

FIG. 14 shows an example of learning data (hereinafter referred to as determination model learning data) used for learning the learned determination model M2 according to the first embodiment. The decision model learning data includes a plurality of learning samples. Each of the plurality of learning samples includes one predicted X-ray image, one current X-ray image, and one determination result. The predicted X-ray image and the current X-ray image correspond to training data during learning of the learned judgment model M2. The determination information corresponds to teacher data during learning of the learned determination model M2.

A predicted X-ray image is an X-ray image in which the change over time of the imaging target region of the subject from the time of imaging of the past X-ray image to the present is reflected in the past X-ray image. The predicted X-ray image may be one actually output by the learned prediction model M1, or one generated by image processing of a past image.

The determination information is information indicating whether or not the subject regarding the predicted X-ray image and the subject regarding the current X-ray image are the same.

FIG. 15 shows an example of learning data (hereinafter referred to as non-lesion prediction model learning data) used for learning the learned non-lesion prediction model M3 according to the first modification of the first embodiment. The non-lesion prediction model learning data includes a plurality of learning samples. Each of the plurality of learning samples includes one first non-lesion image, one first age information, one second non-lesion image, and one second age information. The first non-lesion image, first age information, and second age information correspond to training data during learning of the learned non-lesion prediction model M3. The second non-lesion image corresponds to teacher data during learning of the learned non-lesion prediction model M3.

The first non-lesion image and the second non-lesion image are X-ray images obtained by removing the lesion area from the X-ray image obtained by imaging the imaging target site. The first non-lesion image and the second non-lesion image are X-ray images obtained by imaging the same subject at different times. The subject for the first non-lesion image and the subject for the second non-lesion image are the same.

The first age information is information indicating the age of the subject at the time when the first non-lesion image was captured. The second age information is information indicating the age of the subject at the time when the second non-lesion image was captured. The second age is different than the first age. The second non-lesion image is an X-ray image in which changes over time of the imaging target region excluding the lesion area from the first age to the second age are reflected in the first non-lesion image.

FIG. 16 shows an example of learning data (hereinafter referred to as non-lesion prediction model learning data) used for learning the learned lesion prediction model M4 according to the first modification of the first embodiment. The non-lesion prediction model learning data includes a plurality of learning samples. Each of the plurality of learning samples includes one first lesion image, one first age information, one second lesion image, and one second age information. The first lesion image, the first age information, and the second age information correspond to training data during learning of the learned lesion prediction model M4. The second lesion image corresponds to teacher data during learning of the learned lesion prediction model M4.

The first lesion image and the second lesion image are X-ray images generated by extracting a lesion area from an X-ray image obtained by imaging an imaging target site. The first lesion image and the second lesion image are X-ray images obtained by imaging the same subject at different times. The subject for the first lesion image and the subject for the second lesion image are the same.

The first age information is information indicating the age of the subject at the time when the first lesion image was captured. The second age information is information indicating the age of the subject at the time when the second lesion image was captured. The second age is different than the first age. The second lesion image is an X-ray image in which temporal changes in the lesion area from the first age to the second age are reflected in the first lesion image.

FIG. 17 shows an example of learning data (hereinafter referred to as determination model learning data) used for learning the learned determination model M5 according to the first modification of the first embodiment. The decision model learning data includes a plurality of learning samples. Each of the plurality of learning samples includes one synthesized image, one current X-ray image, and one judgment result. The synthesized image and the current X-ray image correspond to training data during learning of the learned judgment model M5. The determination information corresponds to teacher data during learning of the learned determination model M5.

The synthesized image may be one actually generated by the synthesizing function 446D of the first modified example of the first embodiment, or one generated by image processing.

The determination information is information indicating whether or not the subject regarding the composite image and the subject regarding the current X-ray image are the same.

An example of a method of generating a trained model used in the first embodiment and each modified example of the first embodiment will be described below. A processing circuit installed in a trained model generation device (not shown) uses a trained model generation function to generate a trained model, and trains a multi-layered network model (learning program) with learning data to generate a learned model. Generate a model. A trained model generation function corresponds to a model learning program and is executed in a processing circuit. For example, the processing circuit generates a learned model by calculating parameters (weights) in a multi-layer network model according to error backpropagation using differences (errors) between training data and teacher data. The trained model generating apparatus outputs the generated trained model to the medical image processing apparatus via various storage media, networks, and the like.

(model generation processing)
An example of a model learning device that applies a generative adversarial network (GAN) to generate a trained prediction model used in the first embodiment will be described below with reference to FIGS. 18 and 19. .

FIG. 18 is a diagram showing a configuration example of the model learning device 60. As shown in FIG. The model learning device 60 has, for example, a processing circuit and a memory (not shown). The memory stores a plurality of programs for implementing a generator 61, a discriminator 62, a discriminator training function 63, and a generator training function 64, respectively. The processing circuit executes a generator 61, a discriminator 62, a discriminator training function 63, and a generator training function 64 by reading each of the plurality of programs from memory. . Each of the generator 61 and the discriminator 62 is a trained machine learning model, and is a parameterized synthetic function obtained by synthesizing a plurality of functions.

The model learning device 60 receives input of the first medical data, the first age and the second age, and generates prediction data in which changes over time in the imaging target region from the first age to the second age are reflected in the first medical data. A learned prediction model to be output is generated by repeating a hostile training process using a discriminator 62 that determines whether prediction data is medical data obtained by actual imaging.

Each network model and each function provided in the model learning device 60 may be realized by a processing circuit having a CPU or the like, or may be realized by an ASIC or a programmable logic device (for example, SPLD, CPLD and FPGA).

Moreover, each network model and each function provided in the model learning device 60 may be a program stored in a memory, or may be configured to directly incorporate the program into the circuitry of the processor. That is, each network model and each function provided in the model learning device 60 may be preset in an ASIC in the processing circuit or a programmable logic device. In other words, each network model and each function provided in the model learning device 60 may be built in an ASIC or a programmable logic device.

Moreover, each network model and each function provided in the model learning device 60 may be incorporated in a medical image processing apparatus such as the X-ray diagnostic apparatus 1 described in the first embodiment.

Hereinafter, as an example, a method for generating the learned prediction model M1 used in the first embodiment will be described.

The generator 61 receives past X-ray images, past age and current age, and outputs a predicted X-ray image.

The discriminator 62 receives the predicted X-ray image and the current X-ray image, and determines whether the predicted X-ray image was obtained by actual imaging. Specifically, the discriminator 62 first acquires the predicted X-ray image (fake) output from the generator 61 and the current X-ray image (genuine) generated by actual imaging. Then, the classifier 62 determines which of the predicted X-ray image and the current X-ray image is data obtained by actual imaging (genuine), and which is calculated data generated by the generator 61 ( It judges (identifies) whether it is a counterfeit or not, and outputs the judgment result. For example, the discriminator 62 outputs "0" when determining that the predicted X-ray image output from the generator 61 is genuine, and outputs "1" when determining that the current X-ray image is genuine. to output

The processing circuit in the model learning device 60 alternately trains the discriminator 62 and the generator 61 by repeatedly executing the discriminator training function 63 and the generator training function 64 alternately or mutually. When the discriminator training function 63 and the generator training function 64 have been executed a specified number of times or more, the model learning device 60 terminates the training of the discriminator 62 and the generator 61 . The trained generator 61 is used, for example, as a learned prediction model in the first embodiment and the like.

The procedure of model generation processing will be described below with reference to FIG. FIG. 19 is a flowchart showing an example of the flow of processing for generating a learned prediction model M1 (hereinafter referred to as prediction model generation processing).

(Prediction model generation processing)
(Step S201)
The model learning device 60 trains the discriminator 62 by the discriminator training function 63 . The processing circuit in the model learning device 60 executes the discriminator training function 63 so that the rate at which the discriminator 62 outputs correct determination results (hereinafter referred to as the success rate of determination of the discriminator 62) increases. , to train the discriminator 62 . At this time, the model learning device 60 acquires, for example, the determination result output from the classifier 62, the imaging information related to the predicted X-ray image used in the determination, and the subject information related to the current X-ray image, It is determined whether the determination result of the discriminator 62 is correct. After that, the model learning device 60 updates each parameter of the classifier 62 so as to output a correct judgment result based on the judgment result. Thereby, the discriminator 62 is trained (learned) so that the success rate of determination increases.

(Step S202)
Model learning device 60 trains generator 61 by generator training function 64 . Processing circuitry in the model learning device 60 trains the generator 61 by executing a generator training function 64 to output a predicted X-ray image that reduces the success of the decision of the classifier 62 . At this time, the model learning device 60 uses the predicted X-ray image input to the classifier 62 when the decision result of the classifier 62 is erroneous, and the generator 61 when the generator 61 outputs this predicted X-ray image. Each parameter of the generator 61 is updated so as to output a predicted X-ray image that causes the discriminator 62 to output an erroneous judgment result by learning the past X-ray images input to the generator 61 . Thereby, the generator 61 is trained (learned) so as to output a predicted X-ray image that reduces the success rate of the determination of the discriminator 62 .

(Step S203)
The model learning device 60 determines whether or not the number of executions of the discriminator training function 63 and the generator training function 64 is equal to or greater than a specified number of times. That is, the processing circuitry determines whether each of the discriminator training function 63 and the generator training function 64 has been executed a specified number of times or more. The specified number of times is, for example, 10,000 times.

If the number of executions of the discriminator training function 63 and the generator training function 64 is equal to or greater than the specified number of times (step S203-Yes), the process proceeds to step S204. If the number of executions of the classifier training function 63 and the generator training function 64 is less than the specified number of times (step S203-No), the process returns to step S201, and the model learning device 60 performs the classifier training function 63 and the generator The processing of steps S201 and S202 is repeatedly executed until each of the training functions 64 is executed a predetermined number of times or more. As a result, the discriminator training function 63 and the generator training function 64 are alternately and repeatedly executed.

(Step S204)
When the discriminator training function 63 and the generator training function 64 have been executed a specified number of times or more, the model learning device 60 ends the training of the generator 61 and the discriminator 62 . A trained generator 61 is used as a trained model.

In this application example, the generator 61 and the discriminator 62 are repeatedly trained using the generator 61 and the discriminator 62, which are two layered network models that compete with each other. As a result, the accuracies of the generator 61 and the discriminator 62 are improved so as to compete with each other. As a result, a highly accurate generator 61 (learned prediction model) can be generated.

In this application example, the discriminator 62 is a two-channel discriminator that accepts two input images, but is not limited to this. The discriminator 62 may be a one-channel discriminator that obtains only the corrected projection data (fake) output from the generator 61 . In this case, the discriminator 62 determines whether the predicted X-ray image output from the generator 61 is authentic. The classifier 62 may also be a multi-channel classifier that acquires one predicted X-ray image (fake) and multiple current X-ray images (genuine). In this case, the discriminator 62 determines which of the three or more acquired X-ray images is a fake.

According to at least one embodiment described above, it is possible to improve the identification accuracy of a subject with respect to a plurality of pieces of medical data.

While several embodiments of the invention have been described, these embodiments have been presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, as well as the scope of the invention described in the claims and equivalents thereof.

DESCRIPTION OF SYMBOLS 1... X-ray diagnostic apparatus 10... Imaging apparatus 11... High-voltage generator 12... X-ray generator 13... X-ray detector 14... C-arm 142... C-arm drive device 30... Bed apparatus 31... Base 32... Bed drive Apparatus 33 Top plate 34 Support frame 40 Console device 41 Memory 42 Display 43 Input interface 44 Processing circuit 441 System control function 442 Drive control function 443 X-ray control function 444 Image generation function 445 Display control function 446 Prediction function 446A Extraction function 446B Non-lesion prediction function 446C Lesion prediction function 446D Synthesis function 447 Judgment function 447A Judgment function 448 Model identification function 60 Model learning device 61 Generator 62 Classifier 63 Classifier training function 64 Generator training function M1, M6 Trained prediction model M11 Head trained prediction model M12 Chest trained prediction model M13 Abdomen trained prediction models M2, M5 , M7... Learned determination model M21... Learned decision model for head M22... Learned decision model for chest M23... Learned decision model for abdomen M3... Learned non-lesion prediction model M4... Learned lesion prediction model

Claims (10)

Receiving first medical data acquired with respect to an imaging target region of a first subject at a first age, the first age, and a second age different from the first age, and ranging from the first age to the second age inputting the first medical data, the first age, and the second age to a trained prediction model that outputs prediction data in which the change over time of the imaging target site is reflected in the first medical data, and learning a prediction unit that outputs the prediction data to a prediction model;
receiving the second medical data acquired with respect to the imaging target region of the second subject at the second age and the prediction data, and determining whether or not the second subject and the first subject are the same; a determination unit that inputs the second medical data and the prediction data to a learned determination model that outputs a determination result, and causes the learned determination model to output the determination result;
A medical image processing apparatus comprising: When the determination result indicates that the second subject and the first subject are the same, the determination unit associates the first medical data with the second medical data.
The medical image processing apparatus according to claim 1. wherein the first medical data and the second medical data are X-ray images or reconstructed images;
The medical image processing apparatus according to claim 1 or 2. an extraction unit that extracts lesion data indicating a lesion area in the first medical data from the first medical data;
inputting the lesion data to a learned lesion prediction model that outputs lesion prediction data in which the change in the lesion area over time from the first age to the second age is reflected in the lesion data, and the learned lesion prediction model; a lesion prediction unit that outputs the lesion prediction data to
a synthesizing unit that generates synthetic data in which the lesion prediction data is superimposed on the prediction data,
The determination unit receives the second medical data and the synthesized data, and stores the learned determination model for outputting a determination result as to whether or not the second subject and the first subject are the same. 2 inputting the medical data and the synthetic data, and causing the trained determination model to output the determination result;
The medical image processing apparatus according to any one of claims 1 to 3. identifying a trained prediction model corresponding to the incidental information about the first medical data from among the plurality of trained prediction models, and identifying a trained judgment model corresponding to the incidental information from among the plurality of trained judgment models; further comprising a model identification unit to identify,
The prediction unit inputs the first medical data to the identified trained prediction model and causes the identified trained prediction model to output the prediction data;
The determination unit inputs the second medical data to the identified learned determination model, and causes the identified learned determination model to output the determination result.
The medical image processing apparatus according to any one of claims 1 to 4. The incidental information is at least one of the subject's sex, the subject's race, and the imaging target site,
The medical image processing apparatus according to claim 5. The first medical data is medical data acquired in the past from the second medical data,
The medical image processing apparatus according to any one of claims 1 to 6. The prediction unit uses, as the first medical data, the medical data acquired at the time closest to the second medical data among a plurality of pieces of medical data acquired regarding the imaging target region of the first subject for the learning. input to the predictive model,
The medical image processing apparatus according to claim 7. the second medical data is medical data acquired in the past from the first medical data;
The medical image processing apparatus according to any one of claims 1 to 6. extracting lesion data indicating a lesion area in the first medical data from the first medical data acquired at the first age with respect to the imaging target site of the first subject, and subtracting the lesion data from the first medical data; an extraction unit that extracts non-lesion data indicating a non-lesion area by
A learned non-lesion data that receives the non-lesion data and outputs non-lesion prediction data in which the change over time of the non-lesion region from the first age to a second age different from the first age is reflected in the non-lesion data. a non-lesion prediction unit that inputs the non-lesion data to a lesion prediction model and outputs the non-lesion prediction data to the learned non-lesion prediction model;
inputting the lesion data into a learned lesion prediction model that receives the lesion data and outputs lesion prediction data in which the change in the lesion area over time from the first age to the second age is reflected in the lesion data; a lesion prediction unit that outputs the lesion prediction data to the learned lesion prediction model;
a combining unit that generates combined data by combining the non-lesion prediction data and the lesion prediction data;
receiving the second medical data acquired with respect to the imaging target region of the second subject at the second age and the synthesized data, and determining whether or not the second subject and the first subject are the same; a determining unit that inputs the second medical data and the synthesized data to a learned determination model that outputs a determination result, and causes the learned determination model to output the determination result;
A medical image processing apparatus comprising:

Images