JP2020175009A - Medical image processing apparatus and x-ray diagnostic apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent deterioration of visibility of a device image when the number of indwelling devices increases. A medical image processing apparatus according to an embodiment includes an acquisition unit, a device image generation unit, and an interpolated image generation unit. The acquisition unit acquires an X-ray image obtained by imaging the subject with X-rays. The device image generation unit generates a device image obtained by extracting a device drawn on the acquired X-ray image based on the acquired X-ray image and another X-ray image. The interpolated image generation unit generates an interpolated image by interpolating the missing portion of the generated device image. [Selection diagram] Fig. 1

Description

Embodiments of the present invention relate to medical image processing devices and X-ray diagnostic devices.

X-ray diagnostic equipment is used for treatment in addition to examination. For example, as a treatment for an aneurysm, a method called coil embolization is known. In coil embolization, the operator visually observes the X-ray image displayed in real time, inserts a catheter and guide wire into the blood vessel to advance to the aneurysm, and fills the aneurysm with a coil (device) to embolize. To do. As a result, the inside of the aneurysm is thrombused to prevent the aneurysm from rupturing.

In such coil embolization, it is important to fill the coil evenly to prevent recurrence. Therefore, the operator needs to fill the coil while observing the filling condition. There is a blank roadmap (hereinafter, also referred to as BRM) as a function of supporting the observation of such a filling situation. The BRM generates a difference image (BRM image) showing only the coil currently being placed by obtaining the difference between the X-ray image showing the current filling status of the coil and the mask image showing the coil already placed. It is a function to do. According to BRM, the surgeon can fill the coil while observing only the indwelling coil among the coils in the aneurysm.

Such a BRM is usually not particularly problematic, but according to the study of the present inventor, when the number of indwelling coils increases, the visibility of the device image may be deteriorated. There is room for improvement. For example, when the number of indwelling coils increases in the mask image, the indwelling coils are partially missing and displayed in the device image, and between the X-ray image and the mask image. Images due to misalignment are likely to occur. As a result, the visibility of the device image may be reduced.

Japanese Unexamined Patent Publication No. 2012-81179

Ian Goodfellow, 2 outsiders, "Chapter 9 Convolutional Networks", Deep Learning, MIT Press, 2016, p. 330-372, Internet <URL: http://www.deeplearningbook.org>

An object to be solved by the present invention is to prevent a decrease in visibility of a device image when the number of indwelling devices increases.

The medical image processing apparatus according to the embodiment includes an acquisition unit, a device image generation unit, and an interpolation image generation unit.
The acquisition unit acquires an X-ray image obtained by imaging the subject with X-rays.
The device image generation unit generates a device image obtained by extracting the device drawn on the acquired X-ray image based on the acquired X-ray image and another X-ray image.
The interpolated image generation unit generates an interpolated image by interpolating the defective portion of the generated device image.

FIG. 1 is a block diagram showing a configuration of a medical information processing system including an X-ray diagnostic apparatus and a medical image processing apparatus according to the first embodiment. FIG. 2 is a block diagram showing a configuration of an X-ray diagnostic apparatus according to the first embodiment. FIG. 3 is a block diagram showing a configuration of a medical image processing device according to the first embodiment. FIG. 4 is a flowchart for explaining the operation in the first embodiment. FIG. 5 is a schematic diagram for explaining the operation of steps ST40 to ST60 in the first embodiment. FIG. 6 is a schematic diagram for explaining a part of the operation of step ST40 in the first embodiment. FIG. 7 is a block diagram showing the configuration of the X-ray diagnostic apparatus according to the second embodiment. FIG. 8 is a block diagram showing a configuration of a medical image processing apparatus according to a second embodiment. FIG. 9 is a flowchart for explaining the operation in the second embodiment. FIG. 10 is a schematic diagram for explaining the operation of step ST1 in the second embodiment. FIG. 11 is a schematic diagram for explaining the operation of steps ST40 to ST60 in the second embodiment. FIG. 12 is a schematic diagram for explaining the operation of step ST50 in the third embodiment. FIG. 13 is a schematic diagram for explaining the operation of step ST50 in the fourth embodiment. FIG. 14 is a schematic diagram for explaining the operation of step ST50 in the fourth embodiment in comparison with the first embodiment. FIG. 15 is a schematic diagram for explaining the operation of step ST50 in the fifth embodiment. FIG. 16 is a schematic diagram for explaining the operation of step ST50 in the sixth embodiment. FIG. 17 is a schematic diagram for explaining the operation of step ST50 in the seventh embodiment. FIG. 18 is a schematic diagram for explaining the operation of step ST50 and the like in the eighth embodiment. FIG. 19 is a flowchart for explaining the operation in the ninth embodiment. FIG. 20 is a schematic diagram for explaining the operation of steps ST2 and ST71 in the ninth embodiment. FIG. 21 is a flowchart for explaining the operation in the tenth embodiment. FIG. 22 is a schematic diagram for explaining the operation of steps ST72 to ST74 in the tenth embodiment. FIG. 23 is a schematic diagram for explaining the memory in the eleventh embodiment. FIG. 24 is a flowchart for explaining the operation in the eleventh embodiment.

Hereinafter, each embodiment will be described with reference to the drawings.

<First Embodiment>
FIG. 1 is a block diagram showing a configuration of a medical information processing system including an X-ray diagnostic apparatus and a medical image processing apparatus according to the first embodiment, and FIG. 2 is a block diagram showing a configuration of the X-ray diagnostic apparatus. is there. FIG. 3 is a block diagram showing a configuration of a medical image processing device. The medical information processing system shown in FIG. 1 includes an X-ray diagnostic apparatus 1 and a medical image processing apparatus 90 capable of communicating with each other via a network Nw. The medical information processing system is not limited to this, and can be transformed into a configuration in which the X-ray diagnostic apparatus 1 or the medical image processing apparatus 90 is omitted. Further, the X-ray diagnostic apparatus 1 will be described by taking a biplane structure as an example, but the present invention is not limited to this, and a single plane structure may be used. Since the C arm 9 and the Ω arm 19 operate in substantially the same manner except for the difference in the imaging angle, the case where the C arm 9 is mainly used will be described as an example.

Here, the X-ray diagnostic apparatus 1 includes an imaging apparatus 10 and a console apparatus 70. The imaging device 10 includes a high voltage generator 3, a first X-ray tube 5, a first X-ray detector 7, a C arm 9, a first collimator 11, a second X-ray tube 15, a second X-ray detector 17, and an Ω arm 19. It includes a second collimator 21, a top plate 24, a sleeper 25, and a drive unit 27.

The high voltage generator 3 generates a tube current supplied to the first X-ray tube 5 and the second X-ray tube 15 and a tube voltage applied to the first X-ray tube 5 and the second X-ray tube 15. The high voltage generator 3 is controlled by the processing circuit 74, supplies a tube current to the first X-ray tube 5 and the second X-ray tube 15, and applies the tube voltage to the first X-ray tube 5 and the second X-ray tube 15. To do.

The first X-ray tube 5 has an X-ray focus (hereinafter referred to as a first focus) based on the tube current supplied from the high voltage generator 3 and the tube voltage applied by the high voltage generator 3. X-rays (hereinafter referred to as first X-rays) are generated. The first X-ray generated from the first focal point is applied to the subject P through an X-ray emission window provided on the front surface of the first X-ray tube 5. A part of the first X-rays generated from the first focal point is shielded by the first collimator 11 provided between the first X-ray tube 5 and the X-ray emission window. That is, the first collimator 11 is moved according to the first irradiation range input from the input interface 73, and the irradiation range of the first X-ray is limited.

The first X-ray detector 7 detects the first X-ray emitted from the first X-ray tube 5 and transmitted through the subject P. For example, the first X-ray detector 7 has a flat panel detector (hereinafter referred to as a first FPD). The first FPD has a plurality of semiconductor detection elements. There are two types of semiconductor detection elements: direct conversion type and indirect conversion type. The direct conversion type is a type in which incident X-rays are directly converted into an electric signal. The indirect conversion type is a type in which incident X-rays are converted into light by a phosphor and the light is converted into an electric signal.

A projection data generation circuit and a projection data storage circuit (not shown) are provided in the subsequent stage of the first X-ray detector 7. The projection data generation circuit is a charge / voltage converter that converts the charge read in parallel from the first FPD in rows or columns into a voltage, and an A / that converts the output of this charge / voltage converter into a digital signal. It includes a D converter and a parallel serial converter that converts a digitally converted parallel signal into a time-series serial signal. The projection data generation circuit supplies this serial signal as time-series projection data to the projection data storage circuit. The projection data storage circuit sequentially stores the time-series projection data supplied from the projection data generation circuit to generate two-dimensional projection data. This two-dimensional projection data is stored in the memory 71.

The C-arm 9 holds the first X-ray tube 5 and the first X-ray detector 7 so as to face each other with the subject P and the top plate 24 interposed therebetween, so that X-ray imaging of the subject P on the top plate 24 is performed. Has a configuration capable of performing. Further, the C-arm 9 makes it possible to change the distance between the first focal point and the first X-ray detector 7 (distance between the source image receiving surfaces (Source Image Distance: hereinafter, also referred to as the first SID)), and the first X-ray tube 5 And the first X-ray detector 7. The "first SID" is also simply referred to as a "SID". The top plate 24 is provided on the bed 25 so as to be movable along the longitudinal direction, the minor axis direction, and the vertical direction of the top plate 24.

Specifically, the C arm 9 can move along the long axis direction and the short axis direction of the top plate 24. Further, the C arm 9 is supported by the support arm via the holding portion. The support arm has a substantially arc shape, and its base end is attached to a moving mechanism for a rail provided on the ceiling. The C-arm 9 is rotatably held by the holding portion about an axis in the X direction orthogonal to both the Z direction perpendicular to the top plate 24 and the Y direction along the long axis direction of the top plate 24. .. Further, the C arm 9 has a substantially arc shape centered on the axis in the Y direction, and is held by the holding portion so as to be slidable along the substantially arc shape. That is, the C arm 9 can perform a sliding operation centered on the axis in the Y direction. Further, the C arm 9 can perform a rotational motion (hereinafter, referred to as "main rotary motion") about an axis in the X direction about the holding portion, and various angles depending on the combination of the slide and this rotation. It is possible to acquire an X-ray image from a direction (imaging angle). Further, the C arm 9 can also rotate about an axis in the Z direction, whereby, for example, the rotation center axis of the above-mentioned slide operation can be set in the X direction. The imaging axis passing through the first focal point of the first X-ray tube 5 and the center of the detection surface of the first X-ray detector 7 intersects the rotation center axis of the slide operation and the rotation center axis of the main rotation operation at one point. Designed to do. The intersection is generally called an isocenter. The isocenter does not displace even if the C arm 9 performs the above-mentioned sliding operation or main rotation operation. Therefore, when the site of interest (eg, an aneurysm) is located in the isocenter, the site of interest can be easily observed in the moving image of the medical image obtained by the sliding motion or the main rotation motion of the C arm 9.

The second X-ray tube 15 is at the focal point of X-rays (hereinafter referred to as the second focal point) based on the tube current supplied from the high voltage generator 3 and the tube voltage applied by the high voltage generator 3. X-rays (hereinafter referred to as second X-rays) are generated. The second X-ray generated from the second focal point is applied to the subject P through an X-ray emission window provided on the front surface of the second X-ray tube 15. A part of the second X-ray generated from the second focal point is shielded by the second collimator 21 provided between the second X-ray tube 15 and the X-ray emission window. That is, the second collimator 21 is moved according to the second irradiation range input from the input interface 73, and the irradiation range of the second X-ray is limited.

The second X-ray detector 17 detects X-rays generated from the second X-ray tube 15 and transmitted through the subject P. For example, the second X-ray detector 17 has a second FPD. A projection data generation circuit and a projection data storage circuit (not shown) are provided in the subsequent stage of the second X-ray detector 17. The projection data generation circuit is a charge / voltage converter that converts the charge read in parallel from the second FPD in rows or columns into a voltage, and an A / that converts the output of this charge / voltage converter into a digital signal. It includes a D converter and a parallel serial converter that converts a digitally converted parallel signal into a time-series serial signal. The projection data generation circuit supplies this serial signal as time-series projection data to the projection data storage circuit. The projection data storage circuit sequentially stores the time-series projection data supplied from the projection data generation circuit to generate two-dimensional projection data. This two-dimensional projection data is stored in the memory 71.

The Ω arm 19 holds the second X-ray tube 15 and the second X-ray detector 17 so as to face each other with the subject P on the top plate 24 interposed therebetween, so that the X-ray image of the subject P can be taken. It has a structure that can be used. Further, the Ω arm 19 can change the distance between the second focal point and the second X-ray detector 17 (distance between the source image receiving surfaces (Source Image Distance: hereinafter, also referred to as the second SID)), so that the second X-ray tube 15 can be changed. And the second X-ray detector 17. The "second SID" is also simply referred to as a "SID".

Specifically, the Ω arm 19 can move along the major axis direction and the minor axis direction of the top plate 24. Further, the Ω arm 19 is supported by the support arm via the holding portion. The support arm has a substantially arc shape, and its base end is attached to a moving mechanism for a rail provided on the ceiling. The Ω arm 19 is rotatably held by the holding portion about the axis in the Z direction perpendicular to the top plate 24 and the axis in the Y direction along the long axis direction of the top plate 24. Further, the Ω arm 19 has a substantially arc shape centered on the axis in the Y direction, and is held by the holding portion so as to be slidable along the substantially arc shape. That is, the Ω arm 19 can perform a sliding operation centered on the axis in the Y direction. Further, the Ω arm 19 can perform a rotational operation (hereinafter, referred to as “main rotation operation”) about an axis in the Z direction about the holding portion, and various angles depending on the combination of the slide and this rotation. It is possible to acquire an X-ray image from a direction (imaging angle). The imaging axis passing through the second focal point of the second X-ray tube 15 and the center of the detection surface of the second X-ray detector 17 intersects the rotation center axis of the sliding operation and the rotation center axis of the main rotation operation at one point. Designed to do. The intersection is generally called an isocenter. The isocenter does not displace even if the Ω arm 19 performs the above-mentioned sliding operation or main rotation operation. Therefore, when the site of interest (eg, an aneurysm) is located in the isocenter, the site of interest can be easily observed in the moving image of the medical image obtained by the sliding motion or the main rotation motion of the Ω arm 19.

Such a C arm 9 and an Ω arm 19 are suitable locations to which a plurality of power sources for realizing operations related to the support arm under the rail, the axis in the X direction, the axis in the Y direction, and the axis in the Z direction correspond. Be prepared for. These power sources form a drive unit 27.

The drive unit 27 reads the drive signal from the system control function 741 and causes each of the C arm 9 and the Ω arm 19 to perform a slide motion, a rotary motion, and a linear motion. Further, the C arm 9 and the Ω arm 19 are each provided with a state detector that detects information on the angle (imaging angle) or the posture and position. The state detector is composed of, for example, a potentiometer that detects the rotation angle and the amount of movement, an encoder that is a position detection sensor, and the like. As the encoder, a so-called absolute encoder such as a magnetic type, a brush type, or a photoelectric type can be used. Further, as the state detector, various types of position detection mechanisms such as a rotary encoder that outputs rotational displacement as a digital signal and a linear encoder that outputs linear displacement as a digital signal can be appropriately used.

The console device 70 includes a memory 71, a display 72, an input interface 73, a processing circuit 74, and a network interface 76.

The memory 71 includes a memory body for recording electrical information such as ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hard Disk Drive), and image memory, and a memory controller and memory interface attached to the memory body. It is equipped with peripheral circuits. The memory 71 is, for example, a program executed in the processing circuit 74, detection data (two-dimensional projection data) received from the first X-ray detector 7 and the second X-ray detector 17, and a medical image generated by the processing circuit 74. , Data used for processing of the processing circuit 74, various tables, data in the middle of processing, data after processing, and the like are stored. Examples of medical images include X-ray images (X-ray fluoroscopic images), mask images (other X-ray images), difference images, device images, interpolated images, emphasized images, contrasted blood vessel X-ray images (DSA images), and superimposition. There are images and so on. The program has, for example, an acquisition function for acquiring an X-ray image obtained by imaging a subject with X-rays on a computer, and the acquired X-ray image based on the acquired X-ray image and another X-ray image. A device image generation function for generating a device image obtained by extracting the device drawn in the above, and an interpolated image generation function for generating an interpolated image by interpolating a defective portion of the generated device image are realized. Supplementally, as such a program, for example, a program that is pre-installed in a computer from a network or a non-transient computer-readable storage medium and realizes each function of the medical image processing apparatus 77 in the computer is used. ..

The display 72 is composed of a display main body that displays various information such as medical images, an internal circuit that supplies a display signal to the display main body, and peripheral circuits such as a connector and a cable that connect the display main body and the internal circuit. There is. The internal circuit generates display data by superimposing ancillary information such as subject information and projection data generation conditions on the image data supplied from the processing circuit 74, and D / A conversion and TV format are performed on the obtained display data. Convert and display on the display body. For example, the display 72 outputs a medical image generated by the processing circuit 74, a GUI (Graphical User Interface) for receiving various operations from the operator, and the like. For example, the display 72 is a liquid crystal display or a CRT (Cathode Ray Tube) display. The display 72 is an example of a display unit. Further, the display 72 may be a desktop type, or may be composed of a tablet terminal or the like capable of wireless communication with the console device 70 main body. The display 72 is an example of a display unit.

The input interface 73 inputs subject information, sets X-ray conditions, geometric information of the imaging device 10, inputs various command signals, and the like. The subject information includes, for example, a subject ID, a subject name, a date of birth, an age, a weight, a sex, a test site, and the like. The subject information may include the height of the subject. The geometric information of the image pickup apparatus 10 is, for example, the image pickup angle and the first SID by the C arm 9 and the image pickup angle and the second SID by the Ω arm 19. The input interface 73 performs an input operation by touching, for example, a trackball, a switch button, a mouse, a keyboard, or an operation surface for instructing movement of the C arm 9 and the Ω arm 19 and setting an area of interest (ROI). It is realized by a touch pad and a touch panel display in which a display screen and a touch pad are integrated. The input interface 73 is connected to the processing circuit 74, converts the input operation received from the operator into an electric signal, and outputs the input operation to the processing circuit 74. Further, the input interface 73 may be composed of a tablet terminal or the like capable of wireless communication with the console device 70 main body. Further, in the present specification, the input interface 73 is not limited to the one provided with physical operating parts such as a mouse and a keyboard. For example, an example of the input interface 73 includes an electric signal processing circuit that receives an electric signal corresponding to an input operation from an external input device provided separately from the device and outputs the electric signal to the processing circuit 74. ..

By calling and executing the program in the memory 71, the processing circuit 74 calls and executes the system control function 741, the image processing function 742, the device image generation function 743, the interpolated image generation function 744, the emphasized image generation function 745, and the display corresponding to the program. It is a processor that realizes the control function 746. The word "processor" used in the description of the processing circuit 74 means, for example, a circuit such as a CPU, a GPU, an integrated circuit (ASIC) for a specific application, or a programmable logic device. Programmable logic devices include, for example, simple programmable logic devices (SPLDs), composite programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), and the like. This type of processor may be configured to incorporate the program directly into the circuit of the processor instead of calling and executing the program in the memory 71. Further, in FIG. 2, a system control function 741, an image processing function 742, a device image generation function 743, an interpolated image generation function 744, a emphasized image generation function 745, and a display control function 746 are realized by a single processing circuit 74. Although described as those, a processing circuit may be formed by combining a plurality of independent processors, and each function may be realized by executing a program by each processor. Further, the system control function 741, the image processing function 742, the device image generation function 743, the interpolated image generation function 744, the emphasized image generation function 745, and the display control function 746 are the system control circuit, the image processing circuit, and the device image generation circuit, respectively. It may be called an interpolated image generation circuit, an emphasized image generation circuit, and a display control circuit, and may be implemented as individual hardware circuits.

The system control function 741 temporarily stores information such as a command signal by an operator input from the input interface 73 and various initial setting conditions, and then transmits these information to each processing function of the processing circuit 74.

Further, the system control function 741 controls the drive unit 27 and the sleeper 25 by using, for example, the information regarding the drive of the C arm 9, the Ω arm 19, and the top plate 24 input from the input interface 73. For example, the system control function 741 controls the movement and rotation of the C arm 9 and the Ω arm 19, and the movement and tilt of the top plate 24.

Further, the system control function 741 reads, for example, information such as various initial setting conditions, and controls X-ray conditions such as tube voltage, tube current, and irradiation time in the high voltage generator 3. The X-ray condition may include the product of tube current and irradiation time (mAs).

The image processing function 742, for example, performs image processing such as filtering processing on the two-dimensional projection data in the memory 71 to generate an X-ray image, and saves the X-ray image in the memory 71. Here, the X-ray image based on the output of the first X-ray detector 7 may be called the first X-ray image, and the X-ray image based on the output of the second X-ray detector 17 may be called the second X-ray image. The first X-ray image and the second X-ray image are captured at two different imaging angles. However, in the following description, since the case where only the C arm 9 is used is mainly described as an example, the "first X-ray image" is simply referred to as an "X-ray image". Further, the image processing function 742 performs synthesis processing, subtraction (subtraction) processing, and the like on the obtained plurality of X-ray image data, and stores the obtained other X-ray images (mask images), difference images, and the like in the memory 71. Save to. The X-ray image obtained by subtracting the X-ray image obtained without using the contrast agent from the X-ray image obtained with the contrast agent is a contrast angiography X-ray image or DSA (Digital). Subtraction Angiography) Also called an image. The first X-ray tube 5, the first X-ray detector 7, the memory 71, and the image processing function 742 are examples of acquisition units in the X-ray diagnostic apparatus 1, and acquire an X-ray image obtained by imaging a subject with X-rays. The memory 71 and the image processing function 742 are examples of acquisition units in the medical image processing device 77, and acquire an X-ray image obtained by imaging a subject with X-rays.

The device image generation function 743 generates a device image obtained by extracting the device drawn on the acquired X-ray image based on the acquired X-ray image and another X-ray image. In the following description, other X-ray images are also referred to as mask images. The device image may be referred to as a defective image when showing a device including a defective portion. The device image generation function 743 is an example of a device image generation unit.

The interpolated image generation function 744 generates an interpolated image by interpolating the missing portion of the generated device image. The interpolated image generation function 744 is an example of an interpolated image generation unit.

The enhanced image generation function 745 generates an enhanced image in which the device is emphasized in the acquired X-ray image or the difference image between the X-ray image and the other X-ray image based on the generated interpolated image. .. For example, the enhanced image generation function 745 may generate an enhanced image by superimposing the interpolated image on the X-ray image or the difference image. The emphasized image generation function 745 is an example of an emphasized image generation unit. Further, the emphasized image generation function 745 is not essential and may be omitted.

The display control function 746 controls, for example, to read a signal from the system control function 741, acquire desired medical image data from the memory 71, and display it on the display 72. Not limited to this, the display control function 746 can display various images and various information on the display 72. The display control function 746 is an example of a display control unit.

The network interface 76 is a circuit for connecting the console device 70 to the network Nw and communicating with other devices such as the medical image processing device 90. As the network interface 76, for example, a network interface card (NIC) can be used. In the following description, the description that the network interface 76 intervenes in communication with other devices will be omitted.

The memory 71, the display 72, the input interface 73, the image processing function 742 of the processing circuit 74, the device image generation function 743, the interpolated image generation function 744, the emphasized image generation function 745, and the display control function 746 refer to the medical image processing device 77. It is configured. The medical image processing device 77 may be built in the X-ray diagnostic device 1, or may be provided outside the X-ray diagnostic device 1 as a device separate from the X-ray diagnostic device 1.

On the other hand, as shown in FIG. 3, the medical image processing apparatus 90 includes a memory 91, a display 92, an input interface 93, a processing circuit 94, and a network interface 96.

The memory 91 includes a memory main body that records electrical information such as ROM, RAM, HDD, and image memory, and peripheral circuits such as a memory controller and a memory interface attached to the memory main body. The memory 91 is, for example, a program executed in the processing circuit 94, a medical image received from the X-ray diagnostic apparatus 1, a medical image generated by the processing circuit 94, data used for processing the processing circuit 94, various tables, and processing in progress. Data and data after processing are stored. Examples of medical images include X-ray images (X-ray fluoroscopic images), mask images (other X-ray images), difference images, device images, interpolated images, emphasized images, contrasted blood vessel X-ray images (DSA images), and superimposition. There are images and so on. The program has, for example, an acquisition function for acquiring an X-ray image obtained by imaging a subject with X-rays on a computer, and the acquired X-ray image based on the acquired X-ray image and another X-ray image. A device image generation function for generating a device image obtained by extracting the device drawn in the above, and an interpolated image generation function for generating an interpolated image by interpolating a defective portion of the generated device image are realized. Supplementally, as such a program, for example, a program that is installed in the computer in advance from a network or a non-transient computer-readable storage medium and realizes each function of the medical image processing apparatus 90 in the computer is used. ..

The display 92 is composed of a display main body that displays various information such as medical images, an internal circuit that supplies a display signal to the display main body, and peripheral circuits such as a connector and a cable that connect the display main body and the internal circuit. There is. The internal circuit generates display data by superimposing ancillary information such as subject information and projection data generation conditions on the image data supplied from the processing circuit 94, and D / A conversion and TV format are performed on the obtained display data. Convert and display on the display body. For example, the display 92 outputs a medical image emphasized by the processing circuit 94, a GUI for receiving various operations from the operator, and the like. For example, the display 92 is a liquid crystal display or a CRT display. The display 92 is an example of a display unit. Further, the display 92 may be a desktop type, or may be composed of a tablet terminal or the like capable of wireless communication with the main body of the medical image processing device 90. The display 92 is an example of a display unit.

The input interface 93 inputs subject information, geometric information of the image pickup apparatus 10, various command signals, and the like. The subject information includes, for example, a subject ID, a subject name, a date of birth, an age, a weight, a sex, a test site, and the like. The subject information may include the height of the subject. The geometric information of the image pickup apparatus 10 is, for example, the image pickup angle and the first SID by the C arm 9 and the image pickup angle and the second SID by the Ω arm 19. The input interface 93 is, for example, a trackball for performing instructions related to medical information processing such as image processing, setting of an area of interest (ROI), a switch button, a mouse, a keyboard, and a touch pad for performing input operations by touching an operation surface. , And a touch panel display or the like in which the display screen and the touch pad are integrated. The input interface 93 is connected to the processing circuit 94, converts the input operation received from the operator into an electric signal, and outputs the input operation to the processing circuit 94. Further, the input interface 93 may be composed of a tablet terminal or the like capable of wireless communication with the main body of the medical image processing device 90. Further, in the present specification, the input interface 93 is not limited to the one provided with physical operating parts such as a mouse and a keyboard. For example, an example of the input interface 93 includes an electric signal processing circuit that receives an electric signal corresponding to an input operation from an external input device provided separately from the device and outputs the electric signal to the processing circuit 94. ..

The processing circuit 94 realizes the image processing function 942, the device image generation function 943, the interpolated image generation function 944, the emphasized image generation function 945, and the display control function 946 corresponding to the program by calling and executing the program in the memory 91. It is a processor to do. The term "processor" used in the description of the processing circuit 94 means, for example, a circuit such as a CPU, a GPU, an integrated circuit (ASIC) for a specific application, or a programmable logic device. Programmable logic devices include, for example, simple programmable logic devices (SPLDs), composite programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), and the like. This type of processor may be configured to incorporate the program directly into the circuit of the processor instead of calling and executing the program in the memory 91. Further, in FIG. 3, it has been described that the image processing function 942, the device image generation function 943, the interpolated image generation function 944, the emphasized image generation function 945, and the display control function 946 are realized by a single processing circuit 94. , A plurality of independent processors may be combined to form a processing circuit, and each processor may execute a program to realize each function. Further, the image processing function 942, the device image generation function 943, the interpolation image generation function 944, the emphasis image generation function 945, and the display control function 946 are the image processing circuit, the device image generation circuit, the interpolation image generation circuit, and the emphasis image generation circuit, respectively. And may be called a display control circuit, and may be implemented as an individual hardware circuit.

Here, the image processing function 742, for example, performs image processing such as filtering processing on the two-dimensional projection data in the memory 91 to generate an X-ray image, and saves the X-ray image in the memory 91. Here, the X-ray image based on the output of the first X-ray detector 7 may be called the first X-ray image, and the X-ray image based on the output of the second X-ray detector 17 may be called the second X-ray image. The first X-ray image and the second X-ray image are captured at two different imaging angles. However, in the following description, since the case where only the C arm 9 is used is mainly described as an example, the "first X-ray image" is simply referred to as an "X-ray image". Further, the image processing function 942 performs synthesis processing, subtraction (subtraction) processing, and the like on the obtained plurality of X-ray image data, and stores the obtained other X-ray image (mask image), difference image, and the like in the memory 91. Save to. The X-ray image obtained by subtracting the X-ray image obtained without using the contrast agent from the X-ray image obtained with the contrast agent is a contrast angiography X-ray image or DSA (Digital). Subtraction Angiography) Also called an image. The memory 91 and the image processing function 942 are examples of acquisition units in the medical image processing apparatus 90, and acquire an X-ray image obtained by imaging a subject with X-rays. However, the acquisition unit is not limited to this. For example, when an X-ray image is transmitted from the X-ray diagnostic apparatus 1, the network interface 96 that receives the X-ray image and stores it in the memory 91 and the memory 91 are examples of the acquisition unit.

The device image generation function 943 generates a device image obtained by extracting the device drawn on the acquired X-ray image based on the acquired X-ray image and another X-ray image. In the following description, other X-ray images are also referred to as mask images. The device image may be referred to as a defective image when showing a device including a defective portion. The device image generation function 943 is an example of a device image generation unit.

The interpolated image generation function 944 generates an interpolated image by interpolating the missing portion of the generated device image. The interpolated image generation function 944 is an example of an interpolated image generation unit.

The enhanced image generation function 945 generates an enhanced image in which the device is emphasized in the acquired X-ray image or the difference image between the X-ray image and the other X-ray image based on the generated interpolated image. .. For example, the enhanced image generation function 945 may generate an enhanced image by superimposing the interpolated image on the X-ray image or the difference image. The emphasized image generation function 945 is an example of an emphasized image generation unit. Further, the emphasized image generation function 745 is not essential and may be omitted.

The display control function 946 controls, for example, to acquire desired medical image data from the memory 91 and display it on the display 92 in response to an operation of the input interface 93. Not limited to this, the display control function 946 can display various images and various information on the display 92. The display control function 946 is an example of a display control unit.

The network interface 96 is a circuit for connecting the medical image processing device 90 to the network Nw and communicating with other devices such as the X-ray diagnostic device 1. As the network interface 96, for example, a network interface card (NIC) can be used. In the following description, the description that the network interface 96 intervenes in communication with other devices will be omitted.

The image processing function 942 in the medical image processing device 90, the device image generation function 943, the interpolated image generation function 944, the emphasized image generation function 945, and the display control function 946 are the image processing function 742 in the X-ray diagnostic device 1. It is the same function as the device image generation function 743, the interpolation image generation function 744, the emphasized image generation function 745, and the display control function 746. That is, as the medical information processing system, either the functions of the processing circuit 94 of the medical image processing device 90 or the functions of the processing circuit 74 of the X-ray diagnostic device 1 (excluding the system control function 741). The action is performed.

Next, the operation of the medical image processing apparatus configured as described above will be described with reference to the flowchart of FIG. 4 and the schematic views of FIGS. 5 and 6. The processing circuit 74 of the X-ray diagnostic apparatus 1 and the processing circuit 94 of the medical image processing apparatus 90 are substantially the same with respect to the image processing function, the device image generation function, the interpolated image generation function, the emphasized image generation function, and the display control function. To perform the operation of. Along with this, from the viewpoint of avoiding repetition of duplicated words and facilitating understanding, in the following description, the operation of each function will be described by taking the processing circuit 94 of the medical image processing apparatus 90 as a typical example. The description of such a representative example can be applied to the description of the operation of the processing circuit 74 of the X-ray diagnostic device 1 and the medical image processing device 77 by appropriately reading the device name, the reference code, and the like. This also applies to each of the following embodiments.

First, the processing circuit 74 of the X-ray diagnostic apparatus 1 controls the imaging apparatus 10 to adjust the imaging angle and SID of the C arm 9 in response to the operation of the input interface 73 by the operator, and performs X-ray fluoroscopic imaging. Start. As a result, an X-ray image of the subject P can be obtained as a moving image. This X-ray image is transmitted from the X-ray diagnostic apparatus 1 to the medical image processing apparatus 90 during the X-ray fluoroscopic imaging.

In step ST10, the medical image processing device 90 acquires an X-ray image from the X-ray diagnostic device 1, stores the X-ray image in the memory 91, and displays the X-ray image on the display 92. Under this fluoroscopy, coil embolization is initiated. In coil embolization, the operator visually observes the X-ray image displayed in real time, inserts a catheter and guide wire into the blood vessel, advances to the aneurysm, and fills the aneurysm with a coil (device). .. Strictly speaking, the real-time display does not mean the process of displaying at the moment of imaging, but the medical image processing in parallel with the process of sequentially acquiring the X-ray images on the X-ray diagnostic apparatus 1 side. This means a process in which the X-ray images are sequentially displayed on the device 90 side.

After step ST10, in step ST20, the processing circuit 94 of the medical image processing apparatus 90 determines whether or not to use the acquired X-ray image as a mask image in response to the operation of the input interface 93 by the operator. For example, when the processing circuit 94 receives the electric signal corresponding to the operation of inputting the mask image acquisition instruction from the input interface 93, the processing circuit 94 determines that the X-ray image is to be a mask image, and proceeds to step ST30. An instruction to acquire this type of mask image is input by an operator's operation every time one coil is placed in the aneurysm, for example. On the other hand, if the determination result is no, the process proceeds to step ST40.

After step ST20, in step ST30, the processing circuit 94 acquires the X-ray image acquired in step ST10 as a mask image and stores it in the memory 91. The mask image in the memory 91 is updated every time an instruction to acquire the mask image is input. The mask image may be stored in association with the geometric information of the imaging device 10. Geometric information is included in, for example, incidental information of the acquired X-ray image.

After step ST30, in step ST40, the processing circuit 94 extracts the device drawn on the acquired X-ray image based on the acquired X-ray image and the mask image (another X-ray image). Generate the device image. Specifically, for example, as shown in FIG. 5, the processing circuit 94 subtracts the mask image gm from the X-ray image gx to obtain the difference image gs. The difference image gs may be referred to as a blank roadmap (BRM) image. Further, the processing circuit 94 extracts the device being placed from the difference image gs and obtains the device image gd. Here, when extracting the indwelling device, for example, as shown in FIG. 6, a black image gbs showing the device and the artifact is obtained by a threshold process for extracting black pixels of the first threshold value or less from the difference image gs. Further, a white image gws showing an artifact is obtained by a threshold process for extracting white pixels having a second threshold value or more from the difference image gs. Next, the black image gbs and the white image gws are aligned to obtain a aligned image gd_r in which the two images are superimposed. The aligned image gd_r can be obtained, for example, by fixing one of the black image gbs and the white image gws, and performing translation processing by a rotation matrix or translational movement on the other image. You may. Next, in the aligned image gd_r, a region (a region near the white pixel) in which the white pixel region of the white image gws is slightly expanded is filled with a background color. As a result, the artifact is deleted from the aligned image gd_r, and the device image gd obtained by extracting only the device from the difference image gs is generated. In this device image gd, when the number of indwelling coils in the mask image gm increases, the indwelling coils are partially lost due to the process of subtracting the mask image gm from the X-ray image gx. Is expressed.

After step ST40, in step ST50, the processing circuit 94 generates an interpolated image gi that interpolates the missing portion of the generated device image gd, as shown in FIG. The missing portion may be interpolated with either a straight line or a curved line.

After step ST50, in step ST60, the processing circuit 94 generates an emphasized image ge that emphasizes the device in the acquired X-ray image gx or the difference image gs based on the generated interpolated image gi. For example, as shown in FIG. 5, the processing circuit 94 generates the emphasized image ge by superimposing the interpolated image gi on the difference image gs.

After step ST60, in step ST70, the processing circuit 94 causes the generated highlighted image ge to be displayed on the display 92. At this time, it is preferable that the processing circuit 94 displays the acquired X-ray image gx together with the emphasized image ge on the display 92. However, when the interpolated image gi is superimposed on the X-ray image gx to generate the emphasized image ge, the X-ray image gx may be omitted and only the emphasized image ge may be displayed. The surgeon proceeds to fill the coil while visually recognizing the emphasized image ge that emphasizes the indwelling coil (device) displayed in real time together with the X-ray image gx. In this real-time display, similarly to the above, the emphasized image ge generated from the X-ray image on the medical image processing device 90 side in parallel with the process of sequentially acquiring the X-ray image on the X-ray diagnostic device 1 side. Means the process of displaying sequentially.

After step ST70, in step ST80, the operator visually determines whether or not the coil filling is completed while visually observing the display 92. As a result of this determination, if the result is no (ST80: No), the process returns to step ST10, and the operations of steps ST10 to ST80 are continued. On the other hand, as a result of the determination in step ST80, when the filling of the coil is completed (ST80: Yes), the coil embolization is completed. This makes it possible to thrombus the inside of the aneurysm and prevent the aneurysm from rupturing.

As described above, according to the first embodiment, an X-ray image obtained by imaging a subject with X-rays is acquired. Based on the acquired X-ray image and another X-ray image, a device image obtained by extracting the device drawn on the acquired X-ray image is generated. An interpolated image is generated by interpolating the missing portion of the generated device image. As described above, the configuration of generating the interpolated image by interpolating the defective portion of the device image can prevent the deterioration of the visibility of the device image when the number of indwelling devices increases. Supplementally, according to the first embodiment, it is possible to improve the visibility of the shape of the indwelling coil, which is hidden behind the indwelling coil and is only visible in fragments. As a result, improvement in coil filling accuracy can be expected.

Further, according to the first embodiment, the emphasized image in which the device is emphasized in the acquired X-ray image or the difference image between the X-ray image and the other X-ray image based on the generated interpolated image. May be generated. In this case, since the emphasized image in which the device in the X-ray image or the difference image is emphasized can be obtained, the background of the X-ray image or the difference image can be visually recognized in addition to the above-mentioned effect.

Further, according to the first embodiment, the emphasized image may be generated by superimposing the interpolated image on the X-ray image or the difference image. In this case, the emphasized image can be easily generated as compared with the process of changing the pixel value of the device portion in the X-ray image or the difference image to generate the emphasized image.

<Second embodiment>
Next, the medical image processing apparatus according to the second embodiment will be described. In the following description, the same parts as those in the above-described drawings are designated by the same reference numerals, detailed description thereof will be omitted, and different parts will be mainly described. This also applies to each of the following embodiments.

In the second embodiment, the interpolated image generation functions 744 and 944 in the processing circuits 74 and 94 of the first embodiment use the trained model. That is, the interpolated image generation functions 744 and 944 of the processing circuits 74 and 94 relative to the trained model that generates an output image indicating the device in which the defective portion is interpolated based on the input image indicating the device including the defective portion. , Generate an interpolated image by inputting a device image. The trained model is expected to be used as a program module that is a part of AI (artificial intelligence) software. However, a trained model may be mounted in the processing circuits 74 and 94, such as ASIC and FPGA. In the present embodiment, a case where the processing circuits 74 and 94 read the trained model from the memories 71 and 91 and operate the model will be described as an example.

Such a trained model can be created as a trained model, which is a trained machine learning model, by causing the machine learning model to perform machine learning based on the training data. Here, the training data is a set of input data and output data in which an input image indicating a device including a defective portion is used as input data and an output image indicating a device in which the defective portion is interpolated is used as output data. In the machine learning model, a plurality of functions are synthesized, in which the difference image gs or the device image gd indicating the device including the defective portion is input and the difference image gs or the interpolated image gd indicating the device in which the defective portion is interpolated is output. It is a composite function with parameters. A parameterized composition function is defined by a combination of multiple adjustable functions and parameters. The machine learning model according to the present embodiment may be a composite function with any parameter that satisfies the above requirements, but is a multi-layer network model (hereinafter referred to as a multi-layer network). A trained model using a multi-layer network includes an input layer that inputs an input image indicating a device including a defective portion, an output layer that outputs an output image indicating a device that interpolates the defective portion, and an input layer and an output layer. It has at least one intermediate layer provided between the two. As such a multi-layer network, for example, a deep neural network (DNN), which is a multi-layer neural network targeted for deep learning, is used. As the DNN, for example, a recurrent neural network (RNN) or a convolutional neural network (Convolutional Neural Network: CNN) may be used. This type of CNN is also referred to as DCNN (Deep CNN). In this embodiment, DCNN is used as the trained model. Regarding CNN, for example, "Ian Goodfellow" mentioned above, two outsiders, "Chapter 9 Convolutional Networks", Deep Learning, MIT Press (MIT press), for example, "Chapter 9 Convolutional Networks", Deep Learning, MIT Press. ), 2016, p. 330-372, Internet <URL: http://www.deeplearningbook.org> ”. In addition, the RNN may include long short-term memory (Long Short-Term Memory: LSTM). The above description of the multi-layer network also applies to all the following machine learning models and trained models.

Along with this, the processing circuits 74 and 94 have a learning control function 747 for generating a trained model, as shown in FIGS. 7 and 8, in addition to the above-mentioned functions based on the programs in the memories 71 and 91. 947 can be executed. The learning control functions 747 and 947 generate a trained model which is a trained machine learning model by causing the machine learning model in the memories 71 and 91 to perform machine learning based on the learning data. The trained model includes a "self-learning model" that further updates the internal algorithm of the trained model by giving feedback to the deliverable output by the trained model.

Here, the learning data may be a plurality of pairs of an input image and an output image written in the memories 71 and 91 from the outside. In this case, for example, the input image may be an image showing a straight line or a curve including a defective portion, or an image showing a figure including the defective portion. The output image is an image obtained by interpolating the missing portion of the input image.

Alternatively, the learning data may be a plurality of pairs of the input image and the output image created by the learning control functions 747 and 947. As the training data, a pair of an input image and an output image may be created from one original image, or a pair of an input image and an output image may be created from two original images.

When creating a pair from one original image, the original image is a difference image gs indicating a device that does not include a defective portion, the output image is the original image, and the input image is the device indicated by the original image. The difference image gs formed by processing the defective portion may be used. When creating a pair from one original image, the interpolated image gi obtained by extracting the device from the difference image gs which is the output image is used as the output image, and the device image obtained by extracting the device from the difference image gs which is the input image. You may use gd as an input image. That is, as the learning data, a pair of the difference image gs not including the defective portion and the difference image gs including the defective portion may be used, or a pair of the device image gd and the interpolated image gi may be used.

When a pair is created from two original images, the first original image is a difference image gs indicating a device including a defective portion, and the second original image is a difference indicating a device including a defective portion. It may be an image gs. Such a first original image and a second original image can be acquired, for example, by imaging with different imaging conditions. In this case, the difference image gs which is the first original image may be used as the input image, and the difference image gs which is the second original image may be used as the output image. Similarly, when creating a pair from two original images, the interpolated image gi obtained by extracting the device from the difference image gs which is the output image is used as the output image, and the device is extracted from the difference image gs which is the input image. The device image gd may be used as an input image.

Further, the memories 71 and 91 store a machine learning model and a trained model in addition to the above-described configuration. Further, the programs in the memories 71 and 91 are executed by the processors of the processing circuits 74 and 94, and in addition to the above-mentioned functions, the learning control functions 747 and 947 are realized in the computer. However, even when the interpolated image generation functions 744 and 944 use the trained model, the memory of the machine learning model and the learning control functions 747 and 947 are not essential and may be omitted. Supplementally, when the trained model is used, it is not necessary for the medical image processing devices 77 and 90 to generate the trained model by learning the machine learning model, and the trained model acquired from the outside may be used. For example, the memories 71 and 91 may store a trained model in advance before the X-ray diagnostic apparatus 1 is shipped from the factory, or may be learned from a server device (not shown) after the X-ray diagnostic apparatus 1 is shipped from the factory. You may memorize the model. Further, the memories 71 and 91 are not limited to the trained model based on AI, and may store an existing interpolated image generation program that is not based on AI. This also applies to each of the following embodiments.

Other configurations are the same as in the first embodiment.

Next, the operation of the medical image processing apparatus configured as described above will be described with reference to the flowchart of FIG. 9 and the schematic views of FIGS. 10 and 11. The following description will be described in the order of the operation related to the learning control of the machine learning model (FIGS. 9 and 10) and the operation related to the generation of the interpolated image using the trained model (FIGS. 9 and 11).

(Operations related to learning control: FIGS. 9 and 10)
First, in step ST1, the processing circuit 94 causes the machine learning model md1 to perform machine learning by sequentially using the amount of learning data used for machine learning from the learning data set prepared in advance. At this time, the machine learning model md1 outputs an output image obtained by interpolating the defective portion included in the straight line or the curve based on the input image showing the straight line or the curved line including the defective portion in the training data. Further, the processing circuit 94 adjusts the parameters of the machine learning model md1 and the like so that the output image output from the machine learning model md1 approaches the output image in the training data, and applies machine learning to the machine learning model md1. Do. Upon completion of machine learning, the processing circuit 94 obtains a machine learning model md1 whose parameters and the like have been adjusted.

Subsequently, the processing circuit 94 performs transfer learning. That is, the processing circuit 94 inputs the device image gd indicating the device including the defective portion of the learning data into the machine learning model md1, and outputs the interpolated image obtained by interpolating the defective portion from the machine learning model md1. Further, the processing circuit 94 adjusts the parameters of the machine learning model md1 and the like so that the interpolated image output from the machine learning model md1 approaches the interpolated image gi in the training data, and machine learning the machine learning model md1. I do. Upon completion of machine learning, the processing circuit 94 obtains a machine learning model md1 whose parameters and the like have been adjusted.

By such transfer learning, the processing circuit 94 creates the trained model md2 as the trained machine learning model md1. The created trained model md2 is functionalized to generate an output image showing the device in which the defective portion is interpolated, based on the input image showing the device including the defective portion. The processing circuit 94 stores the created learned model md2 in the memory 91. As a result, step ST1 ends. Note that transfer learning is not essential and may be omitted. That is, in machine learning, the first half substep using the input image showing a straight line or curve including the defective portion and the output image obtained by interpolating the defective portion may be omitted. In this case, as machine learning, only the latter substep using the actual data set such as the device image gd and the interpolated image gi is executed. As the latter substep, instead of the device image gd and the interpolated image gi, a difference image gs including a defective portion and a difference image gs obtained by interpolating the defective portion may be used.

(Operation related to generation of interpolated image: FIGS. 9 and 11)
After the end of step ST1, steps ST10 to ST40 are executed in the same manner as described above, and a device image gd including the defective portion is generated.

After step ST40, in step ST50, the processing circuit 94 relatives to the trained model md2, which generates an output image showing the device that interpolates the missing part, based on the input image showing the device including the missing part. An interpolated image gi is generated by inputting the image gd.

After step ST50, in step ST60, the processing circuit 94 generates an emphasized image ge that emphasizes the device in the acquired X-ray image gx or the difference image gs based on the generated interpolated image gi. For example, as shown in the middle right side of FIG. 11, the processing circuit 94 generates the emphasized image ge by superimposing the interpolated image gi on the difference image gs. Alternatively, the processing circuit 94 may generate the emphasized image ge by coloring the interpolated image gi and superimposing the colored interpolated image gi on the X-ray image gx, for example, as shown in the lower right side of FIG. ..

After step ST60, step ST70 is executed in the same manner as described above, and the emphasized image ge is displayed on the display 92. By performing the processes of steps ST40 to ST70 within the collection interval of fluoroscopy, the operator can observe the interpolated coil on the emphasized image ge in real time.

As described above, according to the second embodiment, the device image is used for the trained model that generates an output image showing the device in which the defective portion is interpolated based on the input image showing the device including the defective portion. An interpolated image is generated by inputting. As a result, in addition to the effect of the first embodiment, the visibility of the interpolated image can be improved according to the accuracy of machine learning. Further, when transfer learning is used when machine learning a machine learning model, it is possible to reduce the number of actual data sets such as difference images, device images, and interpolated images prepared as learning data.

<Third embodiment>
The third embodiment is a modification of the first and second embodiments, in which the interpolation image generation functions 744 and 944 in the processing circuits 74 and 94 interpolate the defective portion with a curve.

That is, the interpolating image generation functions 744 and 944 of the processing circuits 74 and 94 generate the interpolating image gi by interpolating the defective portion in the device image gd with a curve. Specifically, for example, the interpolation image generation functions 744 and 944 of the processing circuits 74 and 94 execute an interpolation process of interpolating a defective portion in the device image gd with a curve in the order of collecting the device image gd.

Other configurations are the same as in the first and second embodiments.

According to the above configuration, as shown in FIG. 12, in step ST50, the processing circuit 94 generates the interpolated image gi by interpolating the defective portion in the device image gd with a curve. As a result, the configuration in which the defective portion is interpolated with a curve makes it easier to match the curved shape of the actual device (coil), so that the interpolated image is compared with the case of interpolating with a straight line in the first or second embodiment. Visibility can be improved. Further, since the defective portion in the device image is interpolated in the order of the collected device images, it is possible to suppress an error of interpolating the defective portion into a shape different from the actual device, and it is possible to improve the accuracy of the interpolated image. ..

<Fourth Embodiment>
The fourth embodiment is a modification of each of the first to third embodiments, and is a pixel indicating a device in a past interpolated image in a region interpolated by the interpolated image generation functions 744 and 944 in the processing circuits 74 and 94. It is a form that limits the area around.

Specifically, the interpolated image generation functions 744 and 944 of the processing circuits 74 and 94 set a device neighborhood area in the device image gd based on the position of the device in the past interpolated image gi, and the set device neighborhood area. A new interpolated image gi is generated by interpolating the defective portion in the image. More specifically, the interpolated image generation functions 744 and 944 set the device neighborhood region in the device image gd based on the device neighborhood region in which the distance from the pixel indicating the device in the past interpolation image gi is equal to or less than the threshold value. .. The device neighborhood region where the distance is equal to or less than the threshold value may be, for example, a region where the distance from the pixel indicating the device extracted and interpolated in the past is equal to or less than the threshold value (margin or less), and this region is assigned to the device image. , The device neighborhood area indicating the execution range of the interpolation process may be used. For example, when the pixel indicating the device is p1, a square region having a side of (2n + 1) pixels centered on the pixel p1 may be a region near the device. In this case, "n" corresponds to the threshold value. Alternatively, a substantially circular region having a radius of m pixels centered on the pixel p1 may be used as a region near the device. In this case, "m" corresponds to the threshold value. Here, the size of the threshold value may be determined based on, for example, the SID or the height of the top plate. That is, by determining the size of the threshold value based on the arrangement of the mechanical system such as the C arm 9, the Ω arm 19, and the sleeper 25, an appropriate device neighborhood region can be set according to the geometric enlargement ratio. Further, the term "device proximity region" may be appropriately read as another name such as "device peripheral region" or "interference target region".

Other configurations are the same as those of the first to third embodiments.

According to the above configuration, as shown in FIG. 13, in step ST50, the processing circuit 94 interpolates the missing portion of the device image gd of each frame along the time series, and obtains the interpolated image gi of each frame. Generate. Here, the current frame will be described as T.

The processing circuit 94 is in the current device image gd of the frame T based on the device neighborhood region r1 in which the distance from the pixel indicating the device in the past interpolated image gi (T-1) of the frame T-1 is equal to or less than the threshold value. The device neighborhood region r1 of is set. Specifically, the device neighborhood region r1 in the past interpolated image gi (T-1) is applied to the current device image gd (T) as the device neighborhood region r1.

After that, the processing circuit 94 generates a new interpolated image gi (T) by interpolating the missing portion in the device neighborhood region r1 in the current device image gd (T). As a result, even if the current device image gd (T) has noise ns, the noise ns is ignored and the defective portion is interpolated. Therefore, in addition to the effects of the first to third embodiments, the interpolation is further performed. It is possible to prevent the deterioration of the visibility of the image.

Supplementally, when the device neighborhood region r1 is not used as in the first embodiment, as shown in FIG. 14, if there is noise ns in the current device image gd (T), the missing portion including the noise ns is removed. Interpolate. In this case, the device shape in the current interpolated image gi (T) may be greatly distorted, the device shape may change significantly between adjacent frames, and the visibility of the interpolated image may decrease.

<Fifth Embodiment>
The fifth embodiment is a modification of each of the first to third embodiments, in which the interpolated image generation functions 744 and 944 in the processing circuits 74 and 94 modify the interpolated image.

Specifically, the interpolated image generation functions 744 and 944 of the processing circuits 74 and 94 generate a new interpolated image gi by modifying the generated interpolated image gi so as to match the past interpolated image gi. ..

Other configurations are the same as those of the first to third embodiments.

According to the above configuration, as shown in FIG. 15, in step ST50, the processing circuit 94 interpolates the missing portion of the device image gd of each frame along the time series, and obtains the interpolated image gi of each frame. Generate. Here, the current frame will be described as T.

The processing circuit 94 generates an interpolated image gi (T) that interpolates the missing portion of the current device image gd (T) of the frame T. At this time, if there is noise ns in the current device image gd (T), the missing portion is interpolated including the noise ns. Therefore, as shown in the center of FIG. 15, the current interpolated image gi of the distorted device shape ( T) is generated. This interpolated image gi (T) is not displayed on the display 92 because the matching with the past interpolated image gi (T-1) has not been confirmed.

Next, the processing circuit 94 aligns and compares the generated current interpolated image gi (T) and the past interpolated image gi (T-1), and ignores the unmatched fragment region r_ns. Then, the interpolation process is executed again. Here, as a method for taking measures, for example, an arbitrary matching method such as DP (Dynamic programming) matching can be used as appropriate. In any case, the processing circuit 94 generates a new interpolated image gi (T + 1) by the interpolation processing again.

By modifying the generated interpolated image gi so as to match the past interpolated image gi in this way, a new interpolated image gi is generated. Therefore, in addition to the effects of the first to third embodiments, Further, it is possible to prevent the deterioration of the visibility of the interpolated image.

<Sixth Embodiment>
The sixth embodiment is an example in which the fourth and fifth embodiments are combined, and the interpolated image generation functions 744 and 944 in the processing circuits 74 and 94 determine the process of modifying the interpolated image and the area to be interpolated. This is a form in which the setting process is executed in parallel and the preferred interpolated image is output.

Specifically, the interpolated image generation functions 744 and 944 of the processing circuits 74 and 94 execute the first generation process, the second generation process, and the output process. Here, the first generation process generates the first interpolated image by modifying the generated interpolated image so as to match the past interpolated image. The second generation process generates a second interpolated image by setting a device neighborhood region in the device image based on the position of the device in the past interpolated image and interpolating the missing portion in the set device neighborhood region. To do. More specifically, the interpolated image generation functions 744 and 944 set the device neighborhood region in the device image based on the device neighborhood region in which the distance from the pixel indicating the device in the past interpolated image is equal to or less than the threshold value. The output process outputs the interpolated image of the first interpolated image and the second interpolated image that is closer to the past interpolated image as the generated interpolated image. Here, the interpolated image generation functions 744 and 944 are examples of a first generation unit, a second generation unit, and an output unit.

Other configurations are the same as those of the first to third embodiments.

According to the above configuration, as shown in FIG. 16, in step ST50, the processing circuit 94 is a first generation processing thread (# 1) including steps ST51A and ST52A, and a first thread (# 1) including steps ST51B and ST52B. 2 Execute the generation processing thread (# 2) in parallel. For example, the processing circuit 94 executes two threads in parallel by executing each step in the order of steps ST51A, ST51B, ST52A, and ST52B.

That is, in step ST51A of the thread (# 1), the processing circuit 94 generates an interpolated image gi from the device image gd.

In step ST51B of the thread (# 2), the processing circuit 94 sets the device neighborhood region of the device image gd based on the device neighborhood region where the distance from the pixel indicating the device in the past interpolated image is equal to or less than the threshold value.

In step ST52A of the thread (# 1), the processing circuit 94 modifies the interpolated image gi generated in step ST51A based on the past interpolated image gi to generate the first interpolated image gi.

In step ST52B of the thread (# 2), the processing circuit 94 executes interpolation within the device neighborhood region set in step ST51B to generate a second interpolated image gi.

Not limited to this, when the processing circuit 94 has two processors, the first processor executes steps ST51A and ST52A of the thread (# 1) in order, and in parallel with this, the second processor threads. Steps ST51B and ST52B of (# 2) may be executed in order.

In any case, after the execution of the two threads, in step ST53, the processing circuit 94 generates an interpolation image of the first interpolation image gi and the second interpolation image gi, whichever is closer to the past interpolation image. Output as an image. Hereinafter, the processes after step ST60 are executed in the same manner as described above.

Therefore, according to the sixth embodiment, the process of setting the area to be interpolated in the fourth embodiment and the process of modifying the interpolated image of the fifth embodiment are executed in parallel, and the preferred interpolated image is executed. Is output, so that the effects of the fourth or fifth embodiment can be obtained alternately.

<7th Embodiment>
The seventh embodiment is an example in which the second and third embodiments are combined, and the interpolated image generation functions 744 and 944 in the processing circuits 74 and 94 perform processing using the trained model and curve the defective portion. This is a form in which the process of interpolating with is executed in parallel and the preferred interpolated image is output.

Specifically, the interpolated image generation functions 744 and 944 of the processing circuits 74 and 94 execute the third generation process, the fourth generation process, and the output process. The names of the "third generation process" and the "fourth generation process" referred to here are formal names representing processes different from the above-mentioned "first generation process" and "second generation process", and precede them. It does not have a first generation process and a second generation process. In other words, the names of the "third generation process" and the "fourth generation process" may be changed to other names. This also applies to the names of the "third interpolated image" and the "fourth interpolated image".

Here, in the third generation process, a device image is input to a trained model that generates an output image indicating a device in which the defective portion is interpolated based on an input image indicating a device including the defective portion. Generate a third interpolated image. The fourth generation process generates a fourth interpolated image by interpolating the defective portion with a curve. The output process outputs the interpolated image of the third interpolated image and the fourth interpolated image, which is closer to the past interpolated image, as the generated interpolated image. Here, the interpolated image generation functions 744 and 944 are examples of a third generation unit, a fourth generation unit, and an output unit.

Other configurations are the same as those of the first to third embodiments.

According to the above configuration, as shown in FIG. 17, in step ST50, the processing circuit 94 has a thread (# 1) for the first generation processing including steps ST51C and ST52C, and a second generation including step ST52D. Execute the processing thread (# 2) in parallel. For example, the processing circuit 94 executes two threads in parallel by executing the processing of steps ST51C and ST52C and the processing of step ST52D in parallel.

That is, in step ST51C of the thread (# 1), the processing circuit 94 inputs the device image to the trained model.

In step ST52C of the thread (# 1), the processing circuit 94 generates a third interpolated image gi by the trained model based on the device image input in step ST51C.

On the other hand, in step ST52D of the thread (# 2), the processing circuit 94 interpolates the defective portion of the device image with a curve to generate the fourth interpolated image gi.

Not limited to this, when the processing circuit 94 has two processors, the first processor executes steps ST51C and ST52C of threads (# 1) in order, and in parallel with this, the second processor threads. Step ST52D of (# 2) may be executed.

In any case, after the execution of the two threads, in step ST53D, the processing circuit 94 generates an interpolation image of the third interpolation image gi and the fourth interpolation image gi, whichever is closer to the past interpolation image. Output as an image. Hereinafter, the processes after step ST60 are executed in the same manner as described above.

Therefore, according to the seventh embodiment, the process using the trained model of the second embodiment and the process of interpolating the defective portion of the third embodiment with a curve are executed in parallel, and the preferred interpolation is performed. Since the image is output, the effects of the second or third embodiment can be obtained alternately.

<8th Embodiment>
The eighth embodiment is a modification of the second or seventh embodiment, and is a mode in which the device image gd input to the trained model is processed.

Specifically, the interpolated image generation functions 744 and 944 of the processing circuits 74 and 94 substantially set pixel values in a region different from the device neighborhood region in the device image based on the position of the device in the past interpolated image gi. It is corrected to 0, and the corrected device image is input to the trained model. More specifically, the interpolated image generation functions 744 and 944 set the device neighborhood region in the device image based on the device neighborhood region in which the distance from the pixel indicating the device in the past interpolated image gi is equal to or less than the threshold value. The device neighborhood region may be set as, for example, a region in which the distance from the pixel indicating the device extracted and interpolated in the past is equal to or less than the threshold value (below the margin). The size of the threshold value may be determined based on the heights of the first SID, the second SID, and the top plate 24, as described above. That is, by determining the size of the threshold value based on the arrangement of the mechanical system such as the C arm 9, the Ω arm 19, and the sleeper 25, an appropriate device neighborhood region can be set according to the geometric enlargement ratio. Further, "substantially 0" here may be a zero value or a background value of the difference image gs.

Other configurations are the same as in the second or seventh embodiment.

According to the above configuration, as shown in FIG. 18, in step ST50 (or ST51C, ST52C), the processing circuit 94 has a past interpolated image gi with respect to the device image gd of each frame in chronological order. Based on the above, the pixel value of the region different from the device neighborhood region r1 in the device image gd is corrected to substantially 0. After that, the processing circuit 94 generates the interpolated image gi of each frame by inputting the modified device image gd into the trained model. Here, the current frame will be described as T.

The processing circuit 94 corrects the pixel value of the region other than the device neighborhood region r1 in the current device image gd of the frame T to 0 value based on the past interpolated image gi (T-1) of the frame T-1. .. Specifically, the device neighborhood region r1 in the past interpolated image gi (T-1) is applied to the current device image gd (T) as the device neighborhood region r1, and the pixel value of the region different from the device neighborhood region r1. Is substantially modified to 0 to obtain the modified device image gd (T).

After that, the processing circuit 94 inputs the modified device image gd (T) to the trained model to interpolate the missing portion in the device neighborhood region r1 to obtain a new interpolated image gi (T). Generate. As a result, even if the current device image gd (T) has noise ns, the pixel of noise ns is corrected to a 0 value and the missing portion is interpolated. Therefore, in addition to the effect of the second or seventh embodiment, The accuracy of the trained model can be improved.

<9th embodiment>
The ninth embodiment is a modification of each of the first to eighth embodiments, and is a form in which a contour image showing the contour of a blood vessel is superimposed and displayed on the above-mentioned interpolated image gi or emphasized image ge. The following description will be described by exemplifying a case where the combination is performed with the first embodiment, but the present invention is not limited to this, and the combination may be combined with the second to eighth embodiments.

Along with this, the image processing functions 742 and 942 of the processing circuits 74 and 94 further acquired a contrast image obtained by X-ray imaging of the subject into which the contrast medium was injected, and extracted the contour of the blood vessel in the contrast image. Generate a contour image. For example, the processing circuits 74 and 94 obtain a DSA image by subtracting the X-ray image gx obtained without using a contrast medium from the X-ray image obtained with a contrast medium in chronological order. .. Further, the processing circuits 74 and 94 obtain an added image of the contrast-enhanced portion from the DSA image of the time series and extract the contour of the added contrast-enhanced portion to generate a contour image showing the contour of the blood vessel. The memories 71 and 91 and the image processing functions 742 and 942 are examples of contour extraction units in the medical image processing devices 77 and 90.

Further, the display control function 746 of the processing circuits 74 and 94 superimposes the contour image and the interpolated image and displays them on the displays 72 and 92.

Other configurations are the same as those of the first to eighth embodiments.

Next, the operation of the medical image processing apparatus configured as described above will be described with reference to the flowchart of FIG. 19 and the schematic diagram of FIG. The following operation will be described with reference to the case where it is combined with the first embodiment, but the present invention is not limited to this, and it may be combined with the second to eighth embodiments.

In step ST2, as shown in FIG. 20, the processing circuit 94 subtracts the X-ray image gx obtained without using the contrast medium from the X-ray image obtained with the contrast medium, and performs the DSA image g_dsa. To get. Further, the processing circuits 74 and 94 obtain the added image g_a obtained by adding the contrast-enhanced parts from the time-series DSA images g_dsa, and extract the contour of the added contrast-enhanced parts to obtain the contour image gc showing the contour of the blood vessel. Generate.

After step ST2, steps ST10 to ST50 are executed in the same manner as described above, and the interpolated image gi is generated from the device image gd.

After step ST50, in step ST71, the processing circuit 94 superimposes the contour image gc generated in step ST2 and the interpolated image gi generated in step ST50 and displays them on the display 92. The contour image gc is not limited to the interpolated image gi, and may be superimposed on the emphasized image ge and displayed.

Hereinafter, the processes after step ST80 are executed in the same manner as described above.

As described above, according to the ninth embodiment, a contrast image obtained by imaging a subject into which a contrast agent is injected by X-ray is further acquired. A contour image is generated by extracting the contour of the blood vessel in the contrast image. The contour image and the interpolated image are superimposed and displayed on the display unit (display). As a result, in addition to the effects of the first to eighth embodiments, the visibility of the interpolated image or the emphasized image can be improved by the configuration in which the contour image is superimposed.

Supplementally, as shown in FIGS. 5 and 6, for example, the interpolation process deletes the artifacts in the difference image gs, and the interpolation image gi shows only the device in place. When displaying such an interpolated image gi, it is difficult to grasp the position of the insertion destination of the device. Therefore, the visibility is improved by superimposing the contour image gc obtained from the past DSA image g_dsa image. Can be done.

<10th Embodiment>
The tenth embodiment is a modification of each of the first to eighth embodiments, and is a form in which a past interpolated image is superimposed and displayed on the above-mentioned interpolated image gi or emphasized image ge.

Along with this, the display control function 746 of the processing circuits 74 and 94 superimposes the interpolated image and the past interpolated image using a color different from the interpolated image and displays them on the displays 72 and 92.

Other configurations are the same as those of the first to eighth embodiments.

Next, the operation of the medical image processing apparatus configured as described above will be described with reference to the flowchart of FIG. 21 and the schematic diagram of FIG. 22.

Now, it is assumed that steps ST10 to ST50 are executed in the same manner as described above, and the interpolated image gi is generated from the device image gd.

After step ST50, in step ST72, the processing circuit 94 prepares past interpolated images in different colors. For example, each time the processing circuit 94 completes the placement of one coil, the interpolated image of the completed coil is stored in the memory 91 as a past interpolated image in response to the operation of the input interface 93. Then, the processing circuit 94 reads out the past interpolated images in the memory 91 and changes the colors of each other. In the example shown in FIG. 22, the processing circuit 94 reads out three past interpolated images gi and colors the device portions in the past interpolated image gi in red, green, and blue, respectively, so that the past after coloring is performed. Prepare the interpolated images g_r, gi_g, and gi_b of.

After step ST72, in step ST73, the processing circuit 94 generates a composite image g_rgb by synthesizing the past interpolated images g_r, gi_g, and gi_b after coloring.

After step ST73, in step ST74, the processing circuit 94 superimposes the generated composite image g_rgb and the interpolated image gi generated in step ST50 and displays them on the display 92. The contour image gc is not limited to the interpolated image gi, and may be superimposed on the emphasized image ge and displayed.

Hereinafter, the processes after step ST80 are executed in the same manner as described above.

As described above, according to the tenth embodiment, the interpolated image and the past interpolated image using a color different from the interpolated image are superimposed and displayed on the display unit (display). As a result, in addition to the effects of the first to eighth embodiments, the operator can visually recognize the indwelling state of the indwelled device by the configuration in which the past interpolated images are superimposed. Supplementally, the interpolation process deletes the artifacts in the difference image gs, and the interpolation image gi shows only the device in place. When displaying such an interpolated image gi, it is difficult to grasp the position of the insertion destination of the device. Therefore, the accuracy of the procedure can be improved by superimposing and displaying the past interpolated image gi.

The tenth embodiment may be executed at the same time as the ninth embodiment. That is, the display control functions 746 and 946 of the processing circuits 74 and 94 superimpose the contour image, the interpolated image, and the past interpolated image using a color different from the interpolated image and display them on the displays 72 and 92. You may. In this case, the effects of the ninth and tenth embodiments can be obtained at the same time.

<11th Embodiment>
The eleventh embodiment is a modification of each of the first to tenth embodiments, and is a geometry corresponding to geometric information at the time of imaging an X-ray image and another X-ray image (mask image gm). This is a form in which a device image is generated when the target information matches. The following description will be described by exemplifying the case of combining with the ninth embodiment, but the present invention is not limited to this, and may be combined with each of the first to eighth embodiments or the tenth embodiment.

Along with this, as shown in FIG. 23, the memories 71 and 91 and the geometric information when another X-ray image (mask image gm) in the X-ray diagnostic apparatus 1 that captures the X-ray image gx is imaged. , Is stored in association with another X-ray image (mask image gm). Here, in the case of the C arm 9, for example, the imaging angle by the C arm 9 and the above-mentioned first SID can be used as the geometric information. Similarly, in the case of the Ω arm 19, the imaging angle by the Ω arm 19 and the above-mentioned second SID can be used. The mask image gm and the geometric information of the X-ray diagnostic apparatus 1 are written to the memories 71 and 91 in association with each other by, for example, the image processing functions 742 and 942 of the processing circuits 74 and 94. Further, the memories 71 and 91 may further store the geometric information and the mask image gm, the past interpolated image gi, and the contour image gc in association with each other. Similarly, the geometric information and mask image gm, the past interpolated image gi, and the contour image gc are associated with each other by, for example, the image processing functions 742 and 942 of the processing circuits 74 and 94, and the memory 71, It is written in 91. When combined with each of the first to eighth embodiments in which the contour image gc is not used, the contour image gc is omitted. The memories 71 and 91 are examples of storage units.

Further, the device image generation functions 743 and 943 of the processing circuits 74 and 94 include geometric information of the X-ray diagnostic apparatus when the X-ray image gx is captured and the stored mask image gm (other X-rays). When the geometric information corresponding to the image) matches, the device image gd is generated based on the X-ray image gx and the mask image gm. Supplementally, if the geometrical information does not match, an artifact occurs in the difference image gs between the X-ray image gx and the mask image gm, so that the difference image gs (or device image gd) is not generated. When the difference image gs and the device image gd are not generated, for example, the display control functions 746 and 946 of the processing circuits 74 and 94 display the acquired X-ray image gx on the displays 72 and 92.

Other configurations are the same as in the ninth embodiment.

Next, the operation of the medical image processing apparatus configured as described above will be described with reference to the flowchart of FIG. 24.

Now, step ST2 is executed in the same manner as the tactic, and the contour image gc is generated.

After step ST2, in step ST3, the processing circuit 94 associates the contour image gc with the geometric information of the X-ray diagnostic apparatus 1 when the X-ray image corresponding to the contour image gc is captured, and the memory 91. Save to. In this example, the geometric information is the imaging angle by the C-arm.

After step ST3, steps ST10 to ST20 are executed in the same manner as described above, and it is determined whether or not the acquired X-ray image is used as a mask image. Here, when it is determined that the X-ray image is to be a mask image, the process proceeds to step ST30 in the same manner as described above. On the other hand, if the determination result is no, the process proceeds to step ST32, unlike the above.

After step ST20, in steps ST30 to ST31, the processing circuit 94 acquires the X-ray image acquired in step ST10 as a mask image, and obtains the mask image and the geometric information when the mask image is imaged. It is associated and saved in the memory 91.

After step ST31, in step ST32, the processing circuit 94 corresponds to the geometric information of the X-ray diagnostic apparatus when the X-ray image gx acquired in step ST10 is captured and the mask image gm in the memory 91. It is determined whether or not the geometric information to be used matches. When the two geometric information match as a result of the determination in step ST32, the processing circuit 94 executes the processing after step ST40 in the same manner as described above. In the processing after step ST40, the processing circuit 94 may search the memory 91 based on the geometric information and use the obtained contour image gc for the superimposed display in step ST71. Further, during the superimposed display in step ST71, when the X-ray image (live image) acquired in step ST10 is subjected to enlargement processing, reduction processing, rotation processing, or translation processing, the corresponding mask image gm and contour image gc Is also subjected to the same enlargement processing, reduction processing, rotation processing, or translation processing. As a result, the interpolated image and the contour image corresponding to the enlargement processing, the reduction processing, the rotation processing, or the translation processing are superimposed and displayed. As the enlargement processing and reduction processing referred to here, a method by image processing and a method of changing the SID can be appropriately used. When changing the SID, the geometric information used for the determination in step ST32 is used as the imaging angle, and the SID is not used for the determination. Further, the rotation process means rotation on a plane in a state where the shooting angle is constant. As the translation process, a method of parallel movement of the top plate 24 and the C arm 9 and a method of image processing can be appropriately used. That is, the superimposed display of the interpolated image and the contour image corresponding to the above-mentioned enlargement processing, reduction processing, rotation processing, or translation processing of the X-ray image (live image) is executed when the imaging angle is constant (ST32; Yes). Will be done. In other words, when the imaging angle is changed, steps ST40 to ST71 are not executed, and the process proceeds to step ST33.

On the other hand, if the result of the determination in step ST32 is no, the process proceeds to step ST33. Here, the case of no corresponds to a situation in which the imaging angle or the first SID is changed by the C arm 9 after imaging the mask image gm. For example, depending on the procedure, the imaging angle by the C arm 9 or the first SID may be temporarily changed and then returned to the original imaging angle or the first SID. When the geometric information is temporarily changed in this way, steps ST40 to ST71 are not executed by shifting to step ST33.

After step ST32, in step ST33, the processing circuit 94 displays the X-ray image acquired in step ST10 on the display 92, and proceeds to step ST80.

Hereinafter, the processes after step ST80 are executed in the same manner as described above.

As described above, according to the eleventh embodiment, the geometric information at the time of imaging in the acquisition unit is stored in association with other X-ray images. When the geometric information of the acquisition unit when the X-ray image is acquired and the geometric information corresponding to the other stored X-ray image match, the X-ray image and the other X-ray image are matched. Generate a device image based on the line image. In this way, since the device image is generated based on the X-ray image having the same geometric information at the time of imaging and another X-ray image, an artifact may occur in the device image that is the source of the interpolated image. It can be suppressed.

According to at least one embodiment described above, an X-ray image obtained by imaging a subject with X-rays is acquired. Based on the acquired X-ray image and another X-ray image, a device image obtained by extracting the device drawn on the acquired X-ray image is generated. An interpolated image is generated by interpolating the missing portion of the generated device image. As described above, the configuration of generating the interpolated image by interpolating the defective portion of the device image can prevent the deterioration of the visibility of the device image when the number of indwelling devices increases.

The term "processor" used in the above description refers to, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an integrated circuit for a specific application (Application Specific Integrated Circuit: ASIC), or a programmable logic device (for example,). It means circuits such as Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD), and Field Programmable Gate Array (FPGA). Processor is a storage circuit. The function is realized by reading and executing the program stored in. In this case, the processor may be configured to incorporate the program directly into the circuit of the processor instead of storing the program in the storage circuit. The function is realized by reading and executing the program incorporated in the circuit. Note that each processor of the present embodiment is not limited to the case where each processor is configured as a single circuit, and a plurality of independent circuits are provided. They may be combined to form one processor to realize the function. Further, the plurality of components in FIGS. 2, 3, 7 or 8 may be integrated into one processor to realize the function. You may try to do it.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, as well as in the scope of the invention described in the claims and the equivalent scope thereof.

1 X-ray diagnostic device 3 High voltage generator 5 1st X-ray tube 7 1st X-ray detector 9 C arm 10 Imaging device 11 1st collimator 15 2nd X-ray tube 17 2nd X-ray detector 19 Ω arm 21 2nd collimeter 24 Top plate 25 Sleeper 27 Drive unit 70 Console device 71,91 Memory 72,92 Display 73,93 Input interface 74,94 Processing circuit 741 System control function 742,942 Image processing function 743,943 Device image generation function 744,944 Interpolated image Generation function 745,945 Emphasis image generation function 746,946 Display control function 747,947 Learning control function 76,96 Network interface 90 Medical image processing device gx X-ray image gm Mask image gs Difference image gd, gd (T-1), gd (T) device image gi, gi (T-1), gi (T), gi (T + 1), gi_r, gi_g, gi_b interpolated image ge emphasized image gbs black image gws white image gd_r alignment image md1 machine learning Model md2 Trained model g_dsa DSA image g_a Addition image gc Contour image gi_rgb Composite image

Claims (14)

An acquisition unit that acquires an X-ray image of a subject imaged by X-rays,
A device image generation unit that generates a device image obtained by extracting a device drawn on the acquired X-ray image based on the acquired X-ray image and another X-ray image.
An interpolated image generator that generates an interpolated image by interpolating the missing portion of the generated device image,
A medical image processing device comprising. Based on the generated interpolated image, an enhanced image generation unit that generates an enhanced image that emphasizes the device in the acquired X-ray image or a difference image between the X-ray image and the other X-ray image is further added. The medical image processing apparatus according to claim 1. The medical image processing apparatus according to claim 2, wherein the enhanced image generation unit generates the enhanced image by superimposing the interpolated image on the X-ray image or the difference image. The interpolated image generation unit inputs the device image to a trained model that generates an output image showing the device that interpolates the defective portion based on the input image indicating the device including the defective portion. The medical image processing apparatus according to any one of claims 1 to 3, which generates an interpolated image. The medical image processing apparatus according to any one of claims 1 to 3, wherein the interpolated image generation unit generates the interpolated image by interpolating the defective portion with a curve. The interpolated image generation unit sets a device neighborhood region in the device image based on the position of the device in the past interpolated image, and interpolates the missing portion in the set device neighborhood region to perform new interpolation. The medical image processing apparatus according to any one of claims 1 to 5, which generates an image. The medical use according to any one of claims 1 to 5, wherein the interpolated image generation unit generates a new interpolated image by modifying the generated interpolated image so as to match the past interpolated image. Image processing device. The interpolated image generation unit
A first generation unit that generates a first interpolated image by modifying the generated interpolated image so as to match the past interpolated image.
A second generation that generates a second interpolated image by setting a device neighborhood region in the device image based on the position of the device in the past interpolated image and interpolating the missing portion in the set device neighborhood region. Department and
An output unit that outputs an interpolated image of the first interpolated image and the second interpolated image that is closer to the past interpolated image as the generated interpolated image.
The medical image processing apparatus according to any one of claims 1 to 5. The interpolated image generation unit
A third interpolation image is generated by inputting the device image to a trained model that generates an output image showing a device in which the defective portion is interpolated based on an input image showing a device including a defective portion. 3 generator and
A fourth generation unit that generates a fourth interpolation image by interpolating the defective portion with a curve, and
An output unit that outputs the interpolated image of the third interpolated image and the fourth interpolated image that is closer to the past interpolated image as the generated interpolated image.
The medical image processing apparatus according to any one of claims 1 to 3. The interpolated image generation unit corrects the pixel value of a region different from the device neighborhood region in the device image to substantially 0 based on the position of the device in the past interpolated image, and the corrected device image is described as described above. The medical image processing apparatus according to claim 4 or 9, which is input to the trained model. Contour extractor and
Display control unit and
With more
The acquisition unit further acquires a contrast image obtained by X-ray imaging of the subject into which the contrast medium has been injected.
The contour extraction unit generates a contour image that extracts the contour of the blood vessel in the contrast image, and generates a contour image.
The display control unit superimposes the contour image and the interpolated image and displays them on the display unit.
The medical image processing apparatus according to claim 1. The medical image processing apparatus according to claim 1, further comprising a display control unit that superimposes the interpolated image and a past interpolated image using a color different from the interpolated image and displays them on the display unit. A storage unit that stores geometrical information when the other X-ray image is imaged in the X-ray diagnostic apparatus that captures the X-ray image in association with the other X-ray image.
With more
When the device image generation unit matches the geometric information of the X-ray diagnostic apparatus when the X-ray image is captured and the geometric information corresponding to the other stored X-ray image. The medical image processing apparatus according to any one of claims 1 to 12, wherein the device image is generated based on the X-ray image and the other X-ray image. An acquisition unit that acquires an X-ray image of a subject imaged by X-rays,
A device image generation unit that generates a device image obtained by extracting a device drawn on the acquired X-ray image based on the acquired X-ray image and another X-ray image.
An interpolated image generator that generates an interpolated image by interpolating the missing portion of the generated device image,
An X-ray diagnostic device comprising.