JP7586653B2 - Apparatus and program - Google Patents

Description

The embodiments disclosed in this specification and drawings relate to an apparatus and a program.

Conventionally, ultrasound diagnostic devices have an algorithm (hereinafter referred to as an automatic IMT measurement function) that automatically detects the intima media thickness (hereinafter referred to as IMT), which indicates the thickness of the carotid artery wall, from a two-dimensional B-mode image of the carotid artery. When an area for measuring the plaque thickness of the carotid artery is specified by the user in the acquired two-dimensional B-mode image, the automatic IMT measurement function detects the vascular wall by analyzing the brightness within the specified area and measures the plaque thickness from the detected vascular wall.

When measuring IMT using the automatic IMT measurement function, there is a problem that the vascular wall that should be detected cannot be correctly identified due to noise mixed into the two-dimensional B-mode image, the presence or absence of plaque in the carotid artery area in the two-dimensional B-mode image, and the presence or absence of a carotid artery branch in the two-dimensional B-mode image, and therefore the measurement value expected by the user cannot be obtained. In addition, the automatic IMT measurement function requires the user to specify the location to be measured, which makes the operation complicated.

JP 2006-000456 A

Christopher M. Bishop, "Pattern recognition and machine learning", (USA), 1st edition, Springer, 2006, pp. 225-290

One of the problems that the embodiments disclosed in this specification and the drawings attempt to solve is to improve operability and accuracy in measuring the intima-media thickness in the carotid artery. However, the problems that the embodiments disclosed in this specification and the drawings attempt to solve are not limited to the above problem. Problems that correspond to the effects of each configuration shown in the embodiments described below can also be positioned as other problems.

The device according to the present embodiment includes an acquisition unit, a detection unit, and an image processing unit . The acquisition unit acquires first ultrasound data based on B-mode data obtained by transmitting and receiving ultrasound to the carotid artery of a subject. The detection unit detects information including a trace line of the vascular wall of the carotid artery in the first ultrasound data and a position of a maximum value of the intima-media thickness of the carotid artery on the trace line by inputting the first ultrasound data to a trained model trained using the plurality of second ultrasound data and information including a trace line of the vascular wall of the carotid artery and a position of a maximum value of the intima-media thickness of the carotid artery on the trace line . The image processing unit generates a superimposed image in which the information including the detected trace line and the position of the maximum value is superimposed on a B-mode image corresponding to the first ultrasound data. The second ultrasound data is at least any one of ultrasound data having a signal-to-noise ratio less than a predetermined threshold, ultrasound data related to a carotid artery having plaque, ultrasound data in which the thickness of plaque varies depending on the position along the blood flow direction, and ultrasound data related to a carotid artery having a branching portion. The detection unit inputs the first ultrasound data, which is at least one of ultrasound data having a signal-to-noise ratio below a predetermined threshold, ultrasound data relating to a carotid artery having plaque, ultrasound data in which the thickness of plaque varies depending on the position along the blood flow direction, and ultrasound data relating to a carotid artery having a branching portion, into the trained model, and detects information including a trace line of the carotid artery vascular wall in the first ultrasound data and the position of the maximum value of the intima-media complex thickness of the carotid artery on the trace line.

FIG. 1 is a diagram showing an example of the configuration of an ultrasound diagnostic apparatus according to an embodiment. FIG. 2 is a diagram showing an example of the input/output relationship of a trained model during operation of measuring the intima-media thickness (IMT) of a subject's carotid artery in the embodiment. FIG. 3 is a flowchart showing an example of a procedure of an IMT detection process according to the embodiment. FIG. 4 is a diagram showing an example of a superimposed image displayed on a display device according to the embodiment. FIG. 5 is a diagram showing an example of input and output of data for training a multi-layered network in the model generation process according to the embodiment. FIG. 6 is a diagram showing an example of multi-noise data used in generating a trained model according to an embodiment. FIG. 7 is a diagram showing an example of ultrasound data relating to a carotid artery that does not have plaque, among plaque presence/absence data used in generating a trained model according to an embodiment. FIG. 8 is a diagram showing an example of ultrasound data relating to a carotid artery having plaque, among plaque presence/absence data used in generating a trained model according to an embodiment. FIG. 9 is a diagram showing an example of thickness difference data used in generating a trained model according to an embodiment. FIG. 10 is a diagram showing an example of ultrasound data relating to a carotid artery having a branching portion, among branching presence/absence data used in generating a trained model according to an embodiment. FIG. 11 is a diagram showing an example of the positional relationship between a scanning section (scanned region) of non-parallel data used in generating a trained model and a carotid artery according to an embodiment. FIG. 12 is a flowchart illustrating an example of a procedure of an IMT detection process according to an application example of the embodiment.

The device and program according to this embodiment will be described below with reference to the drawings. To make the description more specific, an ultrasound diagnostic device will be used as an example of the device according to this embodiment. In the following embodiment, parts with the same reference numerals perform similar operations, and duplicate descriptions will be omitted as appropriate.

(Embodiment)
1 is a diagram showing an example of the configuration of an ultrasound diagnostic device 100 according to this embodiment. As shown in Fig. 1, the ultrasound diagnostic device 100 includes an ultrasound probe 1, an input device (input section) 3, a display device (display section) 5, and a device main body 7.

The ultrasonic probe 1 has a number of piezoelectric transducers, a matching layer provided on the piezoelectric transducers, and a backing material that prevents the propagation of ultrasonic waves backward from the piezoelectric transducers. The ultrasonic probe 1 is detachably connected to the device body 7. The multiple piezoelectric transducers generate ultrasonic waves based on a drive signal supplied from an ultrasonic transmission circuit 71 in the device body 7. The ultrasonic probe 1 may also be provided with buttons that are pressed for various operations such as a freeze operation.

When ultrasonic waves are transmitted from the ultrasonic probe 1 to the subject P, the transmitted ultrasonic waves are reflected one after another by the acoustic impedance discontinuity surfaces in the internal tissues of the subject P. The reflected ultrasonic waves are received as reflected wave signals (hereinafter referred to as echo signals) by multiple piezoelectric transducers of the ultrasonic probe 1. The amplitude of the received echo signal depends on the difference in acoustic impedance at the discontinuity surface where the ultrasonic waves are reflected. Note that when the transmitted ultrasonic pulse is reflected by the surface of a moving blood flow or a heart wall, the echo signal undergoes a frequency shift depending on the velocity component of the moving body in the ultrasonic transmission direction due to the Doppler effect. The ultrasonic probe 1 receives the echo signal from the subject P and converts it into an electrical signal. In this embodiment, the ultrasonic probe 1 is, for example, a 1D array probe in which multiple piezoelectric transducers are arranged along a predetermined direction, a 2D array probe in which multiple piezoelectric transducers are arranged in a two-dimensional matrix, or a mechanical 4D probe capable of performing ultrasonic scanning while mechanically tilting the piezoelectric transducer array in a direction perpendicular to the arrangement direction.

The input device 3 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs the electrical signals to the processing circuit 87. The input device 3 includes, for example, a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touchpad, and a touch panel display. In this embodiment, the input device 3 is not limited to a device that includes physical operating parts such as a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touchpad, and a touch panel display. For example, an example of the input device 3 includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the input device 3 and outputs the electrical signal to the processing circuit 87. The input device 3 may also be provided in the device main body 7. The input device 3 may also be configured as a tablet terminal or the like that can wirelessly communicate with the device main body 7.

For example, in response to pressing an end button or a freeze button on the input device 3 (hereinafter referred to as a freeze operation), the transmission and reception of ultrasound waves ends, and the ultrasound diagnostic device 100 enters a paused state. In response to the freeze operation, the ultrasound diagnostic device 100 transitions from a real-time display mode in which ultrasound images generated in conjunction with the transmission and reception of ultrasound waves are displayed in real time to a cine display mode in which multiple ultrasound images stored in the image memory 83 can be displayed continuously in chronological order (hereinafter referred to as a cine display). At this time, when the operator rotates a trackball or the like, the ultrasound diagnostic device 100 cine-displays multiple frames of ultrasound images stored in the image memory 83 according to the direction of rotation of the trackball. The rotation of the trackball corresponds to a scroll operation that scrolls a series of chronological ultrasound images along the time direction after the freeze operation is input.

The display device 5 may be, for example, a liquid crystal display (LCD), a cathode ray tube (CRT) display, an organic electroluminescence display (OELD), a plasma display, or any other display, as appropriate. The display device 5 may be incorporated into the device body 7. The display device 5 may be a desktop type, or may be configured as a tablet terminal or the like capable of wireless communication with the device body 7.

The display device 5 displays various information. For example, the display device 5 displays ultrasound images generated by the processing circuit 87 and the image generating circuit 79, a user interface (hereinafter referred to as a GUI (Graphical User Interface)) for receiving various operations from the operator, and the like. In the cine display mode, the display device 5 displays ultrasound images in chronological order in response to an instruction from the operator via the input device 3. In addition, when a scroll operation is input after a freeze operation is input (cine display), the display device 5 displays the ultrasound images corresponding to the frame numbers (indexes in the acquisition order) in the cine display frame by frame (or frame by frame) according to the amount of the scroll operation (also called cine flipping).

The device body 7 is a device that generates an ultrasound image based on the echo signal received by the ultrasound probe 1. As shown in FIG. 1, the device body 7 has an ultrasound transmission circuit 71, an ultrasound reception circuit 73, a B-mode processing circuit 75, a Doppler processing circuit 77, an image generation circuit 79, an internal memory circuit (storage unit) 81, an image memory 83 (also called a cine memory or cache), a communication interface 85, and a processing circuit 87.

The ultrasonic transmission circuit 71 is a processor that supplies a drive signal to the ultrasonic probe 1. The ultrasonic transmission circuit 71 is realized by, for example, a trigger generation circuit, a delay circuit, and a pulser circuit. The trigger generation circuit repeatedly generates a rate pulse for forming a transmission ultrasonic wave at a predetermined rate frequency by the system control function 871 in the processing circuit 87. The delay circuit provides each rate pulse with a delay time for each piezoelectric transducer required to focus the ultrasonic waves generated from the ultrasonic probe 1 into a beam and determine the transmission directivity. The pulser circuit applies a drive signal (drive pulse) to the ultrasonic probe 1 by the system control function 871 at a timing based on the rate pulse. By changing the delay time provided to each rate pulse by the delay circuit, the transmission direction from the piezoelectric transducer surface can be adjusted arbitrarily.

The ultrasonic receiving circuit 73 is a processor that performs various processes on the echo signals received by the ultrasonic probe 1 to generate received signals. The ultrasonic receiving circuit 73 is realized by, for example, an amplifier circuit, an A/D converter, a reception delay circuit, an adder, and the like. The amplifier circuit amplifies the echo signals received by the ultrasonic probe 1 for each channel and performs gain correction processing. The A/D converter converts the gain-corrected echo signals into digital signals. The reception delay circuit gives the digital signals a delay time required to determine the reception directivity. The adder adds multiple digital signals that have been given a delay time. The addition process of the adder generates a received signal in which the reflected components from the direction corresponding to the reception directivity are emphasized.

The B-mode processing circuit 75 is a processor that generates B-mode data based on the reception signal received from the ultrasonic reception circuit 73. The B-mode processing circuit 75 performs envelope detection processing, logarithmic amplification processing, and the like on the reception signal received from the ultrasonic reception circuit 73, and generates data in which the signal strength is expressed as brightness (hereinafter referred to as B-mode data). The generated B-mode data is stored in a raw data memory (not shown) as B-mode raw data on a two-dimensional ultrasonic scan line.

The Doppler processing circuit 77 is a processor that generates a Doppler waveform and Doppler data based on the reception signal received from the ultrasound reception circuit 73. The Doppler processing circuit 77 extracts a blood flow signal from the reception signal, generates a Doppler waveform from the extracted blood flow signal, and generates data (hereinafter referred to as Doppler data) in which information such as average velocity, variance, and power is extracted for multiple points from the blood flow signal. The generated Doppler data is stored in a RAW data memory (not shown) as Doppler RAW data on a two-dimensional ultrasound scan line.

The image generation circuit 79 generates a GUI for the operator to input various instructions via the input device 3. The image generation circuit 79 is a processor having a function (scan converter) of generating data of various ultrasound images based on data generated by the B-mode processing circuit 75 and the Doppler processing circuit 77. The image generation circuit 79 has an internal memory (not shown). The image generation circuit 79 generates data of a two-dimensional ultrasound image composed of pixels (hereinafter referred to as first ultrasound data) by performing RAW-pixel conversion. The first ultrasound data is generated frame by frame according to a preset frame rate. The generated first ultrasound data is stored in the internal memory frame by frame. Hereinafter, the first ultrasound data will be described as data (B-mode image) generated based on B-mode data obtained by transmitting and receiving ultrasound to the carotid artery of the subject P.

The image generation circuitry 79 may generate volume data consisting of voxels in a desired range by performing an interpolation process or the like that takes into account spatial position information on the first ultrasound data generated continuously in time. The image generation circuitry 79 may generate volume data by performing a RAW-voxel conversion that includes an interpolation process that takes into account spatial position information on the B-mode RAW data stored in the RAW data memory. The image generation circuitry 79 stores the generated volume data in the internal storage circuitry 81.

The image generation circuitry 79 may perform, for example, rendering processing or multi-planar reconstruction (hereinafter referred to as MPR (Multi Planar Reconstruction)) processing on various volume data to generate a rendering image or an MPR image. The image generation circuitry 79 also performs various image processing such as dynamic range, brightness, contrast, gamma curve correction, and RGB conversion on the generated first ultrasound data. The image generation circuitry 15 may also appropriately perform the image processing described below that is performed by the image processing function 877.

The internal memory circuit 81 is realized, for example, by a magnetic or optical storage medium, or a storage medium readable by a processor such as an integrated circuit storage device. For example, the internal memory circuit 81 corresponds to a HDD (Hard Disk Drive), SSD (Solid State Drive), or semiconductor memory that stores various information. In addition to an HDD or SSD, the internal memory circuit 17 may be a drive device that reads and writes various information between the internal memory circuit 17 and a portable storage medium such as a CD (Compact Disc), DVD (Digital Versatile Disc), or flash memory, or a semiconductor memory element such as a RAM (Random Access Memory).

The internal memory circuitry 81 stores programs and the like for implementing various functions according to this embodiment. The internal memory circuitry 81 stores a group of data such as diagnostic information (e.g., patient ID, doctor's findings, etc.), diagnostic protocols, a body mark generation program, and a conversion table that pre-sets the range of color data used for visualization for each diagnostic site. The various data stored in the internal memory circuitry 81 are transferred to an external device via the communication interface 21 by the system control function 871. The internal memory circuitry 81 stores a trained model. The trained model may be stored in the memory of the processing circuitry 87 itself. The trained model is used by the detection function 875 in the processing circuitry 87 to detect (measure) the intima media thickness (hereinafter referred to as IMT) of the carotid artery of the subject P.

Figure 2 is a diagram showing an example of the input/output relationship of the trained model during operation of IMT measurement. As shown in Figure 2, the trained model, upon input of first ultrasound data, outputs the IMT of the carotid artery in the first ultrasound data as an IMT measurement result. That is, the trained model outputs the IMT measurement result by calculating the first ultrasound data. Specifically, upon input of a B-mode image, the trained model outputs a trace line (multiple measurement points related to IMT) of the vascular wall of the carotid artery in the B-mode image and the position of the maximum value of IMT on the trace line (hereinafter referred to as the maximum position). In IMT measurement, the process of detecting IMT using the trained model (hereinafter referred to as the IMT detection process) will be described later. In addition to outputting the trace line and the maximum position, the trained model may further output the maximum value. Furthermore, upon input of a B-mode image, the trained model may output only the trace line of the carotid artery in the B-mode image.

The trained model is generated by performing machine learning on, for example, a multi-layered network using the IMT of the carotid artery in each of a plurality of second ultrasound data different from the first ultrasound data and the plurality of second ultrasound data. The multi-layered network is, for example, a machine learning model such as a deep neural network (hereinafter referred to as DNN) or a convolutional neural network (hereinafter referred to as CNN). Training the multi-layered network corresponds to adjusting a plurality of parameters in the multi-layered network. The plurality of second ultrasound data and the IMT of the carotid artery in each of the plurality of second ultrasound data correspond to training data (learning data) for the multi-layered network.

The model to be machine-learned is not limited to a multi-layered network, and any model can be used as long as the input/output relationship with the trained model can be maintained. If the first ultrasound data input to the trained model during operation is a B-mode image, the second ultrasound data used during training will be a B-mode image. The IMT of the carotid artery used during training, i.e., the correct answer data, has, for example, the trace line and maximum position of the carotid artery. The correct answer data may further have the maximum value of the IMT, or may only have the trace line of the carotid artery. The correct answer data corresponds to data in which the trace line and maximum position, etc. are specified by a doctor, etc. in the second ultrasound data. The process for generating the trained model (hereinafter referred to as the model generation process) will be described later.

The image memory 83 has, for example, a recording medium such as a semiconductor memory that can be read by a processor. The image memory 83 is realized, for example, by a cache memory. The image memory 83 saves various image data corresponding to multiple frames immediately before a freeze operation input via the input device 3. Specifically, in order to perform a cine display, the image memory 83 stores multiple frames of ultrasound images spanning a predetermined past period from the moment the freeze button was pressed so that they are not overwritten by other data. Note that the internal storage circuit 81 and the image memory 83 may be integrated into a single storage device.

The communication interface 85 is connected to an external device via a network. The communication interface 85 communicates data with the external device via the network. The external device is, for example, a medical image management system (PACS (Picture Archiving and Communication System) which is a system for managing data of various medical images, an electronic medical record system which manages electronic medical records to which medical images are attached, etc. Note that the standard for communication with the external device may be any standard, and an example of such a standard is DICOM (Digital Imaging and Communications in Medicine).

The processing circuitry 87 is, for example, a processor that functions as the core of the ultrasound diagnostic device 100. The processing circuitry 87 has, as hardware resources, processors such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit), and memories such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The processing circuit 87 may also be implemented as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), other complex programmable logic devices (CPLDs), or simple programmable logic devices (SPLDs).

The processing circuit 87 has various functions such as a system control function 871, an acquisition function 873, a detection function 875, and an image processing function 877. The processing circuit 87 deploys various programs stored in the internal storage circuit 81 in its own memory and executes them to execute the system control function 871, the acquisition function 873, the detection function 875, and the image processing function 877 corresponding to the programs. Note that instead of being stored in the internal storage circuit 81, the programs may be configured to be directly incorporated into the circuit of the processor. In this case, the processor realizes the above functions by reading and executing the programs incorporated in the circuit.

The processing circuit 87 which executes the system control function 871, the acquisition function 873, the detection function 875, and the image processing function 877 respectively corresponds to a system control unit, an acquisition unit, a detection unit, and an image processing unit. Note that the system control function 871, the acquisition function 873, the detection function 875, and the image processing function 877 are not limited to being realized by a single processing circuit. The processing circuit may be configured by combining multiple independent processors, and the system control function 871, the acquisition function 873, the detection function 875, and the image processing function 877 may be realized by each processor executing a program.

The processing circuitry 87 controls basic operations such as input and output of the ultrasound diagnostic device 100 using a system control function 871. When the system control function 871 is executed, the processing circuitry 87 accepts input of various scan modes, for example, via the input device 3. The processing circuitry 87 executes various ultrasound scans and generates various ultrasound images according to the accepted scan mode. For example, when the scan mode is B mode, the processing circuitry 87 controls the ultrasound transmission circuitry 71, the ultrasound reception circuitry 73, the B mode processing circuitry 75, and the image generation circuitry 79 to generate first ultrasound data corresponding to the B mode image on a frame-by-frame basis.

The processing circuitry 87 acquires the first ultrasound data based on the B-mode data obtained by transmitting and receiving ultrasound to the carotid artery of the subject P by using the acquisition function 873. Specifically, when a freeze operation is input via the input device 3 in the real-time display mode, the processing circuitry 87 acquires the first ultrasound data displayed on the display device 5. The processing circuitry 87 acquires the learned model from the internal storage circuitry 81 in response to the input of the freeze operation. Note that the processing circuitry 87 may acquire the first ultrasound data displayed on the display device 5 in response to the input of an instruction to detect the IMT of the carotid artery of the subject P (hereinafter referred to as an IMT measurement instruction) instead of the input of the freeze operation. In addition, if the learned model is stored in the memory of the processing circuitry 87 itself, the processing circuitry 87 acquires the learned model from its own memory in response to the input of a freeze operation or the like.

The processing circuitry 87 detects the IMT of the carotid artery in the first ultrasound data by inputting the first ultrasound data to the learned model using the detection function 875. Specifically, the processing circuitry 87 detects the trace line of the carotid artery and the position of the maximum IMT value in the B-mode image by inputting the B-mode image of the subject P as the first ultrasound data to the learned model. The output from the learned model is, for example, the coordinates of the trace line of the carotid artery in the input B-mode image and the coordinates of the maximum position on the trace line. The processing circuitry 87 calculates the maximum value of IMT on the trace line based on the coordinates of the trace line and the coordinates of the maximum position. The learned model may output the maximum value of IMT on the trace line. The processing circuitry 87 may also store the coordinates of the trace line, the maximum value of IMT on the trace line, and the coordinates of the maximum position on the trace line output from the learned model in association with the first ultrasound data input to the learned model in the internal storage circuitry 81.

The processing circuitry 87 may input a B-mode image of the subject P to the trained model as the first ultrasound data using the detection function 875, output a trace line of the carotid artery in the B-mode image, and detect the coordinates of the maximum position of the IMT on the trace line based on the coordinates of the output trace line. In this case, the output from the trained model is only the coordinates of the trace line of the carotid artery in the input B-mode image. The processing circuitry 87 detects multiple IMTs along the trace line based on the coordinates of the trace line output from the trained model. Next, the processing circuitry 87 detects the maximum value of IMT based on the detected multiple IMTs, and detects the coordinates of the maximum position based on the coordinates of the detected multiple IMTs and the detected maximum value.

The processing circuitry 87 superimposes the trace line detected by the detection function 875 and the maximum position of the IMT on the trace line on the B-mode image corresponding to the first ultrasound data by the image processing function 877. Specifically, the processing circuitry 87 generates a superimposed image by superimposing the coordinates of the trace line and the coordinates of the maximum position output from the trained model on the B-mode image in a predetermined display mode. The predetermined display mode is, for example, a thick line, a dashed line, a predetermined hue, etc. The processing circuitry 87 outputs the generated superimposed image and the maximum value of the IMT to the display device 5. The processing circuitry 87 displays the superimposed image on the display device 5 together with the maximum value of the IMT. Note that the process related to the generation of the superimposed image may be incorporated into the trained model. At this time, the trained model outputs the superimposed image in response to the input of the first ultrasound data in the detection function 875. The processing circuitry 87 may superimpose the trace line detected by the detection function 875 and the maximum position of the IMT on the trace line on the B-mode image displayed on the display device 5. The processing circuitry 87 may also appropriately perform various image processing executed by the image generation circuitry 79.

The IMT detection process will be described below with reference to FIG. 3. FIG. 3 is a flowchart showing an example of the procedure for the IMT detection process. To make the description more specific, the trained model inputs a B-mode image of the carotid artery of subject P as the first ultrasound data, and outputs the coordinates of the trace line of the carotid artery in the B-mode image and the coordinates of the maximum position. In addition, the input of the B-mode image to the trained model is triggered by a freeze operation in the real-time display mode.

(IMT detection process)
(Step S301)
By transmitting and receiving ultrasound to and from the subject P, the B-mode processing circuit 75 generates B-mode RAW data and stores the generated B-mode RAW data in a RAW data memory. The image generating circuit 79 generates a B-mode image based on the B-mode RAW data read from the RAW data memory. The display device 5 displays the B-mode image. Note that, when the cine display mode is being executed, in this step, the display device 5 displays the B-mode image by cine flipping in response to a scroll operation.

(Step S302)
When a freeze operation is performed via the input device 3 (Yes in step S302), the process of step S303 is executed. When a freeze operation is not performed via the input device 3 (No in step S302), the process of step S301 is executed. Note that when the cine display mode is being executed, if an IMT measurement instruction is input via the input device 3 (for example, pressing a button for starting IMT measurement), the process of step S303 is executed. Also, when the cine display mode is being executed, if an IMT measurement instruction is not input via the input device 3 in this step, the cine flipping described above is executed in step S301.

(Step S303)
The processing circuitry 87 acquires the B-mode image displayed on the display device 5 from the internal storage circuitry 81 or the image memory 83 by the acquisition function 873. The processing circuitry 87 reads out the trained model from the internal storage circuitry 81.

(Step S304)
The processing circuitry 87 inputs the acquired B-mode image to the trained model by the detection function 875. As a result, the processing circuitry 87 outputs the coordinates of the trace line and the coordinates of the maximum position from the trained model. The processing circuitry 87 calculates the maximum value of the IMT based on the coordinates of the maximum position. The processing circuitry 87 may also calculate the average value of multiple IMTs along the trace line.

(Step S305)
The processing circuitry 87 superimposes the trace line and the maximum position on the acquired B-mode image in a predetermined display mode using the coordinates of the trace line and the coordinates of the maximum position by the image processing function 877. In this way, the processing circuitry 87 generates a superimposed image.

(Step S306)
The display device 5 displays the generated superimposed image together with the maximum value of IMT. Fig. 4 is a diagram showing an example of the superimposed image SI displayed on the display device 5. In Fig. 4, the right common carotid artery is displayed as a B-mode image in the superimposed image SI. In the superimposed image SI, trace lines TL relating to IMT are displayed as dotted lines on the proximal and distal walls of the right common carotid artery. In addition, the superimposed image SI displays the maximum value and average value of IMT.

(Step S307)
The IMT detection process ends when an instruction to end the IMT measurement is input via the input device 3 (Yes in step S307). If an instruction to end the IMT measurement is not input via the input device 3 (No in step S307), the process from step S301 onwards is repeated.

Below, the model generation process for generating a trained model will be explained with reference to FIG. 5. FIG. 5 is a diagram showing an example of input and output of data for training a multi-layered network MLN in the model generation process. The generation of the trained model is executed, for example, by a learning device different from the ultrasound diagnostic device 100. The learning device is realized by a stand-alone computer, a server installed on a network, or the like. In addition, the above training data is stored in a memory or storage device installed in the learning device, or in a learning data storage device.

In the following, in order to make the explanation more specific, the multiple second ultrasound data input to the multi-layered network MLN as training data are described as, for example, ultrasound data with a signal-to-noise ratio (hereinafter referred to as S/N ratio) less than a predetermined threshold (hereinafter referred to as high-noise data), ultrasound data related to a carotid artery with plaque or a carotid artery without plaque (hereinafter referred to as plaque presence/absence data), ultrasound data in which the thickness of the plaque varies depending on the position along the blood flow direction (hereinafter referred to as thickness difference data), ultrasound data related to a carotid artery with a branched portion or a carotid artery without a branched portion (hereinafter referred to as branch presence/absence data), and ultrasound data in which the ultrasonic scanning cross section and the carotid artery are not parallel (hereinafter referred to as non-parallel data). Note that the second ultrasound data may include at least one of plaque presence/absence data, thickness difference data, branch presence/absence data, and non-parallel data.

Figure 6 is a diagram showing an example of high-noise data ND used in generating a trained model. As shown in Figure 6, the high-noise data ND has high noise like a haze in a B-mode image showing the carotid artery CA. The correct answer data corresponding to the high-noise data ND shown in Figure 6 has the trace line TL and maximum position MP of the carotid artery CA shown in Figure 6.

Figure 7 is a diagram showing an example of ultrasound data NPD relating to the carotid artery CA that does not have plaque, among the plaque presence/absence data used in generating the trained model. As shown in Figure 7, the ultrasound data NPD does not have a plaque area relating to the carotid artery CA. The correct answer data corresponding to the ultrasound data NPD shown in Figure 7 has the trace line TL and maximum position MP of the carotid artery CA shown in Figure 7.

Figure 8 is a diagram showing an example of ultrasound data PED for the carotid artery CA having plaque PL among the plaque presence/absence data used in generating the trained model. As shown in Figure 8, the ultrasound data PED has an area of plaque PL for the carotid artery CA. The correct answer data corresponding to the ultrasound data PED shown in Figure 8 has the trace line TL and maximum position MP of the carotid artery CA shown in Figure 8.

Figure 9 is a diagram showing an example of thickness difference data PPD used in generating a trained model. As shown in Figure 9, in the thickness difference data PPD, the thickness of the plaque PL varies depending on the position along the blood flow direction BFD. The correct answer data corresponding to the thickness difference data PPD shown in Figure 9 has the trace line TL and maximum position MP of the carotid artery CA shown in Figure 9.

Figure 10 is a diagram showing an example of ultrasound data BPD relating to a carotid artery CA having a branching portion BP among branch presence/absence data used in generating a trained model. As shown in Figure 10, in the ultrasound data BPD, the carotid artery CA has a branching portion BP. The correct answer data corresponding to the ultrasound data BPD shown in Figure 10 has a trace line TL and a maximum position MP of the carotid artery CA shown in Figure 10.

Figure 11 is a diagram showing an example of the positional relationship between the scanning section (scanned area) SS of the non-parallel data NPAD used in generating a trained model and the carotid artery CA. As shown in Figure 11, in the non-parallel data NPAD, both ends of the carotid artery CA are outside the scanning section SS. Note that in the non-parallel data NPAD, one end of the carotid artery CA may be outside the scanning section SS. The correct answer data corresponding to the non-parallel data NPAD shown in Figure 11 has the trace line and maximum position of the carotid artery CA shown in Figure 11.

The noisy data may be any data that is a B-mode image of the carotid artery on which a lot of noise is superimposed. The second ultrasound data may be data that combines at least two of the non-parallel data, plaque presence/absence data, thickness difference data, branch presence/absence data, and noisy data.

(Model generation process)
As shown in FIG. 5, the second ultrasound data as shown in FIG. 6 to FIG. 11 is input to the multi-layered network MLN. The learning device calculates the difference between the output data from the multi-layered network MLN and the correct answer data corresponding to the second ultrasound data input to the multi-layered network MLN. The learning device adjusts a plurality of parameters in the multi-layered network MLN, for example, by the backpropagation method, so that the difference (error) becomes approximately 0. The learning device inputs second ultrasound data different from the second ultrasound data used for the adjustment to the multi-layered network MLN in which the plurality of parameters have been adjusted. In the same manner, the learning device further adjusts a plurality of parameters in the multi-layered network MLN. The learning process for the multi-layered network MLN can be appropriately performed using an existing method described in, for example, Non-Patent Document 1, and therefore will not be described here.

According to the ultrasound diagnostic device 100 as an example of the device according to the embodiment described above, first ultrasound data based on B-mode data obtained by transmitting and receiving ultrasound to the carotid artery of the subject P is obtained, and the first ultrasound data is input to a trained model trained using the IMT of the carotid artery in each of the plurality of second ultrasound data and the plurality of second ultrasound data, thereby detecting the IMT of the carotid artery in the first ultrasound data. The plurality of second ultrasound data includes ultrasound data in which the scanning cross section SS by the ultrasound is non-parallel to the carotid artery (non-parallel data NPAD), ultrasound data on a carotid artery with plaque or a carotid artery without plaque (plaque presence/absence data), ultrasound data in which the thickness of the plaque differs depending on the position along the blood flow direction BFD of the carotid artery (thickness difference data PPD), ultrasound data on a carotid artery with a branch portion BP or a carotid artery without a branch portion BP (branch presence/absence data), and ultrasound data in which the S/N ratio is less than a predetermined threshold (high-noise data). The second ultrasound data may include at least one of plaque presence/absence data, thickness difference data, branch presence/absence data, and non-parallel data.

Specifically, the first ultrasound data and the second ultrasound data correspond to a B-mode image. According to the ultrasound diagnostic device 100 according to the embodiment, a trace line of the vascular wall of the carotid artery and the position of the maximum value of IMT in the first ultrasound data are detected by the trained model, and the detected trace line and the position of the maximum value are superimposed on the B-mode image corresponding to the first ultrasound data, thereby generating a superimposed image SI.

As described above, according to the ultrasound diagnostic device 100 of the embodiment, when measuring IMT, IMT can be detected with a simple operation without specifying the IMT measurement range, and the superimposed image SI can be displayed on the display device 5. As a result, according to this device, the throughput (operability) of the examination related to IMT measurement can be improved. In addition, since the second ultrasonic wave data as shown in Figures 6 to 11 is used as training data to generate a trained model used in the IMT detection process, the detection accuracy of the trace line and maximum position related to IMT measurement can be improved.

(Application example)
The first ultrasound data and the second ultrasound data in this application example are volume data. Therefore, the trained model in this application example is different from the trained model in the embodiment. The volume data corresponding to the first ultrasound data is generated by the image generation circuitry 79 by transmitting and receiving ultrasound to and from the subject P. When the trained model in the application example receives the volume data corresponding to the first ultrasound data, it outputs a trace line of the vascular wall of the carotid artery and a maximum position of the IMT in each of a plurality of cross sections in the input volume data. The plurality of cross sections are, for example, images that are orthogonal to the short-axis cross section of the carotid artery and include at least a part of the center line of the carotid artery.

Specifically, the multiple cross sections correspond to a reference plane corresponding to one scanned region in the volume data, and multiple planes (hereinafter referred to as rotational planes) obtained by rotating the reference plane at a predetermined angle around the core line. The cross section image corresponding to the reference plane corresponds to, for example, a B-mode image along the long axis of the carotid artery directly below the ultrasound probe 1. The B-mode image corresponding to the rotational plane corresponds to an MPR image along the long axis of the carotid artery.

For the sake of concrete explanation, the multiple cross sections are assumed to be three cross sections (hereinafter referred to as three cross sections). The three cross sections correspond to a reference plane and two rotation planes. The specified angles are assumed to be 120° and 240°. In these cases, the images in the multiple cross sections are a B-mode image in the reference plane and two B-mode images corresponding to the two rotation planes rotated 120° and 240° around the core line from the reference plane.

When a freeze operation is input via the input device 3 in the real-time display mode, the processing circuitry 87 acquires, by the acquisition function 873, volume data relating to the B-mode image (or MPR image) displayed on the display device 5 as the first ultrasound data. The processing circuitry 87 may also acquire volume data relating to the B-mode image (or MPR image) displayed on the display device 5 in response to the input of an IMT measurement instruction.

The processing circuitry 87 inputs the volume data corresponding to the first ultrasound data to the trained model using the detection function 875, and detects the trace line and the maximum position of the IMT in each of the three cross sections (the reference plane and two rotation planes) using the trained model. The output from the trained model is the coordinates of the trace line of the carotid artery in the three cross sections and the coordinates of the maximum position on the trace line. The processing circuitry 87 calculates the maximum value of the IMT on the trace line based on the coordinates of the trace line and the coordinates of the maximum position on each of the three cross sections.

The processing circuitry 87 generates two B-mode images corresponding to two of the three planes of rotation using MPR processing with the image processing function 877. The processing circuitry 87 superimposes the trace line detected by the detection function 875 and the maximum IMT position on the trace line on the B-mode image corresponding to each of the three planes. Specifically, the processing circuitry 87 generates three superimposed images by superimposing the coordinates of the trace line and the coordinates of the maximum position output from the trained model on each of the B-mode images of the three planes in a predetermined display mode. The processing circuitry 87 outputs the three generated superimposed images and the maximum IMT value to the display device 5. The processing circuitry 87 displays the three superimposed images on the display device 5 together with the maximum IMT value. The processing related to the generation of the three superimposed images may be incorporated into the trained model. At this time, the trained model outputs three superimposed images in response to the input of the first ultrasound data in the detection function 875.

The IMT detection process in this application example will be described below with reference to FIG. 12. FIG. 12 is a flowchart showing an example of the IMT detection process procedure.

(IMT detection process)
(Step S1201)
The B-mode processing circuitry 75 generates B-mode RAW data by transmitting and receiving ultrasonic waves to and from the subject P, and generates volume data based on the generated B-mode RAW data. The B-mode processing circuitry 75 stores the generated volume data in the internal storage circuitry 81. Other processing is similar to that in step S301, and therefore description thereof will be omitted.

(Step S1202)
If a freeze operation is performed via the input device 3 (Yes in step S1202), the process proceeds to step S1203. If a freeze operation is not performed via the input device 3 (No in step S1202), the process proceeds to step S1201. The other steps are the same as those in step S302, and therefore will not be described.

(Step S1203)
The processing circuitry 87 acquires, by an acquisition function 873, volume data corresponding to the B-mode image displayed on the display device 5 from the internal storage circuitry 81. The processing circuitry 87 reads out the trained model from the internal storage circuitry 81. The processing circuitry 87 generates, by an image processing function 877, three B-mode images corresponding to the three cross sections.

(Step S1204)
The processing circuitry 87 inputs the acquired volume data to the trained model by the detection function 875. As a result, the processing circuitry 87 outputs the coordinates of the trace line and the coordinates of the maximum position in each of the three cross sections from the trained model. The other processes are the same as those in step S304, and therefore will not be described.

(Step S1205)
The processing circuitry 87 superimposes the trace line and the maximum position on the three generated B-mode images in a predetermined display mode using the coordinates of the trace line and the coordinates of the maximum position by the image processing function 877. In this way, the processing circuitry 87 generates three superimposed images.

(Step S1206)
The display device 5 displays the three generated superimposed images together with the maximum IMT value. In this application example, B-mode images as shown in Fig. 4 are displayed for the three slices. The other processing is similar to that in step S306, and therefore description thereof will be omitted.

(Step S1207)
When an instruction to end the IMT measurement is input via the input device 3 (Yes in step S1207), the IMT detection process ends. When an instruction to end the IMT measurement is not input via the input device 3 (No in step S1207), the process from step S1201 onwards is repeated.

(Model generation process)
The model generation process for this application example will be described with reference to Fig. 5. The multiple second ultrasound data input to the multi-layered network MLN as training data are volume data having high-noise data, volume data having plaque presence/absence data, volume data having thickness difference data, volume data having branch presence/absence data, and volume data having non-parallel data. The second ultrasound data may be volume data having at least two of the non-parallel data, plaque presence/absence data, thickness difference data, branch presence/absence data, and high-noise data.

The correct answer data used during learning includes a trace line of the carotid artery vascular wall and a maximum position of IMT for each of multiple cross sections (e.g., three cross sections) in the volume data, which is the second ultrasound data. The correct answer data may further include a maximum value of IMT in each of the multiple cross sections, or may only include a trace line of the carotid artery in each of the multiple cross sections. The correct answer data corresponds to data in which the trace line and maximum position, etc. are specified by a doctor, etc., in a B-mode image in each of the multiple cross sections related to the volume data, which is the second ultrasound data.

As shown in FIG. 5, volume data having data as shown in FIG. 6 to FIG. 11 is input to the multi-layered network MLN as second ultrasound data. The learning device calculates the difference between the output data from the multi-layered network MLN and the correct answer data corresponding to the second ultrasound data input to the multi-layered network MLN. The learning device adjusts multiple parameters in the multi-layered network MLN, for example, by the backpropagation method, so that the difference (error) becomes approximately zero. The learning device inputs second ultrasound data different from the second ultrasound data used for the adjustment to the multi-layered network MLN in which the multiple parameters have been adjusted. In the same manner, the learning device further adjusts multiple parameters in the multi-layered network MLN. The learning process for the multi-layered network MLN can be performed by appropriately using existing methods described in, for example, Non-Patent Document 1, and therefore will not be described here.

According to the ultrasound diagnostic device 100 as an example of a device according to the application example of the above-described embodiment, the learned model detects the trace line of the vascular wall of the carotid artery and the position of the maximum IMT for each of the multiple cross sections in the volume data corresponding to the first ultrasound data, and superimposes the detected trace line and the position of the maximum value on multiple B-mode images corresponding to the multiple cross-sectional images, respectively, to generate multiple superimposed images. As a result, according to this application example, by inputting volume data into the learned model, multiple superimposed images corresponding to each of the multiple cross sections and having IMT measurement results (trace line of the vascular wall of the carotid artery and maximum IMT position) can be displayed. Other effects are similar to those of the embodiment, and therefore will not be described.

As a modified example of this embodiment, the device may be realized, for example, by a medical image processing device, a medical image processing server device, a workstation, or cloud computing. The medical image device, the medical image processing server device, the workstation, or cloud computing has, for example, the configuration within the dotted frame 9 shown in FIG. 1. When the device is realized as a medical image processing server device, a workstation, or cloud computing, the input device 3 and the display device 5 may be connected to a network, for example, as client devices. In this case, for example, the internal memory circuitry 81, the image memory 83, the communication interface 85, and the processing circuitry 87 may be mounted on a server on the network.

When the technical ideas of the present embodiment and the present application example are realized by a program such as a medical image processing program, the program causes a computer to obtain first ultrasound data based on B-mode data obtained by transmitting and receiving ultrasound to the carotid artery of the subject P, and to input the first ultrasound data into a trained model trained using the intima-media complex thickness of the carotid artery in each of the plurality of second ultrasound data and the plurality of second ultrasound data, thereby detecting the intima-media complex thickness of the carotid artery in the first ultrasound data. For example, the IMT detection process can be realized by installing the program in a computer such as a PACS server or integrated server in a hospital information system or an ultrasound diagnostic device 100 and expanding the program in memory. At this time, the program capable of causing a computer to execute the method can also be stored in a storage medium such as a magnetic disk (hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), or a semiconductor memory and distributed. The processing procedure and effects of the program are similar to those of the embodiment, and therefore will not be described.

According to at least one of the embodiments and modified examples described above, it is possible to improve the operability and accuracy of measuring the intima-media thickness in the carotid artery.

Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are within the scope of the invention and its equivalents as set forth in the claims, as well as the scope and spirit of the invention.

REFERENCE SIGNS LIST 1 Ultrasonic probe 3 Input device 5 Display device 7 Device body 9 Medical image processing device, medical image processing server device, or cloud computing 71 Ultrasonic transmission circuit 73 Ultrasonic reception circuit 75 B-mode processing circuit 77 Doppler processing circuit 79 Image generation circuit 81 Internal storage circuit 83 Image memory 85 Communication interface 87 Processing circuit 100 Ultrasonic diagnostic device 871 System control function 873 Acquisition function 875 Detection function 877 Image processing function

Claims (4)

an acquisition unit that acquires first ultrasound data based on B-mode data obtained by transmitting and receiving ultrasound to and from a carotid artery of a subject;
a detection unit that detects information including a trace line of the vascular wall of the carotid artery in the first ultrasound data and a position of a maximum value of the intima-media thickness of the carotid artery on the trace line by inputting the first ultrasound data into a trained model trained using the plurality of second ultrasound data , and information including a trace line of the vascular wall of the carotid artery in the first ultrasound data and a position of a maximum value of the intima-media thickness of the carotid artery on the trace line;
an image processing unit that generates a superimposed image by superimposing information including the detected trace line and the position of the maximum value on a B-mode image corresponding to the first ultrasound data;
Equipped with
The second ultrasound data is at least any one of ultrasound data having a signal-to-noise ratio less than a predetermined threshold, ultrasound data relating to a carotid artery having plaque, ultrasound data in which the thickness of plaque varies depending on the position along the blood flow direction, and ultrasound data relating to a carotid artery having a branched portion;
The detection unit detects information including a trace line of a vascular wall of a carotid artery in the first ultrasound data and a position of a maximum value of an intima-media thickness of the carotid artery on the trace line by inputting the first ultrasound data, which is at least any one of ultrasound data having a signal-to-noise ratio less than a predetermined threshold, ultrasound data relating to a carotid artery having plaque, ultrasound data in which a thickness of plaque varies depending on a position along a blood flow direction, and ultrasound data relating to a carotid artery having a branching portion, into the trained model.
Device. The first ultrasound data and the second ultrasound data are volume data,
the detection unit detects, by the trained model, a trace line of a vascular wall of the carotid artery in each of a plurality of cross sections in the volume data corresponding to the first ultrasound data and a position of a maximum value of the intima-media thickness;
An image processing unit is further provided which generates a plurality of superimposed images by superimposing the detected trace line and the position of the maximum value on a plurality of B-mode images corresponding to the plurality of cross sections, respectively.
2. The apparatus of claim 1 . On the computer,
Acquiring first ultrasound data based on B-mode data obtained by transmitting and receiving ultrasound to and from a carotid artery of a subject;
By inputting the first ultrasound data into a trained model trained using information including a trace line of the vascular wall of the carotid artery in each of a plurality of second ultrasound data and a position of a maximum value of the intima-media thickness of the carotid artery on the trace line , the first ultrasound data is detected, the trace line of the vascular wall of the carotid artery in the first ultrasound data and the position of a maximum value of the intima-media thickness of the carotid artery on the trace line;
generating a superimposed image by superimposing information including the detected trace line and the position of the maximum value on a B-mode image corresponding to the first ultrasound data;
A program for achieving the above,
The second ultrasound data is at least any one of ultrasound data having a signal-to-noise ratio less than a predetermined threshold, ultrasound data relating to a carotid artery having plaque, ultrasound data in which the thickness of plaque varies depending on the position along the blood flow direction, and ultrasound data relating to a carotid artery having a branched portion;
The detecting step includes inputting the first ultrasound data, which is at least any one of ultrasound data having a signal-to-noise ratio less than a predetermined threshold, ultrasound data relating to a carotid artery having plaque, ultrasound data in which the thickness of plaque varies depending on the position along the blood flow direction, and ultrasound data relating to a carotid artery having a branching portion, into the trained model, thereby detecting information including a trace line of a vascular wall of the carotid artery in the first ultrasound data and a position of a maximum value of an intima-media thickness of the carotid artery on the trace line.
program. an acquisition unit that acquires first ultrasound data based on B-mode data obtained by transmitting and receiving ultrasound to and from a carotid artery of a subject;
a detection unit that detects an intima-media thickness of the carotid artery in the first ultrasound data by inputting the first ultrasound data to a trained model trained using an intima-media thickness of the carotid artery in each of a plurality of second ultrasound data and the plurality of second ultrasound data;
Equipped with
The plurality of second ultrasound data include ultrasound data in which a scanning cross section of the ultrasound is non-parallel to a carotid artery, ultrasound data on a carotid artery having plaque or a carotid artery not having plaque, ultrasound data in which a thickness of plaque varies depending on a position along a blood flow direction of the carotid artery, ultrasound data on a carotid artery having a branched portion or a carotid artery not having a branched portion, and ultrasound data in which a signal-to-noise ratio is less than a predetermined threshold.
Device.

Images