CN113436109B - An ultrafast and high-quality plane wave ultrasound imaging method based on deep learning - Google Patents

Abstract

The invention discloses an ultrafast high-quality plane wave ultrasonic imaging method based on deep learning, belonging to the technical field of medical ultrasonic imaging. Firstly, three-dimensional channel data of synthetic aperture ultrasound at different positions of a plurality of persons are collected, and two-dimensional channel data of plane waves are generated by utilizing the three-dimensional channel data; processing the two channel data by adopting a corresponding beam forming technology to obtain ultrasonic RF data of a synthetic aperture and a plane wave in pair, and establishing a data set for deep network training; then, using plane wave RF data in the pair of data sets established in the front as the input of the network, using synthetic aperture RF data as the output label of the network, and obtaining a deep network model after training; and finally, inputting the actually acquired plane wave RF data into the trained deep network model, and outputting the estimation of the synthetic aperture RF data corresponding to the depth network model. The invention not only keeps the ultra-fast imaging speed advantage of plane wave ultrasound, but also improves the imaging quality.

Description

An ultrafast and high-quality plane wave ultrasound imaging method based on deep learning

technical field

The invention relates to the technical field of medical ultrasound imaging, in particular to an ultrafast and high-quality plane wave ultrasound imaging method based on deep learning.

Background technique

Different acquisition methods of ultrasound imaging equipment have their own advantages and disadvantages in terms of imaging speed (frame rate), imaging quality (resolution and signal-to-noise ratio), and system complexity (cost). compromise. Using all array elements to transmit and receive plane wave ultrasound is an acquisition method that can achieve ultrafast imaging, but at the expense of imaging quality.

Ultrasound equipment on the market now adopts line-by-line scanning mode, which improves the imaging quality but reduces the imaging speed. Literature[1]Z.Zhou,Y.Wang,Y.Guo,X.Jiang and Y.Qi,"Ultrafast Plane Wave ImagingWith Line-Scan-Quality Using an Ultrasound-Transfer Generative AdversarialNetwork,"in IEEE Journal of Biomedical and Health Informatics,vol.24,no.4,pp.943-956,April 2020,doi:10.1109/JBHI.2019.2950334. A deep learning method is proposed to use line-by-line scanning ultrasound images as learning labels to improve plane wave ultrasound images the quality of. In the data set established in the literature [1], the plane wave ultrasound image and the line-by-line scan ultrasound image are collected by two sets of equipment respectively, and the two kinds of images are not strictly paired. Generally speaking, compared with the model obtained by training paired samples , using unpaired samples to train deep networks for image mapping, its performance will degrade.

In [2] Jensen, J.A., Nikolov, S.I., Gammelmark, K.L., & Pedersen, M.H. (2006). Synthetic aperture ultrasound imaging. Ultrasonics, 44, e5-e15., synthetic aperture ultrasound transmits signals from one array element, all The array elements receive signals at the same time. After all the array elements transmit and receive all the signals in turn, the beamforming technology is used to add the data of all channels to obtain the imaging result. Therefore, the dynamic focusing of the transmitting unit and the receiving unit can be realized. Compared with the mainstream in the current market Line-by-line scan ultrasound is an effective technique to improve the quality (resolution and contrast) of ultrasound images. The disadvantage of synthetic aperture ultrasound is the huge amount of data that needs to be transmitted and the large amount of imaging calculation, resulting in slow imaging speed and low frame rate.

Reference [3] R.Ali,C.D.Herickhoff,D.Hyun,J.J.Dahl and N.Bottenus,"ExtendingRetrospective Encoding for Robust Recovery of the Multistatic Data Set,"in IEEE Transactions on Ultrasonics,Ferroelectrics,and Frequency Control,vol.67, no.5, pp.943-956, May 2020, doi: 10.1109/TUFFC.2019.2961875. In the case of synthetic aperture ultrasound, one array element transmits signals, all array elements receive signals at the same time, and all array elements transmit and receive all signals sequentially , so using the data collected by the synthetic aperture, according to the parameters of other collection methods, the data corresponding to the collection method can be easily generated.

The method proposed by the invention utilizes synthetic aperture ultrasound data to generate plane wave data, constructs paired data sets, trains a deep network model, and realizes the mapping of the RF data of plane wave imaging to the RF data of synthetic aperture ultrasound imaging. The method proposed in the invention improves the imaging quality by using deep learning while maintaining the ultra-fast imaging speed advantage of plane wave ultrasound.

SUMMARY OF THE INVENTION

The purpose of the present invention is to propose an ultra-fast and high-quality plane wave ultrasound imaging method based on deep learning, which is characterized by comprising the following steps:

Step 1: construct a paired RF data set; use an ultrasonic platform to collect three-dimensional channel data of synthetic aperture ultrasound, and establish a paired RF data set of synthetic aperture-plane wave ultrasound;

Step 2: train a deep network model; construct a deep network and a Loss function, train a deep network using the paired RF data sets obtained in step 1, and obtain a deep network model;

Step 3: Deploy a deep network; input the RF data of the plane wave ultrasound obtained in real time into the deep network model trained in step 2, and obtain the output of the network as the enhanced ultrasound RF data.

The step 1 includes the following sub-steps:

其中，j＝1…N，N为样本数，x_t为发射阵元的坐标，x_r为接收阵元的坐标，t为声波的双程传播时间；Step 11: For each sample, accumulate the synthetic aperture ultrasound channel data d _j (x _t , x _r , t) obtained by all transmitting array elements to obtain two-dimensional plane wave channel data

Among them, j=1...N, N is the number of samples, x _t is the coordinate of the transmitting array element, x _r is the coordinate of the receiving array element, and t is the two-way propagation time of the acoustic wave;

Step 12: Use synthetic aperture ultrasonic beamformer B ₁ to process d _j (x _t , x _r , t) to obtain synthetic aperture RF data o _j (x, t)=B ₁ {d _j (x _t , x _r , t)}, (x, t) is the spatial coordinate of the imaging point;

Step 13: Use the plane wave ultrasonic beamformer B ₂ to process p _j (x _r , t) to obtain plane wave RF data i _j (x, t)=B ₂ {p _j (x _r , t)};

Step 14: After processing all samples according to steps 11 to 13, a paired RF data set D={i _j (x, t), o _j (x, t), j=1...N} is obtained, where N is the sample number.

The Loss function in the step 2 is:

Among them, f(i _j (x, t), W) represents the deep network calculation process, and W represents the network parameters.

The beneficial effects of the present invention are:

The method proposed in the invention not only maintains the speed advantage of ultra-fast imaging of plane wave ultrasound, but also improves the imaging quality by using deep learning.

Description of drawings

Fig. 1 is the general flow chart of the present invention;

Fig. 2 (a) is the flow chart of the paired dataset part; Fig. 2 (b) is the flow chart of the deep network training part; Fig. 2 (c) is the flow chart of the deep network deployment part;

Figure 3 shows the adopted deep network;

Among them: "k3n16s1" indicates that the size of the convolution kernel is 3×3, the number of channels is 16, and the step size is 1; "×2" indicates that the operation is repeated twice, and other labels are the same;

Fig. 4 is the RF data pair in the training set; wherein, (a) is the 16 plane wave RF data samples; (b) is the corresponding synthetic aperture RF data sample;

Fig. 5 is the test example of the test set of the present invention; wherein, (a) is the B ultrasound of the plane wave RF data input by the deep network; (b) is the B ultrasound of the RF data output by the deep network; (c) is the B ultrasound of the synthetic aperture ultrasound overtake.

Detailed ways

The present invention proposes an ultra-fast and high-quality plane wave ultrasound imaging method based on deep learning. The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

Fig. 1 is the general flow chart of the present invention, Fig. 2 (a) is the flow chart of the paired data set part; Fig. 2 (b) is the flow chart of the deep network training part; Fig. 2 (c) is the flow chart of the deep network deployment part Figure; specific can be expressed as follows:

1) Build a paired dataset:

a) Use the ultrasound platform to collect the three-dimensional channel data of synthetic aperture ultrasound of multiple people and multiple parts, denoted as d _j (x _t , x _r , t), where j=1...N, N is the number of samples, and x _t is the emission The coordinates of the array element, x _r is the coordinate of the receiving array element, and t is the two-way propagation time of the sound wave.

b) For each sample, accumulate the synthetic aperture ultrasound channel data d _j (x _t , x _r , t) obtained by all transmitting array elements together to obtain two-dimensional plane wave channel data

c) Process d _j (x _t , x _r , t) using the synthetic aperture ultrasonic beamformer B ₁ in "Lu Wenkai; An Efficient Synthetic Aperture Ultrasound Imaging Method 2021.5.18 China CN202110539280.3" to obtain RF data o _j (x,t)=B ₁ {d _j (x _t ,x _r ,t)}.

d) Using M.Albulayli and D.Rakhmatov, "Fourier Domain Depth Migration for Plane-Wave Ultrasound Imaging," in IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, vol.65, no.8, pp.1321-1333, Aug. .2018, doi:10.1109/TUFFC.2018.2837000. The plane wave ultrasonic beamformer B ₂ processes p _j (x _r , t) to obtain RF data i _j (x, t) = B ₂ {p _j (x _r , t) )}.

e) After all samples are processed, a paired data set D={i _j (x, t), o _j (x, t), j=1...N{, N is the number of samples is obtained.

2) Train a deep network model:

a) Compared with the RF data o _j (x, t) of the synthetic aperture ultrasound, the RF data _ij (x, t) of the plane wave ultrasound is degraded due to the presence of relatively serious crosstalk noise. For the above image enhancement problem, we build a deep network and loss function. Figure 3 shows the deep network structure used in this experiment. As shown in Figure 3, we adopt a two-dimensional U-Net model. The U-Net model includes an encoding process and a decoding process, and there are jumpers between the encoding and decoding processes to extract features of different scales. In addition, other reasonable end-to-end network structures can also be used instead. The loss function of the training process is:

Among them, f( ) represents the calculation process of the deep network, W represents the network parameters, and the training process uses Adam or other reasonable optimizers to optimize the network parameters.

b) Use the paired dataset D established in step 1 to train the aforementioned deep network to obtain the deep network model f(i(x,t),W). Figure 4 presents the sample pairs in the training set.

3) Deploy a deep network:

作为增强后的超声RF数据。a) Input the RF data i(x,t) of the plane wave ultrasound obtained in real time into the deep network model f(i(x,t),W) trained in the second step, and get the output of the network

as enhanced ultrasound RF data.

得到用于显示的B超。b) use

Get a B-ultrasound for display.

The three-channel synthetic aperture ultrasound channel data of multiple parts of 12 people was collected, and the synthetic aperture channel data was used to generate the channel data corresponding to the plane wave, and then the synthetic aperture imaging and the plane wave imaging were performed respectively, and a total of 605 pairs of samples were obtained. Among them, 424 groups are used as training sets, 60 groups are used as validation sets, and 121 groups are used as test sets. The size of each sample image is 3100×256. During training, we randomly divided the training set into 10,000 training samples with a size of 64 × 64, set the batch size to 64, the number of training iterations to 100, and the initial learning rate of 0.0001. Adam optimizer was used for optimization. The experimental platform includes Intel(R) Core(TM) i9-9820X CPU@3.30GHz, 64GB RAM, GeForce RTX2080Ti, GeForce RTX 3090. Figure 5 shows the experimental results of the present invention on the test set. 5(a) is the ultrasound of the plane wave RF data input by the deep network; 5(b) is the ultrasound of the RF data output by the deep network; 5(c) is the ultrasound of the synthetic aperture ultrasound (the standard answer of Fig. 5(b) ). It can be seen from FIG. 5 that the quality of the B-ultrasound image obtained by the present invention is obviously better than that of the plane wave B-ultrasound image. Table 1 presents the performance metrics on the test set.

Table 1 Comparison of test set performance indicators

It can be seen that after the enhancement of this method, the SSIM, PSNR, and SNR of RF images and B-ultrasound images are improved. Table 2 presents the time for deep network processing one frame on different GPU cards.

Table 2 Comparison table of deep network processing time consumption of different GPU cards (unit: frame/second)

It can be seen from Table 2 that the processing speed of raw RF data on Geforce RTX 3090 reaches 130 frames per second, and if the raw RF data is properly downsampled, the processing speed can be further improved.

To sum up, the technology of the present invention greatly improves the imaging quality by using deep learning while maintaining the ultra-fast imaging speed advantage of plane wave ultrasound.

This embodiment is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention. , all should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (2)

1. A plane wave ultrasonic imaging method based on deep learning is characterized by comprising the following steps:

step 1: constructing a pair of RF data sets; acquiring three-dimensional channel data of synthetic aperture ultrasound by using an ultrasound platform, and establishing a pair RF (radio frequency) data set of synthetic aperture-plane wave ultrasound;

the step 1 comprises the following substeps:

step 11: for each sample, the synthetic aperture ultrasonic channel data d obtained by all the transmitting array elements _j (x _t ，x _r T) are added together to obtain two-dimensional plane wave channel data

Wherein j =1 … N, N is the number of samples, x _t Being coordinates of transmitting array elements, x _r In order to receive the coordinates of the array elements, t is the two-way propagation time of the sound waves;

step 12: using a synthetic aperture ultrasound beamformer B ₁ Treatment d _j (x _t ，x _r T) to obtain synthetic aperture RF data o _j (x，t)＝B ₁ {d _j (x _t ，x _r T), (x, t) is the spatial coordinate of the imaging point;

step 13: using plane wave ultrasonic beamformer B ₂ Treatment of p _i (x _r T) obtaining plane wave RF data i _j (x，t)＝B ₂ {p _j (x _r ，t)}；

Step 14: after all samples are processed according to the steps 11 to 13, a pair RF data set D = { i } is obtained _j (x，t)，o _j (x，t)J =1 … N }, N being the number of samples;

step 2: training a deep network model; constructing a deep network and a Loss function, and training the deep network by using the paired RF data sets obtained in the step (1) to obtain a deep network model;

and step 3: deploying a deep network; and (3) inputting the RF data of the plane wave ultrasound acquired in real time into the deep network model trained in the step (2), and acquiring the output of the network as the enhanced ultrasonic RF data.

2. The deep learning-based plane wave ultrasonic imaging method according to claim 1, wherein the Loss function in the step 2 is:

wherein, f (i) _j (x, t), W) represents a deep network computation process, and W represents a network parameter.

Images