CN114549462A - Focus detection method, device, equipment and medium based on visual angle decoupling Transformer model - Google Patents

Abstract

The invention discloses a focus detection method, a device, equipment and a storage medium based on a visual angle decoupling transform model, and the method is realized and comprises the following steps: acquiring MRI image data of a user to be detected, and preprocessing the MRI image data; acquiring slice data according to the preprocessed MRI image data, and inputting the slice data into a pre-trained focus detection model; performing feature extraction on the slice data to acquire high-level feature data; fusing high-level feature data on different scales through different visual angles to obtain high-level feature prediction data; and predicting the focus of the user to be detected through high-level feature prediction data. By fusing the extracted feature data on different scales through different visual angles, the context information of the input slice can be enhanced, multi-scale feature prediction is realized, the positioning precision of the focus is improved, the early-stage tumor recognition effect is strong, and the detection accuracy is effectively improved.

Description

Lesion detection method, device, equipment and medium based on perspective decoupling Transformer model

technical field

The invention relates to the technical fields of deep learning and medical imaging, and in particular to a method, device, equipment and storage medium for lesion detection based on a perspective decoupling Transformer model

Background technique

With the continuous improvement of economic level and the rapid development of medical technology, people have higher demands for health. Therefore, medical imaging technology has been innovated and developed vigorously. The scientific use of medical image analysis to efficiently and accurately analyze tissue and cell images. Classification can help doctors better explore the path of treatment of lesions.

At present, in traditional lesion detection, taking brain tumor as an example, template matching is usually used to calculate the position of a predefined brain tumor template in the image. These methods are affected by the inaccuracy of hand-designed features; in addition, lesion segmentation can also be used. However, these methods need to mark the pixels of each image as labels for training, which requires the cooperation of professional radiology doctors, and the price of obtaining labels is relatively expensive; or in some other methods, it is necessary to The lesions are detected using datasets of large-sized tumors or predicting tumor boxes with existing 2D object detection networks, but these methods have low recognition for small tumors and lack the context fusion of 3D features.

It can be seen that since brain tumors are very small in the early stage, the current method is easy to miss or confuse, and it is difficult to detect, thus affecting the accuracy of detection.

SUMMARY OF THE INVENTION

Based on this, it is necessary to provide a lesion detection method, device, device and storage medium based on the perspective decoupling Transformer model to solve the problems in the automatic brain tumor detection in the prior art. detection, which affects the detection accuracy.

In a first aspect, a lesion detection method based on a perspective decoupling Transformer model is provided, including:

Obtaining the MRI image data of the user to be tested, and preprocessing the MRI image data;

Obtain slice data according to the preprocessed MRI image data and input it into the pre-trained lesion detection model;

Feature extraction is performed on the slice data to obtain high-level feature data;

The high-level feature data is fused at different scales from different perspectives to obtain high-level feature prediction data;

The lesion of the user to be tested is predicted by using the high-level feature prediction data.

In one embodiment, before the input into the pre-trained lesion detection model, it includes:

Collect multiple MRI sample image data, and mark the lesion area of the cross-sectional slice of each MRI sample image data in the form of a bounding box;

Preprocessing the collected MRI sample image data;

Create an original lesion detection model, convert the preprocessed MRI sample image data into sample slice data, input them into the original lesion detection model, and perform iterative learning according to the labels until the pre-trained lesion detection model is obtained .

In one embodiment, the performing feature extraction on the slice data to obtain high-level feature data includes:

dividing the slice data into a plurality of small blocks, the small blocks are single-channel images;

Embedding the single-channel image into a target channel to obtain an image with a target number of channels;

Acquire the high-level feature data according to the image of the target channel number.

In one embodiment, the extracted feature data are fused at different scales through different perspectives to obtain high-level feature prediction data, including:

dividing the high-level feature data into a plurality of small windows, and the small windows include M×M small blocks;

perform self-attention computations in each of the small windows to obtain contextual connections;

According to the context connection, the high-level feature data is fused through different perspectives to obtain high-level feature prediction data.

In one embodiment, the performing self-attention calculation in each of the moving windows to obtain contextual connections includes:

Offset each of the small windows by M/2 block areas to the target direction to form a boundary area;

supplementing the boundary area to the original area to form a plurality of new small windows;

Self-attention computations are performed in the new widget to obtain contextual connections.

In one embodiment, after converting the preprocessed MRI sample image data into sample slice data and inputting it into the original lesion detection model, the method includes:

Extract a preset number of candidate regions through the region generation network;

Calculate the intersection ratio between the candidate area and the real area;

According to the difference between the intersection ratio and a preset intersection ratio threshold, the candidate area is divided into positive samples and negative samples, and the positive samples and the negative samples are sampled, so that the The ratio between the positive samples and the negative samples is configured as a preset ratio;

The candidate regions that meet the preset ratio are sent to the pooling layer for pooling processing, and lesion prediction is performed.

In one embodiment, obtaining slice data according to the preprocessed MRI image data and inputting it into a pre-trained lesion detection model includes:

Perform cross-sectional slice processing on the preprocessed MRI image data from different viewing angles to obtain multiple slices;

Selecting a highly continuous slice sequence from the plurality of slices and inputting it into the pre-trained lesion detection model; or

A slice is selected from the plurality of slices and input into the pre-trained lesion detection model.

In a second aspect, a lesion detection device based on a perspective decoupling Transformer model is provided, including:

an image data acquisition module, used for acquiring the MRI image data of the user to be tested, and preprocessing the MRI image data;

The slice data acquisition module is used to acquire slice data according to the preprocessed MRI image data, and input it into the pre-trained lesion detection model;

a feature extraction module, used to perform feature extraction on the slice data to obtain high-level feature data;

A perspective decoupling module, used to fuse the high-level feature data at different scales through different perspectives to obtain high-level feature prediction data;

A prediction module, configured to predict the lesion of the user to be tested by using the high-level feature prediction data.

In a third aspect, a computer device is provided, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, when the processor executes the computer-readable instructions The steps of implementing the method for lesion detection based on the perspective decoupling Transformer model as described above.

In a fourth aspect, one or more readable storage media are provided, where the readable storage media store computer-readable instructions, and when the computer-readable instructions are executed by the processor, implement the above-mentioned perspective-based decoupling Transformer model. The steps of the lesion detection method.

The above-mentioned method, device, device and storage medium for lesion detection based on perspective decoupling Transformer model, the method implementation includes: acquiring MRI image data of a user to be tested, and preprocessing the MRI image data; MRI image data, obtain slice data, and input it into a pre-trained lesion detection model; perform feature extraction on the slice data to obtain multiple feature data of different scales; The data are fused to obtain high-level feature data; the lesions of the user to be tested are predicted through the high-level feature data. In the present application, feature extraction is performed on slice data, and the extracted feature data is fused at different scales through different perspectives, and the context information of the input slice is fused and enhanced, thereby realizing multi-scale feature prediction and improving the localization accuracy of lesions. , has a strong identification effect on small tumors, avoids misses or confusions in the early stage of tumors, and effectively improves the detection accuracy.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the drawings that are used in the description of the embodiments of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention. , for those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

1 is a schematic flowchart of a lesion detection method based on a perspective decoupling Transformer model according to an embodiment of the present invention;

2 is a schematic flowchart of extracting high-level feature data in an embodiment of the present invention;

FIG. 3 is a schematic diagram of an implementation scenario of viewing angle coupling from different viewing angles according to an embodiment of the present invention;

4 is a schematic structural diagram of a lesion detection device based on a perspective decoupling Transformer model according to an embodiment of the present invention;

5 is a schematic structural diagram of a lesion detection device based on a perspective decoupling Transformer model according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of a computer device in an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

In one embodiment, as shown in FIG. 1 , a method for detecting lesions based on a perspective decoupling Transformer model is provided, including the following steps:

In step S110, the MRI image data of the user to be tested is acquired, and the MRI image data is preprocessed;

In this embodiment of the present application, an MRI (Magnetic Resonance Imaging) of the user may be acquired through a nuclear magnetic resonance device, which is a nuclear magnetic resonance imaging.

In this embodiment of the present application, the acquired MRI image data is preprocessed, including aligning voxels or pixel spacing, data normalization, and cropping and scaling the MRI data to a pixel size of 512×512.

In step S120, obtain slice data according to the preprocessed MRI image data, and input it into the pre-trained lesion detection model;

In this embodiment of the present application, after obtaining the preprocessed MRI image data, the MRI image data can be sliced from different perspectives through image segmentation to obtain the slice data. For example, the quantile can be obtained. The segmentation method performs slice processing on the MRI image data.

In this embodiment of the present application, the slice data may be a slice sequence or a single slice, wherein, taking T slices as an example, the slice sequence may be consecutive T slices.

In step S130, feature extraction is performed on the slice data to obtain high-level feature data; in this embodiment of the present application, a Swin-Transformer network combined with a feature pyramid (feature pyramid network, FPN) structure can be used as the extraction network, when the slice data is input When , the slice can be divided into multiple small blocks through the block division layer, and the divided small blocks can be embedded into the C channel through the linear embedding layer, and the divided small blocks can be embedded in the C channel through the moving window (swin-transformer block) and block merging. After layer processing, high-level feature maps can be output, and then, through convolution calculations and upsampling operations, high-level feature data of the same channel fused at different scales can be obtained.

The high-level feature data may be a feature matrix with high abstraction and rich expression ability.

Wherein, the small block can be a single-channel image, and the C channel can be a vector whose constant is C. By embedding the segmented small block into the C channel, an image with the number of C channels can be obtained.

In the embodiment of the present application, the divided small blocks may be processed by moving the window and the block merging layer for many times, so as to obtain the required high-level feature data, wherein the block merging layer may be a down-sampling operation, and each down-sampling is performed. operation, the number of channels can be doubled for the divided small blocks, which can effectively reduce the resolution of high-level feature data.

In step S140, the feature data is fused at different scales from different perspectives to obtain high-level feature prediction data;

In this embodiment of the present application, the MRI image data may be a three-dimensional image, where W represents the X axis, T represents the Y axis, and H represents the Z axis. The above-mentioned different viewing angles may respectively represent different viewing angles of the three-dimensional image, for example, a T×W viewing angle, a T×H viewing angle, and an H×W viewing angle.

In the embodiment of the present application, the extracted high-level feature data can be fused at different scales through different perspectives through the perspective decoupling Transformer, and the slice data includes a sequence of T slices as an example for illustration. Specifically, the high-level feature data may include Multiple feature data of different scales from a sequence of T slices, respectively, in different perspectives, the attention operation is calculated by moving the window, that is, the input high-level feature data is divided into T slices from different perspectives, and then the window can be moved by moving the window. Divide the divided slices into multiple small windows, and perform self-attention calculation inside each small window, that is, perform self-attention calculation on each window on each slice, so as to obtain the context on the 3D image Contact, and use the sliding feature of the moving window to fuse the divided T slices, and then take the middle slice, that is, T/2 slices, as the final high-level feature prediction data for prediction.

In step S150, the lesion of the user to be tested is predicted by using the high-level feature prediction data.

In the embodiment of the present application, a cascaded RCNN (regional convolutional neural network) model is used to predict the lesions of the user to be tested by using the input high-level feature prediction data. Specifically, the cascaded RCNN model has multiple detection models (FasterRCNN). ), each detection model is trained based on positive and negative samples with different parallel-intersection ratio thresholds, the output of the previous detection model can be used as the input of the latter detection model, and the parallel-intersection ratio threshold of the positive and negative samples can be used with the detection. The order of the models goes up continuously.

In the embodiment of the present application, through the cascaded RCNN model, a preset number of candidate regions, such as 300, can be selected from the input high-level feature prediction data, and the candidate regions can be input into the ROI pooling layer for pooling Then, category classification prediction and bounding box regression prediction are performed to determine whether the user to be tested has lesions.

An embodiment of the present application provides a method for detecting a lesion based on a perspective decoupling Transformer model, including: acquiring MRI image data of a user to be tested, and preprocessing the MRI image data; according to the preprocessed MRI image data, Obtain slice data and input it into a pre-trained lesion detection model; perform feature extraction on the slice data to obtain multiple feature data of different scales; fuse the feature data on different scales through different perspectives, to obtain high-level feature data; predict the lesion of the user to be tested by using the high-level feature data. In this application, feature extraction is performed on slice data, and the extracted feature data is fused at different scales through different perspectives, and the context information of the input slice is fused and enhanced, so as to realize multi-scale feature prediction and improve the localization accuracy of lesions. , has a strong identification effect on small tumors, avoids misses or confusions in the early stage of tumors, and effectively improves the detection accuracy. The method can mark the position of the suspected tumor in the MRI two-dimensional slice when the user takes the MRI image, so as to help the doctor to locate and diagnose quickly.

In one embodiment, the present application also provides an implementation process of a method for detecting a lesion based on a perspective decoupling Transformer model, including the following steps:

In step S110, the MRI image data of the user to be tested is acquired, and the MRI image data is preprocessed;

In this embodiment of the present application, an MRI (Magnetic Resonance Imaging, magnetic resonance imaging, magnetic resonance imaging) of the user may be acquired through a nuclear magnetic resonance device.

In this embodiment of the present application, the acquired MRI image data is preprocessed, including aligning voxels or pixel spacing, data normalization, and cropping and scaling the MRI data to a pixel size of 512×512.

In step S120, obtain slice data according to the preprocessed MRI image data, and input it into the pre-trained lesion detection model;

In an embodiment of the present application, obtaining slice data according to preprocessed MRI image data and inputting it into a pre-trained lesion detection model includes:

Perform cross-sectional slice processing on the preprocessed MRI image data from different viewing angles to obtain multiple slices;

Selecting a highly continuous slice sequence from the plurality of slices and inputting it into the pre-trained lesion detection model; or

A slice is selected from the plurality of slices and input into the pre-trained lesion detection model.

In this embodiment of the present application, after obtaining the preprocessed MRI image data, the MRI image data can be sliced from different perspectives through image segmentation to obtain the slice data. For example, the quantile can be obtained. The segmentation method slices the MRI image data.

In the embodiment of the present application, it may be a slice sequence or a single slice, wherein, taking T slices as an example, the highly continuous slice sequence may be a continuous group of T slices. T consecutive slices of the same MRI image data represent the information of the middle slice T/2, and the edge can be supplemented by slices with 0 pixels.

In this embodiment of the present application, the selected slice or slice sequence may be a slice including an imaging region.

In an embodiment of the present application, before the slice data is acquired according to the preprocessed MRI image data and input into the pre-trained lesion detection model, the method includes:

Collect multiple MRI sample image data, and mark the lesion area of the cross-sectional slice of each MRI sample image data in the form of a bounding box;

Preprocessing the collected MRI sample image data;

Create an original lesion detection model, convert the preprocessed MRI sample image data into sample slice data, input them into the original lesion detection model, and perform iterative learning according to the labels until the pre-trained lesion detection model is obtained

In this embodiment of the present application, the MRI sample image data may be an MRI image of a patient, for example, a brain MRI image.

In the embodiment of the present application, the collected MRI sample image data is preprocessed, including alignment of voxel/pixel spacing, data normalization, and cropping and scaling to a pixel size of 512×512.

In the embodiment of the present application, an original lesion detection model is created, the preprocessed sample slice data is input into the original lesion detection model, and after feature extraction and perspective decoupling processing are performed on the sample slice data through the original lesion detection model, The lesions are predicted by a cascaded RCNN (regional convolutional neural network) model,

In an embodiment of the present application, after converting the preprocessed MRI sample image data into sample slice data and inputting it into the original lesion detection model, the method includes:

Extract a preset number of candidate regions through the region generation network;

Calculate the intersection ratio between the candidate area and the real area;

According to the difference between the intersection ratio and a preset intersection ratio threshold, the candidate region is divided into positive samples and negative samples, and the positive samples and the negative samples are sampled, so that the The ratio between the positive samples and the negative samples is configured as a preset ratio;

The candidate regions that meet the preset ratio are sent to the pooling layer for pooling processing, and lesion prediction is performed.

Specifically, a preset number of candidate regions, such as about 2000, are screened out through the regional convolutional neural network, and the candidate regions are input into the subsequent regional convolutional neural network structure, and the difference between each candidate region and the real region is calculated. According to the difference between the intersection ratio and the preset intersection ratio threshold, the candidate area is divided into positive samples (foreground) and negative samples (background), and the positive and negative samples are sampled so that The ratio between positive and negative samples conforms to a preset ratio, such as 1:3 (the total number of positive and negative samples can be 128), and then the sampled candidate regions are input into the Roi pooling layer for class classification prediction and bounding boxes Regression prediction and iterative learning are performed until a trained lesion detection model is obtained.

The real area is the real lesion area confirmed by the doctor.

In step S130, feature extraction is performed on the slice data to obtain high-level feature data; in this embodiment of the present application, a Swin-Transformer network combined with a feature pyramid network (FPN) structure may be used as the extraction network.

In an embodiment of the present application, feature extraction is performed on the slice data to obtain high-level feature data, including:

dividing the slice data into a plurality of small blocks, the small blocks are single-channel images;

Embedding the single-channel image into a target channel to obtain an image with a target number of channels;

Acquire the high-level feature data according to the image of the target channel number.

Referring to Figure 2, input slice data, the size of the slice data can be 1×1×H×W×T, the slice is subjected to block division processing through the block division layer to divide it into H/4×W/4 small blocks, Each small block is a 4×4×T single-channel image, and it is embedded into the target channel through the linear embedding layer, for example, in the C channel, and the image of C×H/4×W/4×T is obtained, and then the obtained A series of high-level feature maps can be obtained after the C×H/4×W/4×T images are processed by multiple swin-transformer blocks and block merging layers.

Specifically, in an implementation scenario, taking four moving window modules as an example, after the C×H/4×W/4×T feature map is processed by the first moving window module, a 1×1 convolution calculation is performed, After the feature map is added, the 3×3 convolution calculation is performed to obtain the fused C×H/4×W/4×T feature map; at the same time, the first moving window module can process the C×H/4×W/4×T feature map. The H/4×W/4×T feature map is sent to the first merging layer for downsampling to form a 2C×H/4×W/4×T feature map, which is then input to the second moving window module for processing. The second moving window module performs 1×1 convolution calculation on the 2C×H/4×W/4×T feature map, performs feature map addition processing, and performs 3×3 convolution calculation, and then upsampling by 2 times. , fused with the C×H/4×W/4×T feature map calculated by the 1×1 convolution of the first moving window module, and after performing the 3×3 convolution calculation again, the fused C×H/ 8×W/8×T feature map; at the same time, the second moving window module also inputs the processed 2C×H/4×W/4×T feature map to the second merging layer for downsampling operation, forming 4C× The H/4×W/4×T feature map is input to the third moving window module for processing, and the third moving window module performs 1×1 convolution on the 4C×H/4×W/4×T feature map Calculate, perform feature map addition processing, and perform 3×3 convolution calculation, and then upsampling by 2 times, that is, 2C×H/4×W/4× calculated by 1×1 convolution with the second moving window module After the T feature map is fused, and the 3×3 convolution calculation is performed again, the fused C×H/16×W/16×T feature map can be obtained; at the same time, the third moving window will also process the processed 4C×H The /4×W/4×T feature map is sent to the third merging layer for downsampling to form an 8C×H/4×W/4×T feature map, and then sent to the fourth moving window for processing; the fourth moving window After the 8C×H/4×W/4×T feature map is processed, 1×1 convolution calculation is performed, and the upper and lower sampling is performed 2 times, respectively, and the 3×3 convolution calculation is performed to form C×H/32× W/32×T feature map and C×H/64×W/64×T feature map.

It can be seen that after multiple 1x1 convolution and upsampling operations, the high-level feature maps of the same channel fused at different scales can be obtained, that is, C×H/4×W/4×T, C×H/8 ×W/8×T, C×H/16×W/16×T, C×H/32×W/32×T, and C×H/64×W/64×T.

Wherein, the small block can be a single-channel image, and the C channel, that is, a vector whose constant is C, by embedding the segmented small block into the C channel, an image with the number of C channels can be obtained.

In this embodiment of the present application, the divided small blocks can be processed by moving the window and the block merging layer for many times, so as to obtain the required high-level feature data, wherein the block merging layer can be a down-sampling operation, and each down-sampling is performed. operation, the number of channels can be doubled for the divided small blocks, which can effectively reduce the resolution of high-level feature data.

In step S140, the high-level feature data is fused at different scales from different perspectives to obtain high-level feature prediction data;

In this embodiment of the present application, the MRI image data may be a three-dimensional image, and W may be used to represent the X axis, T to represent the Y axis, and H to represent the Z axis. The above-mentioned different viewing angles may respectively represent different viewing angles of the three-dimensional image, for example, a T×W viewing angle, a T×H viewing angle, and an H×W viewing angle.

In an embodiment of the present application, the feature data is fused at different scales from different perspectives to obtain high-level feature prediction data, including:

dividing the high-level feature data into a plurality of small windows, and the small windows include M×M small blocks;

perform self-attention computations in each of the small windows to obtain contextual connections;

And according to the context connection, the high-level feature data of different scales are fused through different perspectives to obtain high-level feature prediction data.

In an embodiment of the present application, the high-level feature data includes multiple high-level feature maps of the same channel fused at different scales, for example, C×H/4×W/4×T, C×H/8×W/8× T, C×H/16×W/16×T, C×H/32×W/32×T, C×H/64×W/64×T, etc.

该

为嵌入的向量，对分数施以softmax激活函数后点成值向量，得到加权的每个输入的嵌入向量的评分，最后将评分相加得到最终的输出结果Z。公式如式如下所述：In an embodiment of the present application, the perspective decoupling Transformer model first divides the extracted high-level feature map into multiple small windows through the moving window module, and each small window contains M×M, which are divided into layers by blocks. The small blocks are divided, and self-attention calculation is performed in each window. Specifically, three different vectors are allocated to each small block in the window through the attention mechanism, namely, the query vector (Q), key-value vector (K) and value vector (V), compute a score for each vector, the score Q K ^T , then normalize the score and divide by

Should

For the embedded vector, apply the softmax activation function to the score and then point it into a value vector to obtain the weighted score of each input embedded vector, and finally add the scores to obtain the final output result Z. The formula is as follows:

Wherein, the query vector (Q), the key value vector (K) and the value vector (V) can be obtained by multiplying a different weight matrix and an embedded vector.

Wherein, the moving window module includes a window attention module and a moving window attention module.

In an embodiment of the present application, performing self-attention calculation in each of the small windows to obtain contextual connections includes:

Offset each of the small windows by M/2 block areas to the target direction to form a boundary area;

supplementing the boundary area to the original area to form a plurality of new small windows;

Self-attention computations are performed in the new widget to obtain contextual connections.

In this embodiment of the present application, since information transmission cannot be performed between the originally divided small windows, the small windows can be shifted to the target direction respectively, for example, each small window can be shifted to the right and down respectively M/2 block areas, at this time, an offset area is formed on the right and below, and the offset area is backfilled to the left and above, that is, back to the original area, and multiple new small windows can be formed. The self-attention calculation is performed in the new small window to obtain the correlation between the plane pixels, thereby obtaining the contextual connection.

Wherein, the offset area is an area from which the original area is offset.

In the embodiment of the present application, referring to FIG. 4 , the perspective decoupling Transformer model in different perspectives, such as the T×W perspective, the T×H perspective, and the H×W perspective, respectively uses the moving window module to segment the input high-level feature map into Multiple small windows, and then calculate the self-attention mechanism separately through the divided small windows, the direct connection of the three-dimensional context can be obtained, and the correlation of the two-dimensional plane pixels can be obtained. At this time, the sliding feature of the moving window can be used to T slices are fused, and finally the middle slice, that is, the T/2 slice, can be taken as the final high-level feature prediction data for prediction.

Among them, arrows F represent pairwise correlations in 2D planar pixels of different viewing angles. This correlation can be determined by the self-attention mechanism described above.

In step S150, the lesion of the user to be tested is predicted by using the high-level feature prediction data.

In the embodiment of the present application, a cascaded RCNN (regional convolutional neural network) model is used to predict the lesions of the user to be tested by using the input high-level feature prediction data. Specifically, the cascaded RCNN model has multiple detection models (FasterRCNN). ), each detection model is trained based on positive and negative samples with different parallel-intersection ratio thresholds, the output of the previous detection model can be used as the input of the latter detection model, and the parallel-intersection ratio threshold of the positive and negative samples can be used with the detection. The order of the models continues to rise.

In the embodiment of the present application, through the cascaded RCNN model, a preset number of candidate regions, such as 300, can be selected from the input high-level feature prediction data, and the candidate regions can be input into the ROI pooling layer for pooling Then, category classification prediction and bounding box regression prediction are performed to determine whether the user to be tested has lesions.

In this embodiment of the present application, feature extraction is performed on slice data, and the extracted feature data is fused at different scales from different perspectives, and the context information of the input slice is fused and enhanced, thereby realizing multi-scale feature prediction and improving the localization of lesions. It has a strong identification effect on small tumors, avoids misses or confusions in the early stage of tumors, and effectively improves the detection accuracy.

It should be understood that the size of the sequence numbers of the steps in the above embodiments does not mean the sequence of execution, and the execution sequence of each process should be determined by its functions and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

In one embodiment, a lesion detection device based on a perspective decoupling Transformer model is provided, and the lesion detection device based on a perspective decoupling Transformer model corresponds to the lesion detection method based on the perspective decoupling Transformer model in the above embodiment. As shown in FIG. 4 , the lesion detection device based on the perspective decoupling Transformer model includes: an image data acquisition module 10 , a slice data acquisition module 20 , a feature extraction module 30 , a perspective decoupling module 40 and a prediction module 50 . The detailed description of each functional module is as follows:

The image data acquisition module 10 is used for acquiring the MRI image data of the user to be tested, and preprocessing the MRI image data;

The slice data acquisition module 20 is configured to acquire slice data according to the preprocessed MRI image data, and input it into the pre-trained lesion detection model;

A feature extraction module 30, configured to perform feature extraction on the slice data to obtain high-level feature data;

A perspective decoupling module 40, configured to fuse the high-level feature data at different scales through different perspectives to obtain high-level feature prediction data;

The prediction module 50 is configured to predict the lesion of the user to be tested by using the high-level feature prediction data.

Referring to FIG. 5 , in an embodiment of the present application, the lesion detection device based on the perspective decoupling Transformer model further includes: a pre-trained lesion detection model generation module 60 for

Collect multiple MRI sample image data, and mark the lesion area of the cross-sectional slice of each MRI sample image data in the form of a bounding box;

Preprocessing the collected MRI sample image data;

Create an original lesion detection model, convert the preprocessed MRI sample image data into sample slice data, and input them into the original lesion detection model, and perform iterative learning according to the markers until the pre-trained lesions are obtained Check the model.

In one embodiment, the feature extraction module 30 is further configured to:

dividing the slice data into a plurality of small blocks, the small blocks are single-channel images;

Embedding the single-channel image into a target channel to obtain an image with a target number of channels;

Acquire the high-level feature data according to the image of the target channel number.

In one embodiment, the viewing angle decoupling module 40 is further configured to:

dividing the high-level feature data into a plurality of small windows, and the small windows include M×M small blocks;

perform self-attention computations in each of the small windows to obtain contextual connections;

According to the context connection, the high-level feature data is fused through different perspectives to obtain high-level feature prediction data.

In one embodiment, the viewing angle decoupling module 40 is further configured to:

Offset each of the small windows by M/2 block areas in the target direction to form an offset area;

supplementing the offset area to the original area to form a plurality of new small windows;

Self-attention computations are performed in the new widget to obtain contextual connections.

In an embodiment, the pre-trained lesion detection model generation module 60 is further used for:

Extract a preset number of candidate regions through the region generation network;

Calculate the intersection ratio between the candidate area and the real area;

According to the difference between the intersection ratio and a preset intersection ratio threshold, the candidate region is divided into positive samples and negative samples, and the positive samples and the negative samples are sampled, so that the The ratio between the positive samples and the negative samples is configured as a preset ratio;

The candidate regions that meet the preset ratio are sent to the pooling layer for pooling processing, and lesion prediction is performed.

In one embodiment, the slice data acquisition module 20 is further configured to:

Perform cross-sectional slice processing on the preprocessed MRI image data from different viewing angles to obtain multiple slices;

Selecting a highly continuous slice sequence from the plurality of slices and inputting it into the pre-trained lesion detection model; or

A slice is selected from the plurality of slices and input into the pre-trained lesion detection model.

In this embodiment of the present application, feature extraction is performed on slice data, and the extracted feature data is fused at different scales from different perspectives, and the context information of the input slice is fused and enhanced, thereby realizing multi-scale feature prediction and improving the localization of lesions. It has a strong identification effect on small tumors, avoids misses or confusions in the early stage of tumors, and effectively improves the detection accuracy.

For the specific limitation of the lesion detection device based on the perspective decoupling Transformer model, please refer to the limitation of the lesion detection method based on the perspective decoupling Transformer model above, which will not be repeated here. Each module in the above-mentioned device for lesion detection based on the perspective decoupling Transformer model may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

In one embodiment, a computer device is provided, the computer device may be a server, and its internal structure diagram may be as shown in FIG. 6 . The computer device includes a processor, memory, and a network interface connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium. The readable storage medium stores computer-readable instructions. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer-readable instructions, when executed by the processor, implement a method for lesion detection based on a perspective decoupling Transformer model. The readable storage medium provided by this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

A computer device, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, characterized in that, when the processor executes the computer-readable instructions, the above-mentioned method of decoupling the Transformer model based on perspective is implemented. The steps of the lesion detection method.

One or more readable storage media, where computer-readable instructions are stored in the readable storage media, wherein when the computer-readable instructions are executed by the processor, the steps of the method for lesion detection based on the perspective decoupling Transformer model as described above are implemented.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile computer. In the read storage medium or the volatile readable storage medium, the computer-readable instructions, when executed, may include the processes of the foregoing method embodiments. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided in this application may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. Module completion, that is, dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it is still possible to implement the foregoing implementations. The technical solutions described in the examples are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the within the protection scope of the present invention.

Claims (10)

1. A focus detection method based on a visual angle decoupling Transformer model is characterized by comprising the following steps:

acquiring MRI image data of a user to be detected, and preprocessing the MRI image data;

acquiring slice data according to the preprocessed MRI image data, and inputting the slice data into a pre-trained focus detection model;

performing feature extraction on the slice data to acquire high-level feature data;

fusing the high-level feature data on different scales through different visual angles to obtain high-level feature prediction data;

and predicting the focus of the user to be detected according to the high-level feature prediction data.

2. The method for visual-angle-based decoupled transform model lesion detection of claim 1, wherein the prior input into the pre-trained lesion detection model comprises:

acquiring a plurality of MRI sample image data and marking a focal region of a cross-sectional slice of each of the MRI sample image data in the form of a bounding box;

preprocessing the acquired MRI sample image data;

and creating an original focus detection model, converting the preprocessed MRI sample image data into sample slice data, inputting the sample slice data into the original focus detection model, and performing iterative learning according to the mark until the pre-trained focus detection model is obtained.

3. The visual-angle decoupling Transformer model-based lesion detection method of claim 1, wherein the feature extraction is performed on the slice data to obtain high-level feature data, comprising

Dividing the slice data into a plurality of small blocks, wherein the small blocks are single-channel images;

embedding the single-channel image into a target channel to obtain images of the number of the target channels;

and acquiring the high-level feature data according to the image with the target channel number.

4. The method for lesion detection based on perspective decoupling transform model of claim 3, wherein the fusing the high-level feature data at different scales through different perspectives to obtain high-level feature prediction data comprises:

dividing the high-level feature data into a plurality of small windows, wherein the small windows comprise M multiplied by M small blocks;

performing self-attention calculation in each small window to obtain context connection;

and according to the context relation, fusing the high-level feature data on different scales through different visual angles to obtain the high-level feature prediction data.

5. The view-decoupled lesion detection method of claim 4, wherein said performing a self-attention calculation in each of said small windows to obtain contextual connections comprises:

shifting each small window to a target direction by M/2 block areas respectively to form a shift area;

supplementing the offset area to the original area to form a plurality of new small windows;

performing a self-attention calculation in the new widget to obtain contextual connectivity.

6. The method for lesion detection based on view-angle decoupling Transformer model of claim 2, wherein the step of converting the pre-processed MRI sample image data into sample slice data, after inputting the sample slice data into the original lesion detection model, comprises:

extracting a preset number of candidate regions through a region generation network;

calculating the intersection ratio between the candidate region and the real region;

dividing the candidate region into a positive sample and a negative sample according to a difference value between the intersection ratio and a preset intersection ratio threshold, and sampling the positive sample and the negative sample to configure the proportion between the positive sample and the negative sample as a preset proportion;

and sending the candidate region according with the preset proportion to a pooling layer for pooling treatment, and performing focus prediction.

7. The visual angle decoupling Transformer model-based lesion detection method according to any one of claims 1 to 6, wherein the acquiring slice data from the preprocessed MRI image data and inputting the slice data into a pre-trained lesion detection model comprises:

performing cross-section slice processing on the preprocessed MRI image data from different visual angles to acquire a plurality of slices;

selecting a highly continuous slice sequence from the multiple slices and inputting the highly continuous slice sequence into the pre-trained lesion detection model; or,

and selecting one slice from the plurality of slices and inputting the selected slice into the pre-trained lesion detection model.

8. A focus detection device based on a visual angle decoupling Transformer model is characterized by comprising:

the image data acquisition module is used for acquiring MRI image data of a user to be detected and preprocessing the MRI image data;

the slice data acquisition module is used for acquiring slice data according to the preprocessed MRI image data and inputting the slice data into a pre-trained focus detection model;

the characteristic extraction module is used for extracting the characteristics of the slice data to obtain high-level characteristic data;

the visual angle decoupling module is used for fusing the high-level feature data on different scales through different visual angles so as to obtain high-level feature prediction data;

and the prediction module is used for predicting the focus of the user to be detected through the high-level feature prediction data.

9. A computer device comprising a memory, a processor, and computer readable instructions stored in the memory and executable on the processor, wherein the processor when executing the computer readable instructions implements the steps of the visual decoupling transform model based lesion detection method of any one of claims 1 to 7.

10. One or more readable storage media storing computer readable instructions, wherein the computer readable instructions, when executed by a processor, implement the steps of the method for lesion detection based on a view-decoupled transform model according to any of claims 1 to 7.

Images