KR20240086365A - Multi-agent reinforcement learning based digital tomosynthesis image denoising system using data transformation - Google Patents

Abstract

The present invention applies a multi-agent reinforcement learning network that introduces convGRU (convolutional gated recurrent unit) and RMC (Reward Map Convolution) to generate a noise removal model for digital tomosynthesis (DT) images, and applies data transformation to the multi-agent reinforcement learning network. This is about a digital tomosynthesis image noise removal system based on agent reinforcement learning.

Description

Multi-agent reinforcement learning based digital tomosynthesis image denoising system using data transformation}

The present invention is a technology related to digital tomosynthesis image noise removal. In detail, the present invention is a technology for removing noise from digital tomosynthesis (DT) images. In detail, the present invention is a technology for removing noise from digital tomosynthesis (DT) images by introducing a convolutional gated recurrent unit (convGRU) and reward map convolution (RMC). This is about a multi-agent reinforcement learning-based digital tomosynthesis image noise removal system that applies an agent reinforcement learning network and data transformation.

Digital tomosynthesis (DT) acquires three-dimensional images by scanning images from limited angles and reconstructing them using algorithms similar to computed tomography (CT) or magnetic resonance (MR) imaging. .

This DT has a lower exposure dose than CT that provides the same 3D image, so it is used in areas sensitive to dose, such as the breast. Additionally, compared to simple X-ray imaging, there is less overlap of structures, so it has the advantage of being able to see structures and lesions more clearly. .

However, DT images have poor image quality due to limited scanning angles and insufficient projection images, making accurate diagnosis difficult.

To solve these shortcomings, research is actively being conducted to improve the quality of DT images. However, in the case of supervised learning with existing methods, a large amount of data and a long time are required to train the model, and supervised learning for neural networks has difficulty minimizing the loss function due to overfitting, and requires a large amount of data and a long time to train the model. There are problems such as distortion.

In contrast, reinforcement learning (RL) does not require a large amount of label data when learning a network, and applying deep learning to RL can accelerate the learning speed. Additionally, if the pixel and patch units of the input image are used as a multi-agent, subtle changes in features and pixel values of various regions can be reflected in the output image.

In particular, multi-agent reinforcement learning (MARL) is a method of extracting the state of the input image and selecting an action that can maximize the reward in the current state. The action to be performed by the model is determined in advance. Since it can be defined, it is possible to obtain an output image optimized for the purpose.

Republic of Korea Patent Publication No. 10-2022-0086937 (2022.06.24)

The present invention was created in response to the above-mentioned needs, and the purpose of the present invention is to apply a multi-agent reinforcement learning network to generate a noise removal model for DT images and to use a convGRU (convolutional gated recurrent unit) and RMC (Reward Map Convolution) ) is to provide a digital tomosynthesis image noise removal system based on multi-agent reinforcement learning that applies data conversion in the form of introduced.

The present invention for the above purpose includes a learning data generator that generates a standard image and a plurality of training input images to which a set level of noise is applied to the standard image; A transform unit that applies wavelet transform and Anscomb transform to the training image; It is connected by a fully convolutional network method, a share network that extracts the features of the input image, a value network that calculates compensation by comparing the input image from which the features are extracted with the standard image and learns to maximize the compensation, and a dictionary definition. A learning model component that configures a multi-agent reinforcement learning model that removes noise from the input image, consisting of a sub-network of the policy network that determines an action appropriate for the state of the input image from the chart; a learning unit that inputs the standard image and a plurality of input images into the multi-agent reinforcement learning model and trains the multi-agent reinforcement learning model to reduce the difference between the standard image and the input image; An inverse transformation unit that removes noise from the input digital tomosynthesis image by applying a trained multi-agent reinforcement learning model; It is characterized by consisting of.

At this time, the share network consists of one convolution layer and three extended convolution layers, the value network consists of two extended convolution layers and one convolution layer, and the policy network consists of two extended convolution layers. It is preferable that it consists of a layer and one convolution layer.

In addition, it is preferable that the expansion ratios of the extended convolutional layers in the shared network are set to 2, 3, and 4, respectively, and the movement range of the filter in all layers of the multi-agent reinforcement learning model is fixed to 1.

Additionally, in the value network, it is desirable to calculate compensation through [Equation 3] and [Equation 4] below.

[Equation 3]

[Equation 4]

( is the reward of the kth agent at time step t, I _k is the pixel value of the standard image, is the state of the kth agent including image features, is the total reward calculated at the kth agent, γ is the attenuation coefficient, ω is the convolution filter weight of the pth agent, F(k) is the center of the receptive field at the kth agent)

In addition, it is preferable that the expansion ratios of the expanded convolutional layer of the policy network are set to 3 and 2, respectively, and then a convGRU (convolutional gated recurrent unit) is applied.

Additionally, it is desirable for the policy network to output appropriate actions through [Equation 5].

[Equation 5]

(π _k is the optimal policy for applying action α _k to state s _k )

Through the present invention, structures and lesions can be clearly diagnosed by effectively removing noise from DT images, which had poor image quality due to limited scanning angles and insufficient projection images, and greatly reducing the amount of radiation exposure that was unavoidable for high-quality images such as conventional CT. It can be reduced.

1 is a block diagram of a digital tomosynthesis image noise removal system according to an embodiment of the present invention;
Figure 2 is a structural diagram of the MARL model for noise removal of DT images according to the present invention;
3 is a flowchart of a digital tomosynthesis image noise removal method according to an embodiment of the present invention;
Figure 4 is a DT image in which noise was removed using the existing MARL model and MARL to which the data transformation of the present invention was applied;
Figure 5 is a graph showing the SNR measurement results for the input image and the output image of each model according to an embodiment of the present invention.

Hereinafter, a multi-agent reinforcement learning-based digital tomosynthesis image noise removal system using the data conversion of the present invention will be described in detail with reference to the attached drawings.

In the present invention, a multi-agent reinforcement learning network is introduced to generate a noise removal model for DT images. If the contrast between internal structures in the DT image is insufficient, it is difficult for the network to separate noise patterns and prevents accurate recognition of noise features during network training. Accordingly, in the present invention, the noise characteristics of DT images are selectively transmitted in a multi-agent reinforcement learning network and wavelet and Anscombe transforms are applied to improve training accuracy.

Afterwards, compared to the existing learning method in which the standard image and input image are fixed, the input image is changed through a multi-agent reinforcement learning model and learning is performed. Through this effect, the insufficient amount of learning data is sufficiently secured to enable model learning.

Figure 1 is a block diagram of a digital tomosynthesis image noise removal system according to an embodiment of the present invention. The main components of the present invention are a learning data generation unit 110, a conversion unit 120, and a learning model constructor 130. ), and a learning unit 140 and an inverse conversion unit 150.

The learning data generator 110 is configured to generate a standard image and a plurality of input images for training by applying a set level of noise to the standard image.

The standard image is a high-resolution reference image, and the input image for training is a DT image with various noise levels. In the embodiment of the present invention, the input image is an image to which Gaussian noise with standard deviations of 0.05, 0.07, 0.10, 0.13, and 0.15 is added. It was created with .

The transform unit 120 is configured to apply wavelet transform and Anscombe transform to the training image, allowing noise characteristics to be accurately recognized during network learning.

Wavelet transform is a transformation method that represents time and frequency components of a signal whose frequency components change over time. In general, the Fourier transform displays frequency components under the assumption that the signal does not change temporally, whereas the wavelet transform displays the frequency component of signals that change temporally, such as chirped signals, ECG (Electrocardiograph), and video signals. It is used to express time and frequency components. In this case, low frequency components are expressed with high frequency resolution, and high frequency components are converted to high time resolution.

In the present invention, a non-redundant subband image including direction information of high-frequency components can be generated through wavelet transformation as shown in [Equation 1], and noise components can be extracted from the complex pattern of the input DT image.

[Equation 1]

Here, M and N refer to the number of pixels on the x and y axes of the input image. is a filter for approximating the image signal (A) with conversion level j or separating the signal along the horizontal (H), vertical (V), and diagonal (D) directions.

The Anscombe transform is a variance stabilizing transformation that converts a random variable with a Poisson distribution into a variable with an approximate standard Gaussian distribution.

In the present invention, signal-dependent noise is converted into signal-independent noise through Anscombe transform as shown in Equation 2, and then the complexity of noise properties included in the signal is reduced.

[Equation 2]

here is a noise-free signal in the spatial domain, and n(x,y) is signal-independent noise.

The learning model component 130 is connected in a fully convolutional network manner, and learns to calculate compensation by comparing a share network that extracts features of the input image and a standard image from which the features are extracted, and maximizes the compensation. It consists of a sub-network of the value network, which determines the action appropriate for the state of the input image from a predefined chart, and the sub-network of the policy network, which forms a multi-agent reinforcement learning model that removes noise from the input image.

Figure 2 is a structural diagram of the MARL model for noise removal of DT images according to the present invention. The MARL model presented in the present invention consists of a total of three sub-networks, and each network is connected in a fully convolutional network.

The shared network consists of one convolution layer (Conv) and three dilated convolution layers (Dilated conv).

The extended convolution layer can quickly extract features of the input image and reduce computational costs by expanding the size of the receptive field. At this time, the expansion ratios of the expanded convolutional layers in the shared network were set to 2, 3, and 4, respectively, and the movement range of the filter in all layers of the model presented in the present invention was fixed to 1.

This shared network plays a role in extracting the state of the input image to which the action is applied, and the state extracted from the shared network is used as input to the value and policy network, respectively.

The value network consists of two extended convolution layers and one convolution layer. In the embodiment of the present invention, the expansion ratio of the value network's expansion convolution was set to 3 and 2.

This value network calculates compensation by comparing the state of the input image extracted from the shared network with the standard image, and learns in a direction that maximizes the compensation, creating an output image very similar to the standard image.

The policy network consists of two expanded convolutional layers and one convolutional layer with expansion ratios of 3 and 2, respectively, and a convolutional gated recurrent unit (convGRU) is applied after the two expanded convolutional layers. . ConvGRU can improve learning accuracy by solving the time dependence problem where the importance of the initial state decreases as the learning time increases.

In other words, in the existing one-character learning method, data loss is prevented through ConvGRU, where the initial value may become blurred as the network deepens.

In this policy network, the most appropriate action is determined based on the current state of the input image.

[Table 1] A set of predefined actions used in the policy network. The tasks executed by the agent were selected from the predefined set listed in [Table 1].

[Table 1]

The predefined task set included existing image filters designed in the spatial domain and the existing image filters were determined empirically for noise removal. The optimal policy at episode t updates the state, and the updated state was used as input data for the shared subnetwork at episode t+1. The detailed configuration of the proposed multi-agent RL network is summarized in [Table 2].

[Table 2]

The learning unit 140 inputs the standard image and a plurality of input images into the multi-agent reinforcement learning model and trains the multi-agent reinforcement learning model to reduce the difference between the standard image and the input image.

The inverse transform unit 150 applies the multi-agent reinforcement learning model trained through the learning unit 140 to remove noise from the input digital tomosynthesis image.

Figure 3 is a flowchart of a digital tomosynthesis image noise removal method according to an embodiment of the present invention.

In the first step (S110), the input image to which data conversion has been applied to the noise image is input to the shared network.

In the second step (S120), the features of the input image extracted from the shared network are transferred to the value and policy network.

In the third step (S130), the value network outputs the current state compensation.

In the present invention, the value network outputs the current state compensation through a compensation calculation process such as [Equation 3] and [Equation 4].

[Equation 3]

[Equation 4]

( is the reward of the kth agent at time step t, I _k is the pixel value of the standard image, is the state of the kth agent including image features, is the total reward calculated at the kth agent, γ is the attenuation coefficient, ω is the convolution filter weight of the pth agent, F(k) is the center of the receptive field at the kth agent)

In this step, convolution was performed on the reward calculation in the Lth learning step to reduce computational cost.

In the fourth step (S140), the policy network outputs an appropriate action based on the reward output from the value network.

In the present invention, an appropriate action is output from the policy network based on the reward output from the value network, as shown in [Equation 5].

[Equation 5]

(π _k is the optimal policy for applying action α _k to state s _k )

The output policy is delivered to the agent to update its status.

In the fifth step (S150), the process is repeated as many times as set, and after learning is completed, the learned MARL is used to remove noise from the DT image. In other words, each step mentioned above is performed for the set number of repetitions, and when learning is completed, the finally learned MARL is used to remove noise from the DT image.

Figure 4 is a DT image from which noise was removed using the existing MARL model and MARL to which the data transformation of the present invention was applied, (a) a standard image, (b) an input image with noise added, and (c) an existing MARL model. (d) an image to which the wavelet transform was applied to MARL, (e) an image to which Anscombe transform was applied to MARL.

As can be seen in Figure 4, when MARL and data transformation are applied, it can be observed that noise is removed compared to the input image. The results of the quantitative comparative analysis performed to evaluate the degree of noise removal are shown in Figure 5.

Figure 5 is a graph showing the SNR measurement results for the input image and the output image of each model according to an embodiment of the present invention. When MARL with data conversion applied as the measurement result is used, the quantitative evaluation result is better than the output image of the existing MARL model. You can see improvement.

The rights of the present invention are not limited to the embodiments described above but are defined by the claims, and those skilled in the art can make various changes and modifications within the scope of the claims. This is self-evident.

110: Learning data generation unit 120: Conversion unit
130: Learning model configuration unit 140: Learning unit
150: Inverse conversion unit

Claims (6)

A learning data generator 110 that generates a standard image and a plurality of input images for training by applying noise of a set level to the standard image;
A transformation unit 120 that applies wavelet transformation and Anscomb transformation to the training image;
It is connected using a fully convolutional network method, a share network that extracts the features of the input image, a value network that calculates compensation by comparing the input image from which the features are extracted with the standard image and learns to maximize the compensation, and a pre-defined network. A learning model component 130 that configures a multi-agent reinforcement learning model that removes noise in the input image, consisting of a sub-network of the policy network that determines an action appropriate for the state of the input image from the chart.
a learning unit 140 that inputs the standard image and a plurality of input images into the multi-agent reinforcement learning model and trains the multi-agent reinforcement learning model to reduce the difference between the standard image and the input image;
An inverse transform unit 150 that removes noise from the input digital tomosynthesis image by applying a trained multi-agent reinforcement learning model; A digital tomosynthesis image noise removal system comprising:
According to paragraph 1,
The share network consists of one convolution layer and three extended convolution layers, the value network consists of two extended convolution layers and one convolution layer, and the policy network consists of two extended convolution layers. A digital tomosynthesis image noise removal system comprising: and one convolution layer.
According to paragraph 2,
In the shared network, the expansion ratios of the extended convolutional layers are set to 2, 3, and 4, respectively, and the moving range of the filter in all layers of the multi-agent reinforcement learning model is fixed to 1. Digital tomosynthesis image noise removal. system.
According to paragraph 2,
The digital tomosynthesis image noise removal system is characterized in that the value network calculates compensation through [Equation 3] and [Equation 4] below.
[Equation 3]

[Equation 4]

( is the reward of the kth agent at time step t, I _k is the pixel value of the standard image, is the state of the kth agent including image features, is the total reward calculated at the kth agent, γ is the attenuation coefficient, ω is the convolution filter weight of the pth agent, F(k) is the center of the receptive field at the kth agent)
According to paragraph 2,
The expansion ratio of the expansion convolution layer of the policy network is set to 3 and 2, respectively, and then a convGRU (convolutional gated recurrent unit) is applied. A digital tomosynthesis image noise removal system.
According to clause 4,
The policy network is a digital tomosynthesis image noise removal system characterized in that it outputs appropriate behavior through [Equation 5].
[Equation 5]

(π _k is the optimal policy for applying action α _k to state s _k )