JP7617724B2 - MODEL LEARNING METHOD, MODEL LEARNING SYSTEM, SERVER DEVICE, AND COMPU - Google Patents

Description

The present invention relates to a model learning method, a model learning system, a server device, and a computer program for the practical application of distributed learning.

Judgment, recognition, etc. using learning models based on deep learning have been put to practical use. It has been confirmed that learning models can be used in a variety of technical fields. Even if a huge amount of training data is used to improve the accuracy of the learning models to a level that makes them practical for use in various fields, it takes a long time for the parameters to converge, and there is no guarantee that the accuracy will improve.

Patent Document 1 discloses a method in which multiple replicas of a learning model to be learned are prepared, and these multiple model replicas learn independently and asynchronously. In Patent Document 1, a parameter server is fragmented into multiple parts, and each of the multiple replicas of the learning model asynchronously acquires parameters from the fragmented parameter servers, learns from them, and returns the parameters to each parameter server, repeating this process. This type of distributed processing is said to allow the parameters of the learning model to converge quickly.

U.S. Pat. No. 8,768,870

Even in the deep learning method using distributed processing as disclosed in Patent Document 1, training data is aggregated. However, data in fields such as medicine, finance, and authentication is personal data and is highly confidential. In order to aggregate data as training data to improve the accuracy of the model, each individual's consent is required to provide the data, and even if consent is obtained, there is always a risk to the security of data management.

The present invention has been made in consideration of the above circumstances, and aims to provide a model learning method, a model learning system, a server device, and a computer program that promote the practical application of learning models based on distributed learning.

A model learning method according to one embodiment of the present disclosure includes distributing global model data stored in a server to multiple nodes, acquiring local model data learned based on the global model using local data processed by the multiple nodes, and updating the global model data based on the multiple local model data acquired from each of the multiple nodes.

The model learning system of one embodiment of the present disclosure includes a plurality of nodes each processing local data, and a server connected to the plurality of nodes for communication, the server distributes global model data to the plurality of nodes, the plurality of nodes each proceeds with learning a local model from the global model using the local data to be processed, and the server acquires learned local model data from the plurality of nodes and updates the global model data based on the acquired data for the plurality of local models.

A server device according to an embodiment of the present disclosure includes a communication unit that communicates with a plurality of nodes and a processing unit for storing global model data, and distributes the global model data to the plurality of nodes using the processing unit, acquires from the communication unit local model data learned based on the global model data using local data processed by the plurality of nodes, and updates the global model data based on the plurality of local model data acquired from each of the plurality of nodes.

A computer program according to an embodiment of the present disclosure causes a computer capable of communication with multiple nodes to execute a process of distributing stored global model data to the multiple nodes, acquiring local model data learned based on the global model using local data processed by the multiple nodes, and updating the global model data based on the multiple local model data acquired from each of the multiple nodes.

In the model learning method, model learning system, server device, and computer program disclosed herein, learning is performed on distributed global model data using local data accessible to multiple nodes, a local model is created, and the data is sent to the server. Learning is possible using local data accessible by each of multiple nodes without aggregating the local data on the server.

In a model learning method according to an embodiment of the present disclosure, the updated global model data may be redistributed to the multiple nodes.

By re-learning the updated global model using local data and then aggregating this data, it is expected that the accuracy of the global model will be improved.

In a model learning method according to one embodiment of the present disclosure, the server performs statistical processing on the local model data acquired from the multiple nodes to update the global model data.

The global model can be calculated by statistical processing such as averaging or weighted averaging of local models. Even if the amount of local data accessible to each node is small, a global model that reflects learning from each local data can be created by statistically processing local models from multiple nodes.

The model learning method of one embodiment of the present disclosure may include a process of calculating an evaluation value of the accuracy of the global model using local data of each of the multiple nodes each time the data of the global model is updated, calculating a total evaluation value of the global model based on the different evaluation values calculated at the multiple nodes, and storing the data of the global model in association with the calculated total evaluation value.

The total evaluation value is calculated based on the accuracy using local data at each node. The total evaluation value may be the average of the accuracy values, or a weighted average based on the amount of local data.

The model learning method of one embodiment of the present disclosure may include a process of continuing to update the data of the global model until the learning satisfies a predetermined criterion based on the total evaluation value, and allowing the provision of the data of the global model corresponding to the learning result that satisfies the predetermined criterion to the multiple nodes and other devices.

When learning is completed to the extent that it is judged based on the total evaluation value that the learning meets a predetermined criterion and can be distributed to devices other than the node, it will be provided not only to the nodes that cooperated in the learning but also to other devices. This makes it possible to widely use a collective intelligence learning model based on the local data of multiple nodes. The predetermined criterion may be set as the accuracy of the learning no longer improving based on the total evaluation value, the total evaluation value being equal to or greater than a predetermined value, etc.

The model learning method of one embodiment of the present disclosure may include a process of storing a total evaluation value calculated for the updated global model each time the data of the global model is updated, and determining a global model to be redistributed based on the total evaluation.

When distributing data from a global model, training a local model, and updating the data of the global model based on the trained local model data is repeated, the global model from the previous update may be evaluated more highly. In an environment where local data is added sequentially, it is expected that accuracy can be improved by occasionally re-learning based on past global models during the repetitions.

The model learning method of one embodiment of the present disclosure may include a process of comparing a total evaluation value calculated for the updated global model with a total evaluation value for the global model after a previous update, and determining the global model with the higher evaluation as the result of the comparison as the global model to be redistributed.

When selecting a past global model, it is a good idea to use the total evaluation value as the basis for your selection.

In a model learning method according to an embodiment of the present disclosure, the plurality of nodes may sequentially store the amount of local data used to learn the local model or the amount of learning of the local model, and the server may include a process of assigning a weight corresponding to the amount of data or the amount of learning to each of the local model data aggregated from the plurality of nodes and using the weight to update the data of the global model.

In a model learning method according to an embodiment of the present disclosure, the plurality of nodes may store characteristics of local data, and the server may include a process of assigning weights corresponding to the characteristics of the local data of each node to each piece of local model data aggregated from the plurality of nodes and using the weights to update the data of the global model.

In the model learning method of one embodiment of the present disclosure, the characteristics of the local data may be a method for acquiring the local data by each node, a specification of a device that outputs the local data, or an accuracy evaluation of the local data.

Data for a global model based on local models is calculated as a weighted average according to the amount of data on which each local model is based or the amount of learning in the local model. The global model may be calculated as a weighted average according to the characteristics of the local data. The scale or quality of the local data varies from place to place, and it is expected that accuracy will be improved compared to uniform statistical processing of the data. The local model is influenced by the method for acquiring the local data, the specifications of the equipment that outputs the local data, the annotation accuracy when the local data is used as training data, etc. By calculating a weighted average with weights corresponding to these influences, it is expected that the data for the global model will be updated appropriately.

In a model learning method according to an embodiment of the present disclosure, each time the multiple nodes calculate an evaluation value for a global model after it has been updated by the server based on local data, the evaluation value and data for the global model may be stored, and the updated global model with the highest evaluation value may be selected.

In addition to the previous update, the global model with the highest evaluation value may be selected from past global models. By re-learning based on past global models, it is expected that accuracy will improve.

The model learning method disclosed herein enables model learning based on a large amount of data without aggregating the data for learning. The model learning method disclosed herein makes model learning practical even when the data is highly confidential.

FIG. 1 is a schematic diagram of a model learning system according to a first embodiment. FIG. 2 is a block diagram showing a configuration of a node. FIG. 2 is a block diagram showing a configuration of a server. 13 is a flowchart showing an example of a learning process procedure in the model learning system. 13 is a flowchart illustrating an example of a learning process procedure for a local model based on a distributed global model. 13 is a flowchart illustrating an example of a global model update process in the server. An example of the screen displayed on the node is shown below. 13 is a flowchart illustrating an example of a learning process procedure in the model learning system of the second embodiment. 13 is a flowchart illustrating an example of a learning process procedure in the model learning system of the third embodiment. 13 is a flowchart illustrating an example of a learning process procedure in the model learning system of the third embodiment. FIG. 13 is a schematic diagram of model learning in the third embodiment. 13 is a flowchart illustrating an example of a learning process procedure of a local model in a node according to the fourth embodiment. 13 is a flowchart illustrating an example of a global model update process in the server of the fourth embodiment. FIG. 13 is a schematic diagram of a model learning system according to a fifth embodiment. 13 is a flowchart illustrating an example of a learning process procedure in the model learning system of the fifth embodiment. 13 is a flowchart illustrating an example of a learning process procedure of a local model according to the fifth embodiment. 13 is a flowchart illustrating an example of a global model update process in the server of the fifth embodiment.

This disclosure will be specifically described with reference to drawings showing embodiments thereof.

(First embodiment)
1 is a schematic diagram of a model learning system 100 according to a first embodiment. The model learning system 100 includes one or more nodes 1 provided for a storage device 2 that stores data, a server 3, and a communication network N that connects the nodes 1 and the server 3 for communication.

The storage device 2 is capable of inputting and outputting data between devices that input and output data to be learned, such as sensors that measure physical quantities and cameras that take images, and stores this data. The storage device 2 may be connected to a computer for a specific purpose that outputs data according to data input by operation. The storage device 2 may be a storage device of an information terminal used by a user. The storage device 2 may be a storage device used in a server device that collects data from client devices.

When node 1 is input with data of the same type as the data stored in storage device 2, node 1 executes deep learning of the model so as to output a recognition result, a judgment result, or new data based on the data. Server 3 is a computer that provides a model to node 1, and also realizes model learning system 100 that learns the model in cooperation with node 1. In order to execute model learning using the data stored in storage device 2 (hereinafter referred to as local data) as training data, it is necessary to be able to access these. In the model learning system 100 of this embodiment, learning can proceed in a state in which access to the local data from server 3 is disabled.

The server 3 initially obtains a zeroth-order global model 51. The server 3 distributes the zeroth-order global model 51 to the node 1 via the communication network N. The entity (data) of the global model 51 distributed from the server 3 to the node 1 is only the trained parameters, or both the trained parameters and the program. The global model 51 may be definition data (including network definitions, losses, and pre-set hyperparameters) that defines the model configuration, and parameters such as weight coefficients to be trained.

The model to be trained can have any architecture as long as it is the subject of learning called deep learning. The type of deep learning model should be appropriately selected depending on the contents of the input data and output data. The model to be trained described below may be a classification system, detection system, or generation system using a CNN (Convolutional Neural Network) including a convolutional layer, or it may be a RNN (Recurrent Neural Network) that learns by taking into account time series elements.

The communication network N includes a public communication network, such as the Internet, and a carrier network. The communication network N may be a dedicated line for the model learning system 100.

Node 1 can access local data stored in the storage device 2 based on the local network LN between node 1 and storage device 2. Node 1 performs deep learning using the accessible local data. It is preferable that the local data has already been annotated by an operator at the location where node 1 is installed. Node 1 obtains the 0th global model 51 distributed from server 3. Node 1 proceeds with learning based on the 0th global model 51 and the local data as training data, and obtains the 1st local model 52.

Node 1 transmits the first local model 52 to server 3. Because local data is not transmitted to server 3, processing such as abstraction and anonymization of local data is not required.

The server 3 receives multiple first local models 52 from each of the multiple nodes 1, and performs statistical processing on the multiple received first local models 52 to create a first global model 51. The server 3 redistributes the first global model 51 to the multiple nodes 1. The redistributed global model 51 may be only the weight coefficient. The redistributed weight coefficient may be the learning target or may be the entire weight coefficient. The redistributed global model 51 may correspond to the difference from the previous update.

The model learning system 100 repeats the following steps: distribution of an nth order global model 51 from the server 3 to the node 1, learning of the nth order global model 51 using local data in the node 1, transmission of the (n+1)th order local model 52 obtained by learning to the server 3, collection of the (n+1)th order local model 52 in the server 3, and creation (update) of the (n+1)th order global model 51. An upper limit may be set in advance for "n", or the update may be manually stopped in the middle of the update by an operator of the server 3. Operation instructions to the server 3 from the operator may be possible only from within the same local network as the server 3 (on-premise type), or may be possible from, for example, the node 1 via the communication network N (cloud type).

This enables distributed learning without allowing server 3 to access local data.

The global model 51, which has been trained through distributed learning to an accuracy that allows it to be distributed to nodes other than node 1, is distributed and used not only to each node 1 but also to information processing devices 4 that do not participate in the creation of the local model 52.

The configuration of the model learning system 100 for realizing this learning method will be described in detail below.

Figure 2 is a block diagram showing the configuration of node 1. Node 1 is a personal computer or a server computer. Node 1 includes a processing unit 10, a memory unit 11, a communication unit 12, a display unit 13, and an operation unit 14.

The processing unit 10 is a processor that uses a CPU (Central Processing Unit) and/or a GPU (Graphics Processing Unit). Based on the node program 1P stored in the memory unit 11, the processing unit 10 executes processing including reading data from the storage device 2, sending and receiving models to and from the server 3, and model learning.

The storage unit 11 uses a non-volatile memory such as a hard disk, a flash memory, or an SSD (Solid State Drive). The storage unit 11 stores data referenced by the processing unit 10. The storage unit 11 stores a node program 1P. The storage unit 11 stores a library 1L for deep learning. The node program 1P and/or the library 1L for deep learning may be a node program 8P and/or a library 8L for deep learning stored in a recording medium 8 that is read by the processing unit 10 and copied to the storage unit 11. The storage unit 11 stores a global model 51 acquired from the server 3 and a local model 52 that is learned using local data.

The communication unit 12 realizes data communication via the communication network N and communication with the storage device 2 via the local network LN. Specifically, the communication unit 12 is, for example, a network card. The processing unit 10 reads data from the storage device 2 via the communication unit 12 and transmits and receives data between the server 3.

The display unit 13 is a display such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 13 displays a screen including information based on data stored in the memory unit 11 or data provided from the server 3. The display unit 13 may be a display with a built-in touch panel.

The operation unit 14 is a user interface such as a keyboard and a pointing device that can input and output data to and from the processing unit 10. The operation unit 14 may be a voice input unit. The operation unit 14 may be a touch panel of the display unit 13. The operation unit 14 may be physical buttons. The operation unit 14 notifies the processing unit 10 of operation data by the operator of the node 1.

FIG. 3 is a block diagram showing the configuration of server 3. Server 3 is a server computer. Server 3 includes a processing unit 30, a storage unit 31, and a communication unit 32. In the following description, server 3 is described as being configured as one server computer, but it may also be configured in a manner in which multiple server computers are communicatively connected via a network for distributed processing. Server 3 may be a cloud type that can be communicatively connected from each node 1 via a communication network N, or may be an on-premise type that is communicatively connected to each node 1 via a virtual private network.

The processing unit 30 is a processor using a CPU and/or a GPU. The processing unit 30 executes the learning process of the global model 51 based on the server program 3P stored in the memory unit 31.

The storage unit 31 uses a non-volatile memory such as a hard disk or SSD. The storage unit 31 stores data referenced by the processing unit 30. The storage unit 31 stores a server program 3P. The storage unit 31 stores a global model 51 to be learned and local models 52 transmitted from multiple nodes 1. The server program 3P may be a server program 9P stored in a recording medium 9 that is read by the processing unit 30 and copied to the storage unit 31.

The communication unit 32 realizes data communication via the communication network N. Specifically, the communication unit 32 is, for example, a network card. The processing unit 30 transmits and receives data between multiple nodes 1 via the communication unit 32.

The learning process procedure in the model learning system 100 configured in this manner will be described. Figure 4 is a flowchart showing an example of the learning process procedure in the model learning system 100.

The server 3 acquires an initial (0th order) global model 51 that has been prepared in advance (step S1). The initial global model 51 may be a model created as a 0th order model at a specific node 1, or may be a model learned at a specific location other than node 1, and may be stored in advance in the storage unit 31. The acquisition in step S1 includes reading out the global model 51 that has been stored in advance in the storage unit 31.

The server 3 distributes the acquired zeroth global model 51 to the node 1 (step S2).

The server 3 obtains the local model 52 to be learned at the node 1 based on the global model 51 distributed to the node 1 (step S3).

The server 3 performs statistical processing on the acquired local model 52 and updates it to the next-generation global model 51 (step S4). In step S4, the server 3 may increment the number of updates (rounds).

The server 3 determines whether the updated global model 51 satisfies the learning completion conditions (step S5).

In step S5, the server 3 may, for example, make a judgment based on whether the number of learning attempts has reached a predetermined number. When input data of training data at a specific node 1 or a specific location is input to the global model 51, the server 3 may make a judgment based on whether the accuracy of outputting the corresponding output data satisfies a predetermined condition (above a predetermined value, no improvement is observed, etc.). The server 3 may also judge whether the learning completion condition is met by a method described below. In step S5, the server 3 may judge whether an instruction to stop learning has been given by the operator's operation even if the number of learning attempts has not reached the predetermined number, and if so, may judge that the condition is met in step S5.

If it is determined that the learning completion condition is not met (S5: NO), the server 3 redistributes the updated global model 51 to multiple nodes 1 (step S6) and returns the process to step S3.

In step S6, the server 3 may redistribute only parameters such as weighting coefficients, rather than redistributing the global model 51 as is.

In step S6, the server 3 may also transmit data indicating the number of updates to the global model 51, i.e., the "n" of the nth global model 51. In step S6, if the server 3 can obtain learning round information as described below, the server 3 may transmit the round information in association with the global model 51. The round information is information indicating the number of times distribution to the node 1 and statistical processing were performed until it is determined that the learning completion condition is satisfied. The server 3 may also transmit data indicating the type of global model 51, for example, the type of architecture that was used for deep learning, to the node 1.

If it is determined that the learning completion condition is met (S5: YES), the server 3 stores the updated global model 51 as a global model 51 that can be distributed to devices other than the node 1 (step S7).

The server 3 transmits the stored global model 51 to multiple nodes 1 or other information processing devices (step S8), and ends the process. The other information processing devices are devices that, like the node 1, use the same type of data as the learning target, but do not provide local data for training.

The server 3 may execute the processing procedure shown in the flowchart of FIG. 4 multiple times, for example, once a month. Each time, the version of the global model is upgraded, making it a more practical model.

Figure 5 is a flowchart showing an example of a learning process procedure for a local model 52 based on a distributed global model 51. The process shown in the flowchart in Figure 5 is executed by each of the multiple nodes 1 when the server 3 distributes the global model 51 in step S2 or step S6.

The processing unit 10 of node 1 receives the distributed global model 51 and stores it in the memory unit 11 (step S301).

The processing unit 10 of node 1 loads the stored global model 51 as an instance (step S302). The processing unit 10 acquires the local data stored in the storage device 2 as training data (step S303) and provides this to the global model 51 to perform learning (step S304).

In step S304, the processing unit 10 inputs the input data included in the local data to the loaded global model 51. The processing unit 10 calculates a loss function for the output data and the result data corresponding to the input data included in the local data. The processing unit may proceed with learning based on the match rate between the output data and the result data corresponding to the input data included in the local data, and the accuracy of the match. The processing unit 10 learns parameters including weight coefficients in the distributed global model 51 based on the calculated loss function.

The processing unit 10 of node 1 determines whether the learning completion condition is met (step S305). In step S305, the processing unit 10 may set the learning completion condition to be that the number of learning times meets a predetermined number (one or more). The processing unit 10 may determine that the learning completion condition is met when the output accuracy of the global model 51 after learning is equal to or greater than a stored predetermined value. The processing unit 10 may determine that the learning completion condition is met when the change in accuracy falls within a predetermined range and can be determined to have converged.

If it is determined that the learning completion condition is not met (S305: NO), the processing unit 10 returns the process to step S304. This allows learning to continue.

If it is determined that the learning completion condition is met (S305: YES), the processing unit 10 ends the learning and stores the global model 51 with updated parameters as the local model 52 (step S306).

The processing unit 10 of the node 1 transmits the stored local model 52 to the server 3 (step S307) and ends the process. This enables the server 3 to obtain the local model 52 trained with the local data from each of the multiple nodes 1.

In step S307, the processing unit 10 may also transmit data indicating "n" indicating whether the local model 52 is of the nth order or whether the original global model 51 is of the nth order. In step S307, if the node 1 obtains round information up to the completion of one learning round as described below, the node 1 may transmit the round information in association with the local model 52.

It should be noted that the process shown in the flowchart of Figure 5 does not result in local data being sent from node 1 to server 3. No anonymization of the local data is performed. The data sent from node 1 is the model itself. The characteristics of the local data are reflected, but no data is sent.

Figure 6 is a flowchart showing an example of a process for updating the global model 51 in the server 3. The process procedure shown in the flowchart in Figure 6 corresponds to the details of step S4 in the process procedure shown in the flowchart in Figure 4.

The processing unit 30 of the server 3 acquires the local model 52 transmitted from the node 1 (step S401) and stores the local model 52 in association with the identification data of the node 1 (step S402). In step S401, the processing unit 30 of the server 3 acquires the local model 52 transmitted asynchronously from each node 1.

The processing unit 30 determines whether the global model 51 should be updated with the acquired local model 52 (step S403). In step S403, the processing unit 30 may determine that the global model 51 should be updated if local models 52 can be acquired from all of the nodes 1 to which the global model 51 has been distributed. In step S403, the processing unit 30 may determine that the global model 51 should be updated if local models 52 can be acquired from a number of representative nodes 1 that have been determined in advance.

If it is determined that an update is not necessary (S403: NO), the processing unit 30 of the server 3 returns the process to step S401. The local models 52 sent from each node 1 are acquired and aggregated until it is determined that an update is necessary.

If it is determined that an update is necessary (S403: YES), the processing unit 30 of the server 3 calculates the average of the local models 52 from the multiple nodes 1 (step S404). The processing unit 30 updates the average as a new global model 51 (step S405).

The processing unit 30 stores the updated global model 51 in association with data indicating the round number (nth) (step S406), and ends the update process of the global model 51. As a result, the (n-1)th global model 51 is updated to the nth global model 51.

Once the update is complete, the global model 51 becomes available to each node 1 and the information processing device 4. Figure 7 shows an example of a screen displayed on the node 1. The update result screen 431 in Figure 7 is displayed on the display unit 13 of the node 1 based on a web page provided by the server 3. The update result screen 431 displays a list of global models 51 stored as distributable models. The update result screen 431 shows the model name and update date and time of the latest global model 51 automatically distributed from the server 3 to the node 1.

The global model 51 for which the update has been completed by the server 3 may be selectively used by each node 1 and information processing device 4. The update result screen 431 may include a link to download in the text representing the model name that identifies the global model 51, making it selectable. The text identifying each global model 51 may include an interface such as a download icon. The selected model may be stored in the node 1 only when the operator of the node 1 or the information processing device 4 selects one of the models on the update result screen 431. In this case, the global model 51 provided as a distributable model is downloaded to the node 1 or the information processing device 4 and becomes available. In this case, it is preferable that each model displays evaluation and accuracy information that serves as a criterion for the operator to determine whether to select it or not.

As described above, in the model learning system 100 of the first embodiment, the local data is not sent to the server 3, and the global model 51 is put into practical use based on the learning results using the local data stored in each location. Even in cases where the amount of data is insufficient using only the local data from each location, it is possible to provide a model that is more accurate and can be put into practical use sooner than learning using a large amount of data aggregated in one location.

Second Embodiment
The model learning system 100 in the second embodiment differs from the first embodiment in the judgment process for satisfying the learning completion condition. The configuration of the model learning system 100 in the second embodiment is the same as that of the model learning system 100 in the first embodiment except for the details of the judgment process described above, so the same reference numerals are used for the common configuration and detailed description will be omitted.

Figure 8 is a flowchart showing an example of a learning process procedure in the model learning system 100 of the second embodiment. Among the process procedures shown in the flowchart of Figure 8, the steps common to the process procedures shown in the flowchart of Figure 4 are given the same step numbers and detailed explanations are omitted.

When the global model 51 is updated in step S4 (S4), the processing unit 30 of the server 3 transmits the updated global model 51 to multiple nodes 1 (step S501).

Each of the multiple nodes 1 that receive the updated global model 51 reads local data from the storage device 2 using the transmitted global model 51 and performs accuracy evaluation on the input data. The processing unit 10 of the node 1 calculates the accuracy when the input data of the local data is input, and transmits it to the server 3. The processing unit 10 may calculate, as the accuracy, the matching rate between the output data when the input data in the local data is input to the global model 51, and the output data in the local data that corresponds to the input data. The processing unit 10 may accept the accuracy evaluation from the operator of the node 1 via the operation unit 14.

The processing unit 30 of the server 3 acquires the evaluation value sent from each node 1 (step S502), and calculates a total evaluation value from the evaluation values acquired from the multiple nodes 1 (step S503). In step S503, the processing unit 30 may calculate the average accuracy, the average matching rate, or the average evaluations received from the operators.

The processing unit 30 determines whether the calculated total evaluation value satisfies a predetermined criterion (step S504), and if it is determined that the calculated total evaluation value satisfies the predetermined criterion (S504: YES), the processing unit 30 proceeds to step S7. In step S504, the processing unit 30 may determine whether to stop updating the global model 51.

If it is determined that the specified criteria are not met (S504: NO), the processing unit 30 proceeds to step S6.

In step S504, the specified criterion is, for example, the evaluation value being the accuracy, and if the evaluation value is the average, the threshold value for that value. Alternatively, the specified condition may be B or higher if the evaluation is A, B, or C, or may be a condition such as good/passable/fail for the evaluation. The specified condition may also be that the accuracy is stable. The specified condition may be satisfied when the change in the evaluation value becomes smaller as the learning progresses.

The evaluation is derived based on the usefulness using local data at node 1, and can be provided as a distributable model if the accuracy is recognized at each node 1.

Third Embodiment
In the third embodiment, the server 3 determines the global model 51 to be distributed by comparing with the previously updated global model 51. Since the configuration of the model learning system 100 of the third embodiment is similar to the configuration of the model learning system 100 of the first embodiment, the same reference numerals are used for the common configuration, and detailed description thereof will be omitted.

Figures 9 and 10 are flowcharts showing an example of a learning process procedure in the model learning system 100 of the third embodiment. Among the process procedures shown in the flowcharts of Figures 9 and 10, the same step numbers are used for the process procedures common to the process procedures shown in the flowchart of Figure 8 of the second embodiment, and detailed descriptions are omitted.

The server 3 transmits the global model 51 after the update in step S4 and the global model 51 before the update to each node 1 (step S511).

The server 3 acquires the evaluation value of each global model 51 from the multiple destination nodes 1 (step S512) and calculates a total evaluation value (step S513). The server 3 stores the updated global model 51 and the total evaluation value in the storage unit 31 in association with the round information (step S514). The round information is data indicating how many times the global model 51 has been updated before one learning session (version) is completed, and is the nth-order "n" information.

The server 3 compares the total evaluation value calculated for the updated global model 51 in step S513 with the total evaluation value for the global model 51 before the update (step S515).

In step S515, the server 3 may compare the values if the total evaluation value is a numerical value, or may compare the evaluation contents if the total evaluation value is a multi-level evaluation. In step S515, the server 3 may compare the total evaluation values for three or more global models 51 if the server 3 has stored a previously updated global model 51 without discarding it.

The server 3 determines the global model 51 with the higher rating as the latest global model 51 to be distributed (step S516). The server 3 may delete (discard) the global model 51 with the lower rating from the storage unit 31.

The server 3 judges whether the total evaluation value calculated in step S513 for the determined global model 51 satisfies a predetermined criterion (step S517). The predetermined criterion is a threshold value for the evaluation value, if the evaluation value is accuracy and the total evaluation value is its average. In addition, the predetermined condition may be B or higher if the evaluation is A, B, or C, or may be a condition such as good/passable/fail for the evaluation. The predetermined condition may also be that the accuracy is stable. The predetermined condition may be satisfied when the change in the evaluation value becomes smaller as the learning progresses.

If it is determined that the specified criteria are not met (S517: NO), the server 3 redistributes the global model 51 to be distributed to each node 1 (step S518) and returns the process to step S3.

If it is determined that the specified criteria are met (S517: YES), the server 3 stores the determined global model 51 in the memory unit 31 as a model that can be distributed to other devices (step S519), and proceeds to step S8.

Figure 11 is an overview diagram of model learning in the third embodiment. The overview diagram in Figure 11 shows that when updating the global model, if the nth order global model 51 after the previous update has a higher evaluation than the n+1th order global model 51 after the current update, the server 3 selects the previous nth order global model 51. Compared to the overview diagram of the model learning system in Figure 1, it shows that the global model 51 is not always adopted in succession as it is updated.

During one update of the global model 51, the local data is increased or updated at each node 1. Even if the previously updated nth global model 51 continues to be used, the evaluation based on the local data accessible at each node 1 may change. Even with the same global model 51, a global model 51 with a higher evaluation can be obtained.

This makes it possible to carry out distributed learning to improve accuracy.

(Fourth embodiment)
In the fourth embodiment, the method of updating the global model 51 based on the local model 52 learned in each node 1 differs from the methods described in the first to third embodiments. The configuration of the model learning system 100 in the fourth embodiment is similar to that of the model learning system 100 in the first embodiment except for the details of the update process of the global model 51, so the same reference numerals are used for the common configuration and detailed description will be omitted.

In the model learning system 100 of the fourth embodiment, the server 3 executes a learning process procedure, which is either the process procedure shown in the flowchart of FIG. 4 of the first embodiment, the process procedure shown in the flowchart of FIG. 8 of the second embodiment, or the process procedure shown in the flowcharts of FIG. 9 and FIG. 10 of the third embodiment. In the fourth embodiment, among these process procedures, the update process of step S4 is different. In addition, in response to the difference in the update process, the contents of the data transmitted from the node 1 are also different from those described in the first embodiment.

Figure 12 is a flowchart showing an example of a learning process procedure for the local model 52 in node 1 of the fourth embodiment. Among the process procedures shown in the flowchart of Figure 12, the steps common to the process procedures shown in the flowchart of Figure 6 in the first embodiment are given the same step numbers and detailed descriptions are omitted.

The processing unit 10 of node 1 acquires the local data stored at that time from the storage device 2 (S303), calculates the amount of local data (number of data items) (step S313), and proceeds to step S304.

Each time learning is performed (S304), the processing unit 10 of node 1 calculates the amount of learning (step S314) and determines whether the learning completion condition is met (S305).

In step S314, the processing unit 10 may calculate the number of times the learning is repeated as the amount of learning, may calculate the degree of improvement in accuracy from the local model 52 in the previous round as the amount of learning, or may calculate the amount of change in the local model 52 as the amount of learning.

In step S305, the processing unit 10 may determine that the learning completion condition is met when, as a result of the learning, the accuracy of the output data from the neural network for the local data satisfies a predetermined condition, for example, the accuracy is equal to or greater than a predetermined value. The processing unit 10 may determine that the learning completion condition is met when the amount of learning (number of times of learning) is equal to or greater than a predetermined amount (predetermined number of times).

The processing unit 10 of the node 1 transmits the amount of data calculated in step S313 or the amount of learning that has been counted together with the trained local model 52 to the server 3 (step S315), and ends the process. This enables the server 3 to obtain the amount of data or the amount of learning of the original local data together with the local model 52 trained with the local data from each of the multiple nodes 1.

Even in the fourth embodiment, node 1 transmits the amount of local data, but the data itself is not transmitted to server 3.

Figure 13 is a flowchart showing an example of a process for updating the global model 51 in the server 3 of the fourth embodiment. Among the process steps shown in the flowchart of Figure 13, those steps that are common to the process steps shown in the flowchart of Figure 7 of the first embodiment are given the same step numbers and detailed descriptions are omitted.

The processing unit 30 of the server 3 acquires the local model 52 and the amount of data or learning transmitted from each node 1 (step S411), and stores the local model 52 and the amount of data or learning in association with the identification data of the node 1 (step S412). In step S411, the processing unit 30 of the server 3 asynchronously acquires the local model 52 transmitted from each node 1.

When the processing unit 30 determines that an update is necessary (S403: YES), the processing unit 30 of the server 3 calculates a weighted average by weighting the amount of data or the amount of learning to the local models 52 from the multiple nodes 1 (step S414). The processing unit 30 updates the weighted average of the local models 52 as a new global model 51 (step S415).

The processing unit 30 stores the updated global model 51 in association with the data indicating the round information (S406), and ends the update process of the global model 51.

The update process shown in the flowchart in Figure 13 continues until the accuracy of the updated global model 51 meets a predetermined standard.

The update method in the fourth embodiment makes it possible to handle models trained using local data with different amounts of data according to the amount of data and to appropriately evaluate them. In addition, it is expected that the performance of the global model 51 will be improved compared to simply averaging local models 52 with different amounts of training.

Fifth Embodiment
The model learning system 100 can be applied to deep learning models including neural networks. For example, the input data is image data, and the output data is a detection result of a subject appearing in the image of the image data, or a judgment result for the subject. In another example, the input data is cash receipts and withdrawals data, and the output data is a judgment result such as an evaluation, or a predicted value regarding a change in a company's business condition (growth or deterioration) or a predicted value regarding an economic forecast. In another example, the input data is measurement data from various sensors installed in a factory or production equipment, and the output data is data regarding production management including abnormality/normality. In another example, the input data is text data, and the output data is a judgment result or predicted data.

In the fifth embodiment, an example will be described in which the model learning system 100 is applied to learning a model that outputs data that assists in diagnosing whether or not a patient is experiencing symptoms of a specific disease, based on medical data obtained about the patient at a medical facility. In the following description, the medical data is, for example, image data of a fundus photograph taken during an examination.

Figure 14 is a schematic diagram of the model learning system 100 of the fifth embodiment. The configuration of the model learning system 100 in the fifth embodiment is basically the same as that of the model learning system 100 of the first embodiment. In the model learning system 100 of the fifth embodiment, the configuration common to the model learning system 100 of the first embodiment is given the same reference numerals and detailed description is omitted.

In the fifth embodiment, the node 1 and the information processing device are provided in a medical facility. The node 1 and the information processing device can acquire image data obtained from an imaging device that takes fundus photographs of patients. In the medical facility where the node 1 is provided, the imaging device outputs image data to the storage device 2. The imaging device includes different types of devices.

In the fifth embodiment, the global model 51, the local model 52, and the model stored as distributable are models that are trained to output data and accuracy that assists in the diagnosis of glaucoma when image data is input. The local data used as training data is image data as input data and image data resulting from segmenting the disc and cup portions of a fundus photograph as output data. The output data may also be a data set including the results of a doctor or technician's determination of whether or not symptoms are present. The image data that is input data for the local data is associated with device data that indicates the type of imaging device. The device data may be a model number or data that identifies the device manufacturer.

In the fifth embodiment of the model learning system 100, the server 3 acquires an initial global model (zeroth-order global model) 51 that has been created in advance with multiple different architectures at a specific medical facility. The server 3 distributes the zeroth-order global models 51 with different architectures to each of the nodes 1 of the medical facilities that cooperate in the training.

Each node 1 receives the distributed multiple 0th order global models 51, and proceeds with learning based on different 0th order global models 51, each using local data as training data, to obtain multiple first order local models 52. Node 1 transmits the first order local models 52 trained with different architectures to server 3.

The server 3 creates a first global model 51 by weighting the local models 52 acquired from each node 1 for each different architecture. The server 3 redistributes the created first global model 51 to multiple nodes 1.

The server 3 repeatedly obtains the (n+1)th order local model 52 created from the distributed nth order global model 51 and updates the (n+1)th order global model 51 from the (n+1)th order local model 52 for each different architecture.

The server 3 compares the global models 51 obtained repeatedly for each different architecture, selects the global model 51 of the architecture with higher accuracy, and provides it to the node 1 and the information processing device as a distributable model.

This makes it possible to learn using a large amount of data from different medical facilities without consolidating image data of test results, which is personal information itself, on server 3.

Figure 15 is a flowchart showing an example of a learning process procedure in the model learning system 100 of the fifth embodiment.

The server 3 acquires an initial (0th order) global model 51 created with a different architecture at a specific medical facility (step S201). The acquisition in step S201 includes reading out the global model 51 previously stored in the memory unit 31.

The server 3 distributes the acquired multiple zeroth-order global models 51 to each node 1 (step S202).

Based on the global model 51 distributed to node 1, server 3 obtains local models 52 to be learned at node 1 for each different architecture (step S203).

The server 3 performs statistical processing on the acquired local model 52 for each different architecture, and updates it to the next-generation global model 51 (step S204). In step S204, the server 3 may increment the number of updates.

The server 3 determines whether the learning completion condition is met (step S205).

In step S205, the server 3 makes a judgment based on, for example, whether the number of updates (number of rounds) has reached a predetermined number. When input data of local data in a specific node 1 is input to a global model 51 of a different architecture, the server 3 may also make a judgment based on whether the accuracy of outputting corresponding output data in any of a plurality of global models 51 satisfies a predetermined condition. The learning completion condition may be, as shown in the second to fourth embodiments, that a total evaluation value is calculated from the evaluation values for the global model 51 sent to each node 1, and the total evaluation value satisfies a predetermined criterion.

If it is determined that the learning completion condition is not met (S205: NO), the server 3 redistributes the updated global model 51 of each architecture to multiple nodes 1 (step S206) and returns the process to step S203.

If it is determined that the learning completion condition is met (S205: YES), the server 3 selects one or more global models 51 that can be distributed from the global models 51 of each architecture after the update (step S207). In step S207, the server 3 selects, for example, one or more global models 51 with relatively high accuracy.

The server 3 transmits the stored global model 51 to multiple nodes 1 or other information processing devices (step S208) and ends the process.

This allows each medical facility to use a model with a highly accurate and practical architecture for the image data of that facility. Once the global model 51 is sent as a distributable model, the server 3 can then update the version by executing the process shown in the flowchart of FIG. 15.

Figure 16 is a flowchart showing an example of a learning process procedure for the local model 52 in the fifth embodiment. The process shown in the flowchart in Figure 16 is executed by each of the multiple nodes 1 when the server 3 distributes the global model 51 in step S202 or step S206. Among the process procedures shown in the flowchart in Figure 16, steps that are common to the process procedures shown in the flowchart in Figure 6 of the first embodiment are given the same reference numerals and detailed explanations are omitted.

When the processing unit 10 of node 1 receives and stores the distributed global model 51 (S301), it executes the processes from S302 to S305 for each architecture. The processing unit 10 stores the learned global model 51 as a local model 52 in association with data identifying the architecture (step S316).

The processing unit 10 calculates an evaluation of the output accuracy of the local model 52 at the time when learning is completed based on the local data (step S317). In step S317, the processing unit 10 may calculate, as the accuracy evaluation, the average accuracy output when the local data input data is provided to the local model 52. The processing unit 10 may calculate, as the accuracy evaluation, the matching rate at which the output data when the local data is input to the local model 52 matches the output data of the training data. The processing unit 10 may accept the accuracy evaluation at the operation unit 14.

The processing unit 10 determines whether learning has been completed for all architectures (step S318).

If it is determined that learning is not complete (S318: NO), the processing unit 10 returns the process to step S302 and executes the processes from step S302 to S317 for the next architecture.

If it is determined that learning has been completed for all architectures (S318: YES), the processing unit 10 of node 1 transmits the stored local model 52 and its accuracy evaluation to the server 3 in association with the device data, which is a characteristic of the local data (step S319). The processing unit 10 then terminates the n-th learning process in node 1.

This allows the server 3 to obtain a local model 52 trained with local data from each of the multiple nodes 1. The local data itself is not transmitted to the server 3, but the quality of the local data can be distinguished by the server 3.

Figure 17 is a flowchart showing an example of a process for updating the global model 51 in the server 3 of the fifth embodiment. The process procedure shown in the flowchart of Figure 17 corresponds to the details of step S204 in the process procedure shown in the flowchart of Figure 15. Among the process procedures shown in the flowchart of Figure 17, the steps common to the process procedure shown in the flowchart of Figure 6 of the first embodiment are given the same step numbers and detailed explanations are omitted.

The processing unit 30 of the server 3 acquires the local model 52 for each architecture, the accuracy assessment, and the device data transmitted from the node 1 (step S421). The processing unit 30 stores the acquired local model 52 for each architecture, the accuracy assessment at the node 1, and the device data in association with the identification code of the node 1 (step S422).

If it is determined that an update is necessary (S403: YES), the processing unit 30 of the server 3 excludes, from among the local models 52 from the multiple nodes 1, local models 52 whose accuracy evaluation is lower than a predetermined evaluation (step S423). The processing unit 30 calculates a weighted average for each architecture by applying weights associated with the device data to the excluded local models 52 (step S424). The processing unit 30 updates the weighted average of the local models 52 as a new global model 51 for each architecture (step S425).

The weights associated with the device data in step S424 are stored in advance in the storage unit 31. The weights may be large for local models 52 trained from local data of image data obtained from newer devices and small for local models 52 trained from local data of image data obtained from older devices. The weights may be large the more widely used the device is in medical facilities, so that a highly versatile model may be created. It is expected that accuracy will be improved when weights corresponding to the quality of the local data are assigned to the local models 52 to calculate the weighted average, compared to when local models 52 trained from local data collected with different qualities are simply averaged.

In step S424, a global model 51 can be created that takes into account weighting according to the evaluation by excluding local models 52 whose accuracy evaluation is lower than a predetermined evaluation. Note that a threshold value for the predetermined evaluation may be set in advance by an operator of the server 3. Access from the operator of the server 3 may be permitted via node 1 to accept the evaluation of the local model 52.

In step S424, in the above example, the weights used are those associated with the device data. This is not limiting, and each node 1 may store an evaluation of the accuracy of annotations for local data, and the server 3 may calculate a weighted average using a weight according to the level of the evaluation. The evaluation of the accuracy of annotations may be performed by an administrator of node 1.

The processing unit 30 stores the global model 51 updated in step S425 in the memory unit 31 for each architecture (step S426) and ends the process.

In the fifth embodiment, since the local data is medical data, the weights used in the weighted average in step S424 may be assigned according to the attributes of the medical data. For example, weights may be assigned according to the male/female ratio of patients, age distribution, regionality, etc. These abstracted data may be reflected in the local model 52.

In the fifth embodiment, the update process shown in the flowchart in FIG. 17 is continued until the updated global model 51 satisfies the learning completion condition.

The update method in the fifth embodiment makes it possible to integrate models trained using local data of different quality by changing the weights according to the quality of the data. This is expected to improve the performance of the global model 51 compared to simply averaging local models 52 of different quality.

The update method in the fifth embodiment allows for evaluation of which of the different CNN architectures is most appropriate for the global model 51, making it practical to use.

The embodiments disclosed above are illustrative in all respects and are not restrictive. The scope of the present invention is defined by the claims, and includes all modifications within the meaning and scope of the claims.

Reference Signs List 1 Node 10 Processing unit 2 Storage device 3 Server 30 Processing unit 51 Global model 52 Local model

Claims (18)

Distributing global model data stored in the server to multiple nodes;
acquiring data of a local model trained on the global model using local data processed by the plurality of nodes together with characteristics of the local data;
updating data of a global model by assigning weights to data of a plurality of local models acquired from each of the plurality of nodes, the weights corresponding to characteristics of local data used in training each of the local models ;
A model learning method , wherein the characteristics of the local data are a method for acquiring the local data by each node or a specification of a device that outputs the local data . Distributing global model data stored in the server to multiple nodes;
acquiring data of a local model trained on the global model using local data processed by the plurality of nodes together with characteristics of the local data;
updating data of a global model by assigning weights to data of a plurality of local models acquired from each of the plurality of nodes, the weights corresponding to characteristics of local data used in training each of the local models ;
The characteristics of the local data are attributes of a person from whom the local data was obtained,
A model learning method , wherein the weights are preset based on the gender ratio, age distribution, or regionality of the persons . The model learning method according to claim 1 or 2 , further comprising the step of redistributing data of the updated global model to the plurality of nodes. The model learning method according to claim 1 , wherein the server performs statistical processing on the data of the local model acquired from the plurality of nodes to update the data of the global model. Each time the data of the global model is updated, an evaluation value of the accuracy of the global model is calculated using local data of each of the plurality of nodes;
calculating a total evaluation value of the global model based on the different evaluation values calculated at the plurality of nodes;
The model learning method according to claim 1 , further comprising: storing data of the global model in association with the calculated total evaluation value. continuing to update the data of the global model based on the total evaluation value until the learning satisfies a predetermined criterion;
The model learning method according to claim 5 , further comprising a process of permitting provision of global model data corresponding to the learning result that satisfies the predetermined criterion to the plurality of nodes and other devices. Each time the data of the global model is updated,
A total evaluation value calculated for the updated global model is stored.
The model learning method according to claim 5 or 6 , further comprising a process of determining a global model for redistribution based on a total evaluation. A total evaluation value calculated for the updated global model is compared with a total evaluation value for the previously updated global model;
The model learning method according to claim 7 , further comprising the step of: determining, as a result of the comparison, a global model having a higher evaluation as a global model to be redistributed. the plurality of nodes sequentially store an amount of local data used in learning a local model or an amount of learning of the local model;
9. The model learning method according to claim 1, further comprising a process in which the server assigns a weight corresponding to the amount of data or the amount of learning to each of the local model data aggregated from the plurality of nodes and uses the weight for updating the data of the global model. The local data is image data, and the characteristic is image quality of the image data output by the device,
The model learning method according to claim 1 , wherein the higher the image quality of the image data, the greater the weighting. The characteristic is a model number of a device that outputs the local data,
The model learning method according to claim 1 , wherein a greater weight is assigned to local data output from a device having a model number that is more widely used. storing, in the plurality of nodes, an evaluation value for a global model updated by the server based on local data, the evaluation value and data of the global model;
The model learning method according to claim 5 , further comprising the step of selecting the updated global model having the highest evaluation value. The system includes a plurality of nodes each processing local data, and a server connected to the plurality of nodes for communication;
The server distributes data of a global model to the plurality of nodes;
each of the plurality of nodes learns a local model from the global model using local data to be processed;
The server,
obtaining trained local model data from the plurality of nodes together with characteristics of the local data used in training;
the characteristics of the local data are a method for acquiring the local data by each node or a specification of a device that outputs the local data;
The server updates the data of the global model by assigning weights to the acquired data of the multiple local models corresponding to characteristics of the local data used to train each local model. The system includes a plurality of nodes each processing local data, and a server connected to the plurality of nodes for communication;
The server distributes data of a global model to the plurality of nodes;
each of the plurality of nodes learns a local model from the global model using local data to be processed;
The server,
obtaining trained local model data from the plurality of nodes together with characteristics of the local data used in training;
The characteristics of the local data are attributes of a person from whom the local data was obtained,
The data of the global model is updated by assigning weights, which are preset according to the gender ratio, age distribution, or regionality of the people, to the data of the multiple local models acquired from each of the multiple nodes as weights corresponding to the characteristics of the local data used in learning each local model.
Model learning system. A communication unit that communicates with a plurality of nodes;
a processing unit for storing data of a global model,
The processing unit
Distributing the global model data to a plurality of nodes;
execute a process of acquiring data of a local model learned based on data of the global model by local data processed by the plurality of nodes from the communication unit together with characteristics of the local data;
the characteristics of the local data are a method for acquiring the local data by each node or a specification of a device that outputs the local data;
The processing unit updates the data of a global model by assigning weights to the data of a plurality of local models acquired from each of the plurality of nodes, the weights corresponding to characteristics of the local data used in training each local model. A communication unit that communicates with a plurality of nodes;
A processing unit for storing data of a global model;
Equipped with
The processing unit
Distributing the global model data to a plurality of nodes;
execute a process of acquiring data of a local model learned based on data of the global model by local data processed by the plurality of nodes from the communication unit together with characteristics of the local data;
The characteristics of the local data are attributes of a person from whom the local data was obtained,
The data of the global model is updated by assigning weights, which are preset according to the gender ratio, age distribution, or regionality of the people, to the data of the multiple local models acquired from each of the multiple nodes as weights corresponding to the characteristics of the local data used in learning each local model.
Server device. A computer that can communicate with multiple nodes.
Distributing the stored global model data to the plurality of nodes;
executing a process of acquiring data of a local model learned based on the global model by local data processed by the plurality of nodes together with characteristics of the local data;
the characteristics of the local data are a method for acquiring the local data by each node or a specification of a device that outputs the local data;
The computer includes:
A computer program that executes a process of updating data of a global model by assigning weights to data of a plurality of local models acquired from each of the plurality of nodes, the weights corresponding to characteristics of the local data used in training each local model. A computer that can communicate with multiple nodes.
Distributing the stored global model data to the plurality of nodes;
executing a process of acquiring data of a local model learned based on the global model by local data processed by the plurality of nodes together with characteristics of the local data;
The characteristics of the local data are attributes of a person from whom the local data was obtained,
The computer includes:
The data of the global model is updated by assigning weights, which are preset according to the gender ratio, age distribution, or regionality of the people, to the data of the multiple local models acquired from each of the multiple nodes as weights corresponding to the characteristics of the local data used in learning each local model.
A computer program that executes a process.

Images