CN113919506A - A model training online method, device, storage medium and device - Google Patents
A model training online method, device, storage medium and device Download PDFInfo
- Publication number
- CN113919506A CN113919506A CN202111175935.XA CN202111175935A CN113919506A CN 113919506 A CN113919506 A CN 113919506A CN 202111175935 A CN202111175935 A CN 202111175935A CN 113919506 A CN113919506 A CN 113919506A
- Authority
- CN
- China
- Prior art keywords
- model
- test
- online
- evaluation value
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012549 training Methods 0.000 title claims abstract description 147
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000012360 testing method Methods 0.000 claims abstract description 331
- 238000011156 evaluation Methods 0.000 claims abstract description 125
- 230000007246 mechanism Effects 0.000 abstract description 8
- 238000004519 manufacturing process Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 229920002803 thermoplastic polyurethane Polymers 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Prevention of errors by analysis, debugging or testing of software
- G06F11/3668—Testing of software
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5022—Mechanisms to release resources
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Quality & Reliability (AREA)
- Computer Hardware Design (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a model training online method, a model training online device, a storage medium and a model training online device. And starting a preset algorithm service to carry out AB test on the test model and the currently online model under the condition that the test evaluation value is greater than the target evaluation value and a test instruction of a user is received, so as to obtain the test result of the test model and the test result of the currently online model. And under the condition that the test result of the test model is higher than that of the currently online model, releasing the memory of the currently online model, and updating the state of the test model to be online. By the aid of the scheme, an information synchronization mechanism is not needed in the training process and the online process of the model, manual participation is not needed, and the online labor cost of model training can be effectively reduced.
Description
Technical Field
The application relates to the field of intelligent customer service dialogue systems, in particular to a model training online method, a model training online device, a model training online storage medium and model training online equipment.
Background
In an intelligent customer service dialog system, a variety of models are generally adopted, and the more common models include text classification, semantic matching, entity recognition and the like. With the development of the natural-language processing (NLP) algorithm, the above models gradually evolve from the initial statistical learning to the deep learning algorithm. The performance requirements of the models for the machines are higher and higher during training, and the common Central Processing Unit (CPU) machines cannot meet the requirements of deep learning, so that many large companies start purchasing machines of GPUs and TPUs for training the deep learning models.
If a large-structure model is adopted in the production environment, Graphics Processing Unit (GPU) and Tensor Processor (TPU) resources are required for training. Generally, the machines of the GPU are mainly offline machines, and since many enterprises isolate the production environment, the machines of the production environment cannot acquire data of the offline service, and the offline service cannot acquire data of the production environment. Therefore, there is a need for a method of interacting a production environment with an off-line GPU machine to achieve model training on-line. For this way, it is common practice to train and online isolate the model, obtain the final model by manually debugging the model continuously, and then perform AB test on the final model and the model currently in use to determine whether to bring the final model online.
Obviously, an information synchronization mechanism is needed in the training process and the online process of the final model, however, most of the information synchronization mechanisms are manually completed by manpower, the complexity of the production environment is improved by the information synchronization mechanism, a large amount of computing resources are consumed in the AB test process, and an online process needs to be additionally developed for different types of models and services, so that the labor cost is greatly increased.
Disclosure of Invention
The application provides a model training online method, a model training online device, a storage medium and model training online equipment, and aims to reduce the labor cost of model training online.
In order to achieve the above object, the present application provides the following technical solutions:
a model training online method comprises the following steps:
under the condition of receiving a test version number input by a user, identifying a model to be online corresponding to the test version number as a test model, and identifying an evaluation value of the test model as a test evaluation value; the model to be on-line and the evaluation value of the model to be on-line are obtained by training the model to be trained;
starting a preset algorithm service to perform AB test on the test model and the currently online model under the condition that the test evaluation value is greater than a target evaluation value and a test instruction of a user is received, so as to obtain the test result of the test model and the test result of the currently online model; the target evaluation value is obtained by training a current online model by using a test set;
and under the condition that the test result of the test model is higher than that of the currently online model, releasing the memory of the currently online model, and updating the state of the test model to be online.
Optionally, the model to be online and the evaluation value of the model to be online are obtained by training the model to be trained, including:
creating a training task of the model to be trained based on the type and the parameters of the model to be trained, and storing a training corpus of the model to be trained to a cloud end;
sending the training task to an off-line graphic processor, triggering the off-line graphic processor to acquire the training corpus from the cloud end, and training the model to be trained according to the type to be trained, the parameters and the training corpus to obtain a model to be on-line and an evaluation value of the model to be on-line; the evaluation value is obtained by training the model to be online by utilizing a test set;
and setting a state label and a version number for the model to be online, and storing the information of the model to be online into a preset model table.
Optionally, the creating a training task of the model to be trained based on the type and the parameter of the model to be trained includes:
obtaining the type, parameters and training corpora of a model to be trained;
setting a state label for the model to be trained, and storing the training corpus of the model to be trained to a cloud end;
storing the type, the parameters and the state labels of the model to be trained in a database;
and under the condition of receiving an access request sent by an off-line graphic processor, creating a training task of the model to be trained based on the type and the parameters of the model to be trained, and updating the state of the model to be trained into training.
Optionally, the setting a state tag and a version number for the model to be online, and storing the information of the model to be online into a preset model table, includes:
under the condition of receiving the model to be online and the evaluation value sent by the off-line graphic processor, setting a version number for the model to be online;
storing the model to be on-line and the evaluation value into a database, and updating the state of the model to be trained;
setting a state label for the model to be online, and storing the information of the model to be online into a preset model table;
and storing the model to be online to a cloud end, and displaying the model to be online and the evaluation value to a user through a preset front end interface.
Optionally, under the condition that the test version number input by the user is received, identifying the model to be on-line corresponding to the test version number as the test model, and identifying the evaluation value of the test model as the test evaluation value, includes:
storing the model to be online and the evaluation value of the model to be online, which are sent by the offline graphics processor, into a database in advance;
under the condition of receiving a test version number input by a user, selecting a model to be on-line corresponding to the test version number from the database, identifying the model as a test model, and identifying the evaluation value of the test model as a test evaluation value;
and updating the state of the test model to be tested.
Optionally, when the test evaluation value is greater than the target evaluation value and a test instruction of the user is received, starting a preset algorithm service to perform AB test on the test model and the currently online model to obtain a test result of the test model and a test result of the currently online model, including:
comparing the test evaluation value with a target evaluation value;
under the condition that the test evaluation value is larger than the target evaluation value, judging whether the model on line currently is the test model or not;
if the model on line currently is the test model, prompting a user that the test model is in an on-line state, and updating the state of the test model to be on line;
and if the currently online model is not the test model, starting a preset algorithm service to perform AB test on the test model and the currently online model under the condition of receiving the test instruction of the user to obtain the test result of the test model and the test result of the currently online model.
Optionally, the method further includes:
and under the condition that the test result of the test model is lower than that of the currently online model, releasing the memory of the test model, and deleting the information of the test model from the model table.
A model training online device, comprising:
the identification unit is used for identifying a model to be on-line corresponding to a test version number as a test model and identifying an evaluation value of the test model as a test evaluation value under the condition of receiving the test version number input by a user; the model to be on-line and the evaluation value of the model to be on-line are obtained by training the model to be trained;
the test unit is used for starting a preset algorithm service to carry out AB test on the test model and the currently online model under the condition that the test evaluation value is greater than a target evaluation value and a test instruction of a user is received, so as to obtain the test result of the test model and the test result of the currently online model; the target evaluation value is obtained by training a current online model by using a test set;
and the online unit is used for releasing the memory of the currently online model and updating the state of the test model to be online under the condition that the test result of the test model is higher than that of the currently online model.
A computer-readable storage medium comprising a stored program, wherein the program performs the model training online method.
A model training online device, comprising: a processor, a memory, and a bus; the processor and the memory are connected through the bus;
the memory is used for storing a program, and the processor is used for executing the program, wherein the program executes the model training online method during running.
According to the technical scheme, under the condition that the test version number input by the user is received, the model to be on-line corresponding to the test version number is identified as the test model, and the evaluation value of the test model is identified as the test evaluation value. And starting a preset algorithm service to carry out AB test on the test model and the currently online model under the condition that the test evaluation value is greater than the target evaluation value and a test instruction of a user is received, so as to obtain the test result of the test model and the test result of the currently online model. The target evaluation value is obtained based on training a model which is currently online by using a test set. And under the condition that the test result of the test model is higher than that of the currently online model, releasing the memory of the currently online model, and updating the state of the test model to be online. By the aid of the scheme, an information synchronization mechanism is not needed in the training process and the online process of the model, and manual participation is not needed, so that the online labor cost of model training is effectively reduced.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic diagram of an online model training method according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of another model training online method provided in an embodiment of the present application;
fig. 3 is a schematic diagram illustrating an architecture of a model training online device according to an embodiment of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
As shown in fig. 1, a schematic diagram of an online model training method provided in an embodiment of the present application is applied to a production environment, and includes the following steps:
s101: and obtaining the type, the parameters and the training corpora of the model to be trained.
Wherein the parameters include, but are not limited to: training algorithm, training task, maximum length of training corpus, whether to splice the above information, etc.
S102: and setting a state label for the model to be trained, and storing the training corpus of the model to be trained to the cloud.
The training process of the model to be trained is performed in an offline environment (i.e., offline graphics processor), that is, the offline graphics processor cannot directly access the database. Therefore, the corpus of the model to be trained needs to be stored in the cloud end so as to be conveniently acquired by the offline graphics processor.
S103: and storing the type, the parameters and the state labels of the model to be trained in a database.
S104: and under the condition of receiving an access request sent by the off-line graphic processor, creating a training task of the model to be trained based on the type and the parameters of the model to be trained, and updating the state of the model to be trained into training.
The off-line graphics processor can send an access request through a preset http interface. In addition, a timing access service is pre-deployed on the off-line graphics processor, so that the off-line graphics processor can be controlled to send access requests at regular time.
S105: and sending the training task to an offline graphics processor, triggering the offline graphics processor to acquire a training corpus of the model to be trained from the cloud, and training the model to be trained according to the type, the parameters and the training corpus of the model to be trained to obtain a model file and a result file.
The model file comprises a model to be online obtained through training of the model to be trained, and the result file comprises an evaluation value used for evaluating the training effect of the model to be online, wherein the evaluation value is obtained through training of the model to be online by utilizing a test set.
It should be noted that, usually, the offline graphics processor pre-configures a resource table, where the resource table is used to record the usage state of its own computing resource, before executing the training task, the offline graphics processor queries the usage state of each computing resource in the resource table, and if there is a computing resource whose usage state is idle, identifies the computing resource whose usage state is idle as a target resource, and executes the training task by using the target resource.
Generally speaking, an offline graphics processor may implement a training process of a model to be trained by calling a pre-deployed training process, specifically, configure the type, parameters, and training corpora of the model to be trained in a code shown in the training process, and run the training process to generate a model file and a result file.
In addition, during the course of performing a training task with a target resource, the offline graphics processing will also identify the use status of the target resource as in use. After obtaining the model file and the result file, i.e., after the training task is completed, the offline graphics processor will also identify the usage status of the target resource as idle.
S106: and under the condition of receiving the model to be online and the evaluation value sent by the offline graphics processor, setting a version number for the model to be online.
S107: and storing the model to be on-line and the evaluation value into a database, and updating the state of the model to be trained to be training completion.
S108: and setting a state label for the model to be online, and storing the information of the model to be online into a preset model table.
S109: and storing the model to be online to the cloud, and displaying the model to be online and the evaluation value to a user through a preset front-end interface.
It should be noted that the flow shown in S101-S109 mainly explains that the production environment issues a training task of the model to be trained to the offline graphics processor, and after the offline graphics processor completes the training task, the model file and the result file obtained by training are fed back to the production environment, so as to implement automatic training of the model to be trained. Compared with the prior art, the training process of the model to be trained does not need manual participation, and the online labor cost of model training is effectively reduced.
S110: and under the condition of receiving the test version number input by the user, selecting the model to be on-line corresponding to the test version number from the database, and identifying the model as the test model.
S111: and identifying the evaluation value of the test model as a test evaluation value.
S112: and updating the state of the test model to be tested.
S113: the test evaluation value is compared with the target evaluation value.
The target evaluation value is obtained by training a model on line currently by using a test set. In the embodiment of the present application, the target evaluation value is used to evaluate the training effect of the model currently on-line.
S114: and under the condition that the test evaluation value is larger than the target evaluation value, judging whether the model on line currently is a test model or not.
If the model on line currently is the test model, executing S115, otherwise executing S116.
In order to avoid redundant database operation caused by repeated online of the test model, it is necessary to determine whether the currently online model is the test model, and if the currently online model is the test model, the currently online model represents that the test model is in an online state, and online operation is not required.
S115: and prompting a user that the test model is in an online state, and updating the state of the test model to be online.
S116: and under the condition of receiving a test instruction of a user, starting a preset algorithm service to perform AB test on the test model and the currently online model to obtain the test result of the test model and the test result of the currently online model.
After execution of S116, execution continues with S117.
Wherein the algorithmic service is to perform the following logic:
1. and inquiring the model table at regular time, and judging whether the model table contains the information of the test model.
2. And under the condition that the model table contains the information of the test model, loading the test model obtained from the cloud, adding first embedded point information into the current online model, and adding second embedded point information into the test model.
3. And shunting the flow of the current service according to the message id to obtain a first flow and a second flow, calling the current online model by the first flow, and calling the test model by the second flow.
4. And associating information shown by different message ids through the first embedded point information to obtain a service index of the first flow, and taking the value of the service index of the first flow as the test result of the currently online model.
5. And associating information shown by different message ids through the second embedded point information to obtain a service index of the second flow, and taking the value of the service index of the second flow as the test result of the test model.
6. And sending a prompt of AB test failure to the user when the model table does not contain the information of the test model.
S117: and comparing the test result of the test model with the test result of the currently online model.
S118: and under the condition that the test result of the test model is higher than that of the currently online model, releasing the memory of the currently online model, and updating the state of the test model to be online.
S119: and under the condition that the test result of the test model is lower than that of the currently online model, releasing the memory of the test model, and deleting the information of the test model from the model table.
S120: and under the condition that the test model is detected to be a new model, loading the test model, and updating the state of the test model to be on-line.
In summary, by using the scheme shown in this embodiment, both the training process and the online process of the model do not need to adopt an information synchronization mechanism, and do not need to participate in the manual work, thereby effectively reducing the labor cost of model training online.
It should be noted that, in the above embodiment, reference is made to S119, which is an alternative implementation manner of the online training method for the model shown in this application. In addition, S120 mentioned in the above embodiment is also an alternative implementation of the online training method for the model shown in this application. For this reason, the flow mentioned in the above embodiment can be summarized as the method shown in fig. 2.
As shown in fig. 2, a schematic diagram of another model training online method provided in the embodiment of the present application includes the following steps:
s201: under the condition that a test version number input by a user is received, a model to be on-line corresponding to the test version number is marked as a test model, and an evaluation value of the test model is marked as a test evaluation value.
The model to be on-line and the evaluation value of the model to be on-line are obtained by training the model to be trained.
S202: and starting a preset algorithm service to carry out AB test on the test model and the currently online model under the condition that the test evaluation value is greater than the target evaluation value and a test instruction of a user is received, so as to obtain the test result of the test model and the test result of the currently online model.
The target evaluation value is obtained by training a model on line currently by using a test set.
S203: and under the condition that the test result of the test model is higher than that of the currently online model, releasing the memory of the currently online model, and updating the state of the test model to be online.
In summary, by using the scheme shown in this embodiment, both the training process and the online process of the model do not need to adopt an information synchronization mechanism, and do not need to participate in the manual work, thereby effectively reducing the labor cost of model training online.
Corresponding to the model training online method provided by the embodiment of the application, the embodiment of the application also provides a model training online device.
As shown in fig. 3, an architecture diagram of a model training online device provided in the embodiment of the present application includes:
an identifying unit 100 for identifying a model to be brought online corresponding to the test version number as a test model and identifying an evaluation value of the test model as a test evaluation value in case of receiving the test version number input by the user.
The identification unit 100 is specifically configured to: establishing a training task of the model to be trained based on the type and the parameters of the model to be trained, and storing a training corpus of the model to be trained to a cloud end; sending the training task to an off-line graphic processor, triggering the off-line graphic processor to acquire a training corpus from a cloud end, and training a model to be trained according to the type, the parameters and the training corpus to obtain a model to be on-line and an evaluation value of the model to be on-line; the evaluation value is obtained by training the model to be online by utilizing a test set; and setting a state label and a version number for the model to be online, and storing the information of the model to be online into a preset model table.
The identification unit 100 is specifically configured to: obtaining the type, parameters and training corpora of a model to be trained; setting a state label for the model to be trained, and storing the training corpus of the model to be trained to a cloud end; storing the type, the parameters and the state labels of the model to be trained in a database; and under the condition of receiving an access request sent by the off-line graphic processor, creating a training task of the model to be trained based on the type and the parameters of the model to be trained, and updating the state of the model to be trained into training.
The identification unit 100 is specifically configured to: setting a version number for the model to be online under the condition of receiving the model to be online and the evaluation value sent by the offline graphics processor; storing the model to be on-line and the evaluation value into a database, and updating the state of the model to be trained to be training completion; setting a state label for the model to be online, and storing the information of the model to be online into a preset model table; and storing the model to be online to the cloud, and displaying the model to be online and the evaluation value to a user through a preset front-end interface.
The identification unit 100 is specifically configured to: storing the model to be online and the evaluation value of the model to be online, which are sent by an offline graphic processor, into a database in advance; under the condition of receiving a test version number input by a user, selecting a model to be on-line corresponding to the test version number from a database, identifying the model as a test model, and identifying the evaluation value of the test model as a test evaluation value; and updating the state of the test model to be tested.
The test unit 200 is configured to start a preset algorithm service to perform AB test on the test model and the currently online model to obtain a test result of the test model and a test result of the currently online model when the test evaluation value is greater than the target evaluation value and a test instruction of the user is received; the target evaluation value is obtained based on training a model which is currently online by using a test set.
Wherein, the test unit 200 is specifically configured to: comparing the test evaluation value with a target evaluation value; under the condition that the test evaluation value is larger than the target evaluation value, judging whether the model on line currently is a test model or not; if the current online model is the test model, prompting a user that the test model is in an online state, and updating the state of the test model to be online; and if the currently online model is not the test model, starting a preset algorithm service to perform AB test on the test model and the currently online model under the condition of receiving a test instruction of a user to obtain the test result of the test model and the test result of the currently online model.
And an online unit 300, configured to release the memory of the currently online model and update the state of the test model to be online when the test result of the test model is higher than the test result of the currently online model.
And the deleting unit 400 is configured to release the memory of the test model and delete the information of the test model from the model table when the test result of the test model is lower than the test result of the currently online model.
In summary, by using the scheme shown in this embodiment, both the training process and the online process of the model do not need to adopt an information synchronization mechanism, and do not need to participate in the manual work, thereby effectively reducing the labor cost of model training online.
The present application also provides a computer-readable storage medium comprising a stored program, wherein the program performs the model training online method provided herein above.
The application also provides an online equipment for model training, including: a processor, a memory, and a bus. The processor is connected with the memory through a bus, the memory is used for storing programs, and the processor is used for running the programs, wherein when the programs run, the model training online method provided by the application is executed, and the method comprises the following steps:
under the condition of receiving a test version number input by a user, identifying a model to be online corresponding to the test version number as a test model, and identifying an evaluation value of the test model as a test evaluation value; the model to be on-line and the evaluation value of the model to be on-line are obtained by training the model to be trained;
starting a preset algorithm service to perform AB test on the test model and the currently online model under the condition that the test evaluation value is greater than a target evaluation value and a test instruction of a user is received, so as to obtain the test result of the test model and the test result of the currently online model; the target evaluation value is obtained by training a current online model by using a test set;
and under the condition that the test result of the test model is higher than that of the currently online model, releasing the memory of the currently online model, and updating the state of the test model to be online.
Optionally, the model to be online and the evaluation value of the model to be online are obtained by training the model to be trained, including:
creating a training task of the model to be trained based on the type and the parameters of the model to be trained, and storing a training corpus of the model to be trained to a cloud end;
sending the training task to an off-line graphic processor, triggering the off-line graphic processor to acquire the training corpus from the cloud end, and training the model to be trained according to the type to be trained, the parameters and the training corpus to obtain a model to be on-line and an evaluation value of the model to be on-line; the evaluation value is obtained by training the model to be online by utilizing a test set;
and setting a state label and a version number for the model to be online, and storing the information of the model to be online into a preset model table.
Optionally, the creating a training task of the model to be trained based on the type and the parameter of the model to be trained includes:
obtaining the type, parameters and training corpora of a model to be trained;
setting a state label for the model to be trained, and storing the training corpus of the model to be trained to a cloud end;
storing the type, the parameters and the state labels of the model to be trained in a database;
and under the condition of receiving an access request sent by an off-line graphic processor, creating a training task of the model to be trained based on the type and the parameters of the model to be trained, and updating the state of the model to be trained into training.
Optionally, the setting a state tag and a version number for the model to be online, and storing the information of the model to be online into a preset model table, includes:
under the condition of receiving the model to be online and the evaluation value sent by the off-line graphic processor, setting a version number for the model to be online;
storing the model to be on-line and the evaluation value into a database, and updating the state of the model to be trained;
setting a state label for the model to be online, and storing the information of the model to be online into a preset model table;
and storing the model to be online to a cloud end, and displaying the model to be online and the evaluation value to a user through a preset front end interface.
Optionally, under the condition that the test version number input by the user is received, identifying the model to be on-line corresponding to the test version number as the test model, and identifying the evaluation value of the test model as the test evaluation value, includes:
storing the model to be online and the evaluation value of the model to be online, which are sent by the offline graphics processor, into a database in advance;
under the condition of receiving a test version number input by a user, selecting a model to be on-line corresponding to the test version number from the database, identifying the model as a test model, and identifying the evaluation value of the test model as a test evaluation value;
and updating the state of the test model to be tested.
Optionally, when the test evaluation value is greater than the target evaluation value and a test instruction of the user is received, starting a preset algorithm service to perform AB test on the test model and the currently online model to obtain a test result of the test model and a test result of the currently online model, including:
comparing the test evaluation value with a target evaluation value;
under the condition that the test evaluation value is larger than the target evaluation value, judging whether the model on line currently is the test model or not;
if the model on line currently is the test model, prompting a user that the test model is in an on-line state, and updating the state of the test model to be on line;
and if the currently online model is not the test model, starting a preset algorithm service to perform AB test on the test model and the currently online model under the condition of receiving the test instruction of the user to obtain the test result of the test model and the test result of the currently online model.
Optionally, the method further includes:
and under the condition that the test result of the test model is lower than that of the currently online model, releasing the memory of the test model, and deleting the information of the test model from the model table.
The functions described in the method of the embodiment of the present application, if implemented in the form of software functional units and sold or used as independent products, may be stored in a storage medium readable by a computing device. Based on such understanding, part of the contribution to the prior art of the embodiments of the present application or part of the technical solution may be embodied in the form of a software product stored in a storage medium and including several instructions for causing a computing device (which may be a personal computer, a server, a mobile computing device or a network device) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111175935.XA CN113919506B (en) | 2021-10-09 | 2021-10-09 | A model training online method, device, storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111175935.XA CN113919506B (en) | 2021-10-09 | 2021-10-09 | A model training online method, device, storage medium and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113919506A true CN113919506A (en) | 2022-01-11 |
| CN113919506B CN113919506B (en) | 2024-12-24 |
Family
ID=79238720
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111175935.XA Active CN113919506B (en) | 2021-10-09 | 2021-10-09 | A model training online method, device, storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113919506B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113722394A (en) * | 2021-08-17 | 2021-11-30 | 北京百悟科技有限公司 | Data synchronization method, device and storage medium |
| WO2024251168A1 (en) * | 2023-06-05 | 2024-12-12 | 阿里巴巴（中国）有限公司 | Data processing method based on large model, and server |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3607466A1 (en) * | 2017-04-07 | 2020-02-12 | INTEL Corporation | Advanced artificial intelligence agent for modeling physical interactions |
| CN112288092A (en) * | 2019-07-23 | 2021-01-29 | 百度时代网络技术（北京）有限公司 | Model evaluation method, model evaluation device, electronic device and storage medium |
| CN113065843A (en) * | 2021-03-15 | 2021-07-02 | 腾讯科技（深圳）有限公司 | Model processing method and device, electronic equipment and storage medium |
| WO2021135552A1 (en) * | 2020-06-28 | 2021-07-08 | 平安科技（深圳）有限公司 | Segmentation effect assessment method and apparatus based on deep learning, and device and medium |
-
2021
- 2021-10-09 CN CN202111175935.XA patent/CN113919506B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3607466A1 (en) * | 2017-04-07 | 2020-02-12 | INTEL Corporation | Advanced artificial intelligence agent for modeling physical interactions |
| CN112288092A (en) * | 2019-07-23 | 2021-01-29 | 百度时代网络技术（北京）有限公司 | Model evaluation method, model evaluation device, electronic device and storage medium |
| WO2021135552A1 (en) * | 2020-06-28 | 2021-07-08 | 平安科技（深圳）有限公司 | Segmentation effect assessment method and apparatus based on deep learning, and device and medium |
| CN113065843A (en) * | 2021-03-15 | 2021-07-02 | 腾讯科技（深圳）有限公司 | Model processing method and device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 徐刚;白璐;赵德方;张瑜;: "GBRT技术在MLS模拟训练成绩评估中应用", 计算机仿真, no. 04, 15 April 2020 (2020-04-15) * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113722394A (en) * | 2021-08-17 | 2021-11-30 | 北京百悟科技有限公司 | Data synchronization method, device and storage medium |
| WO2024251168A1 (en) * | 2023-06-05 | 2024-12-12 | 阿里巴巴（中国）有限公司 | Data processing method based on large model, and server |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113919506B (en) | 2024-12-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11100475B2 (en) | Document tracking and correlation | |
| US11011173B2 (en) | Interacting with a user device to provide automated testing of a customer service representative | |
| US11232508B2 (en) | Quote-to-cash intelligent software agent | |
| US11361337B2 (en) | Intelligent case management platform | |
| CN113919506B (en) | A model training online method, device, storage medium and equipment | |
| CN106943747B (en) | Virtual role name recommendation method and device, electronic equipment and storage medium | |
| CN112885376A (en) | Method and device for improving voice call quality inspection effect | |
| CN106547598A (en) | The method for lifting Android application program launching speed | |
| CN108345639B (en) | Task management system and task management method | |
| CN110941519B (en) | Chip testing method and device of network equipment | |
| CN110287318B (en) | Service operation detection method and device, storage medium and electronic device | |
| CN110991871A (en) | Risk monitoring method, device, equipment and computer readable storage medium | |
| CN113051178A (en) | Test data construction method, device, equipment and medium | |
| US11463328B2 (en) | Training a machine learning algorithm to create survey questions | |
| CN114202363B (en) | Artificial intelligence-based call method, device, computer equipment and medium | |
| CN111754189A (en) | Task delivery method and device, computer equipment and computer readable storage medium | |
| CN112151034B (en) | Voice control method and device of equipment, electronic equipment and storage medium | |
| CN110209565A (en) | A kind of metadata schema adjustment method and its device | |
| CN109165327B (en) | Man-machine conversation method, device and computer readable storage medium | |
| CN110674275A (en) | Knowledge question answering method and device | |
| CN115273858A (en) | A text processing method, device, equipment and medium | |
| CN114372479A (en) | Text recognition method and device, electronic equipment and computer readable storage medium | |
| CN107368304A (en) | Creation method, terminal and the storage medium of desktop shortcuts | |
| CN110826339B (en) | Behavior recognition method, behavior recognition device, electronic equipment and medium | |
| CN114186046A (en) | Information processing method, information processing apparatus, server, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |